anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
127 Commits 235 Branches 31 Tags
6c6ff9f27c5cccd40bfdbbfee5b0805d64fe7e05
Commit Graph
127 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
tzhouam 6c6ff9f27c Enhance error handling and logging in pipeline and stage processing. Added detailed logging for prompt loading failures, file writing issues, and stage shutdown processes. Introduced a new logging utility for orchestrator metrics and streamlined stats handling in the PipelinedOmniLLM class. 
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2025-11-14 18:44:28 +08:00
tzhouam b8b82347d3 debug ar model runner 
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2025-11-14 18:44:28 +08:00
tzhouam 62b0e81795 Port changes from feat/multi-request-stream (vs main) 
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2025-11-14 18:44:28 +08:00
齐保元 8f7b102108 [fix] support loading model from local path 
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2025-11-14 18:44:28 +08:00
齐保元 5c038ae553 [feature] cpu worker support 
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2025-11-14 18:44:28 +08:00
齐保元 8b30818f2e [feature] cpu worker support 
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2025-11-14 18:44:28 +08:00
Hongsheng Liu 86841b69e9 Merge pull request #48 from Gaohan123/refractor_v2 
[Refractor] Dependency refractored to vLLM v0.11.0
 2025-11-07 15:47:53 +13:00
Gaohan123 e5adae0a07 fix other reviews 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-06 19:25:55 +08:00
Hongsheng Liu 5cc8ab06db Merge pull request #47 from Gaohan123/refractor 
[Bugfix] fix huggingface download problem for spk_dict.pt
 2025-11-06 16:03:31 +13:00
Gaohan123 1aa5ae61cb fix reviews 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-05 23:26:58 +08:00
Gaohan123 e2b803fbe6 fix a bug 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-05 13:05:12 +08:00
Gaohan123 c80348ce71 supplement some new infomation 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-05 10:47:30 +08:00
Gaohan123 5b683c75de supplement some infomation 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-05 10:46:05 +08:00
Gaohan123 6e1478222e Dependency refractored to vLLM v0.11.0 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-04 22:06:41 +08:00
Gaohan123 d4a364f052 fix huggingface download problem for spk_dict.pt 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-01 16:45:34 +08:00
Zhou Taichang 6d34a3ed11 Merge pull request #46 from Gaohan123/fix_install_v2 
[Bugfix] Further supplement installation guide
 2025-10-30 17:25:14 +08:00
Gaohan123 03ae52a064 add installation of uv 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-10-30 17:18:19 +08:00
Gaohan123 1702033c3e fix some typos 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-10-30 17:16:25 +08:00
Gaohan123 27f06e27fc Further supplement installation guide 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-10-30 14:45:54 +08:00
Hongsheng Liu aa2aa52a82 Merge pull request #44 from Gaohan123/fix_install 
[bugfix] fix problem of installation
 2025-10-27 22:11:33 +08:00
Gaohan123 78830c8ad9 fix problem of installation 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-10-27 22:06:35 +08:00
Hongsheng Liu 0b56cd3d5b Merge pull request #36 from Gaohan123/remove_unused 
Remove unused files and fix some bugs
 2025-10-25 23:21:52 +08:00
Gaohan123 c19e4d26d1 fix requirements 2025-10-25 18:02:16 +08:00
Gaohan123 9b6ce0d245 fix typo 2025-10-25 18:02:16 +08:00
Gaohan123 fed8785eb9 fix some docs 2025-10-25 18:02:16 +08:00
Gaohan123 6cf73c4e80 Remove unused files and fix some bugs 2025-10-25 18:02:11 +08:00
Hongsheng Liu 6248bbf646 Merge pull request #35 from tzhouam/end2end_fixup 
End2end fixup
 2025-10-25 17:04:42 +08:00
tzhouam f6158e7576 debug 2025-10-24 18:01:02 +08:00
tzhouam 605d1de013 Update entry points in vllm_omni module 
- Reintroduced AsyncOmniLLM to the main entry points.
- Added import for patch module.
- Cleaned up comments regarding AsyncOmniLLM support.
 2025-10-24 17:54:19 +08:00
tzhouam d45f8a6be7 debug typo 2025-10-24 17:47:49 +08:00
tzhouam 61b344dc39 Add import for OmniModelsRegistry in model executor initialization 2025-10-24 17:44:31 +08:00
tzhouam 3b1324ea39 Merge branch 'main' into end2end_fixup 2025-10-24 17:41:03 +08:00
tzhouam 3265e857eb Refactor OmniLLM and MultimodalOutputProcessor initialization 
- Removed AsyncOmniLLM from the main entry points as it is not supported yet.
- Updated the constructor of MultimodalOutputProcessor to set a default value for engine_core_output_type.
- Adjusted output type handling in various methods to utilize the new default engine_core_output_type for consistency.
 2025-10-24 17:15:34 +08:00
Hongsheng Liu a884dddc89 Merge pull request #32 from Gaohan123/precommit_set 
add precommit and solve conflict
 2025-10-24 17:04:49 +08:00
Gaohan123 171f3c1ea7 fix conflicts 2025-10-24 16:49:19 +08:00
Gaohan123 22f36de34e fix error 2025-10-24 16:21:52 +08:00
Gaohan123 eafc7c44c3 fix error for pre-commit 2025-10-24 16:21:50 +08:00
Gaohan123 a04e945166 add precommit 2025-10-24 16:20:58 +08:00
Hongsheng Liu 1bf3424e96 Merge pull request #19 from tzhouam/feat/omni-schedulers 
[Core]Add scheduling components for vLLM-omni
 2025-10-24 14:08:57 +08:00
Hongsheng Liu 9e79a9a9b1 Merge pull request #18 from tzhouam/feat/hidden-state-input-request-input-processor-patch 
[Inputs, Engine]Add Omni model components and input processing for hidden states support
 2025-10-24 14:05:36 +08:00
tzhouam eb006f66e5 Enhance OmniScheduler error handling with logging 
- Added logging for exceptions when wrapping scheduled requests in OmniScheduler.
- Removed unnecessary blank lines for cleaner code structure.
 2025-10-24 12:00:36 +08:00
tzhouam 87b3ae7e70 Merge branch 'feat/omni-schedulers' of https://github.com/tzhouam/vllm-omni into feat/omni-schedulers 2025-10-24 11:57:45 +08:00
tzhouam c23d24ca19 Refactor comments in diffusion_scheduler.py for clarity and consistency 
- Translated comments from Chinese to English for better accessibility.
- Improved clarity of comments related to request handling and scheduling logic.
- Ensured comments align with the code functionality and maintain a uniform style.
 2025-10-24 11:57:39 +08:00
Zhou Taichang 582639c77a Merge branch 'main' into feat/hidden-state-input-request-input-processor-patch 
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
 2025-10-24 11:52:45 +08:00
tzhouam 63c72f4c2b Merge branch 'feat/hidden-state-input-request-input-processor-patch' of https://github.com/tzhouam/vllm-omni into feat/hidden-state-input-request-input-processor-patch 2025-10-24 11:40:49 +08:00
tzhouam a97409efef Add dtype mapping method in OmniProcessor 
- Introduced a static method `_dtype_to_name` to map PyTorch data types to their string representations.
- Updated the `dtype_str` assignments in the `OmniProcessor` class to utilize the new mapping method for improved readability and maintainability.
 2025-10-24 11:40:43 +08:00
tzhouam 4cd3b36fa0 Refactor OmniProcessor and update parsing logic 
- Removed redundant imports of `torch` and `numpy` in `OmniProcessor` class.
- Improved comments in `parse_singleton_prompt_omni` function for clarity on token handling.
- Ensured `current_platform` is imported only once for better code organization.
 2025-10-24 11:27:30 +08:00
Hongsheng Liu 4a4c3c1b82 Merge pull request #20 from tzhouam/feat/multimodal-output-processor 
[Engine]Refactor output processing for multimodal capabilities in vLLM-omni
 2025-10-24 11:25:23 +08:00
Hongsheng Liu 40d20978f1 Merge pull request #17 from tzhouam/feat/gpu-omni-model-runner-base 
[Worker]Add OmniGPUModelRunner and OmniModelInputForGPU classes
 2025-10-24 11:23:19 +08:00
Zhou TaichangandCopilot d0423129f8 Apply suggestion from @Copilot 
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
 2025-10-24 11:14:58 +08:00