anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
127 Commits 235 Branches 31 Tags
6c6ff9f27c5cccd40bfdbbfee5b0805d64fe7e05
Commit Graph
85 Commits
Author SHA1 Message Date
tzhouam 6c6ff9f27c Enhance error handling and logging in pipeline and stage processing. Added detailed logging for prompt loading failures, file writing issues, and stage shutdown processes. Introduced a new logging utility for orchestrator metrics and streamlined stats handling in the PipelinedOmniLLM class. 
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2025-11-14 18:44:28 +08:00
tzhouam b8b82347d3 debug ar model runner 
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2025-11-14 18:44:28 +08:00
tzhouam 62b0e81795 Port changes from feat/multi-request-stream (vs main) 
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2025-11-14 18:44:28 +08:00
齐保元 8f7b102108 [fix] support loading model from local path 
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2025-11-14 18:44:28 +08:00
齐保元 5c038ae553 [feature] cpu worker support 
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2025-11-14 18:44:28 +08:00
齐保元 8b30818f2e [feature] cpu worker support 
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2025-11-14 18:44:28 +08:00
Gaohan123 e5adae0a07 fix other reviews 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-06 19:25:55 +08:00
Gaohan123 1aa5ae61cb fix reviews 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-05 23:26:58 +08:00
Gaohan123 e2b803fbe6 fix a bug 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-05 13:05:12 +08:00
Gaohan123 6e1478222e Dependency refractored to vLLM v0.11.0 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-04 22:06:41 +08:00
Gaohan123 d4a364f052 fix huggingface download problem for spk_dict.pt 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-01 16:45:34 +08:00
Gaohan123 6cf73c4e80 Remove unused files and fix some bugs 2025-10-25 18:02:11 +08:00
tzhouam f6158e7576 debug 2025-10-24 18:01:02 +08:00
tzhouam 605d1de013 Update entry points in vllm_omni module 
- Reintroduced AsyncOmniLLM to the main entry points.
- Added import for patch module.
- Cleaned up comments regarding AsyncOmniLLM support.
 2025-10-24 17:54:19 +08:00
tzhouam d45f8a6be7 debug typo 2025-10-24 17:47:49 +08:00
tzhouam 61b344dc39 Add import for OmniModelsRegistry in model executor initialization 2025-10-24 17:44:31 +08:00
tzhouam 3b1324ea39 Merge branch 'main' into end2end_fixup 2025-10-24 17:41:03 +08:00
tzhouam 3265e857eb Refactor OmniLLM and MultimodalOutputProcessor initialization 
- Removed AsyncOmniLLM from the main entry points as it is not supported yet.
- Updated the constructor of MultimodalOutputProcessor to set a default value for engine_core_output_type.
- Adjusted output type handling in various methods to utilize the new default engine_core_output_type for consistency.
 2025-10-24 17:15:34 +08:00
Gaohan123 171f3c1ea7 fix conflicts 2025-10-24 16:49:19 +08:00
Gaohan123 22f36de34e fix error 2025-10-24 16:21:52 +08:00
Gaohan123 eafc7c44c3 fix error for pre-commit 2025-10-24 16:21:50 +08:00
Gaohan123 a04e945166 add precommit 2025-10-24 16:20:58 +08:00
Hongsheng Liu 1bf3424e96 Merge pull request #19 from tzhouam/feat/omni-schedulers 
[Core]Add scheduling components for vLLM-omni
 2025-10-24 14:08:57 +08:00
tzhouam eb006f66e5 Enhance OmniScheduler error handling with logging 
- Added logging for exceptions when wrapping scheduled requests in OmniScheduler.
- Removed unnecessary blank lines for cleaner code structure.
 2025-10-24 12:00:36 +08:00
tzhouam 87b3ae7e70 Merge branch 'feat/omni-schedulers' of https://github.com/tzhouam/vllm-omni into feat/omni-schedulers 2025-10-24 11:57:45 +08:00
tzhouam c23d24ca19 Refactor comments in diffusion_scheduler.py for clarity and consistency 
- Translated comments from Chinese to English for better accessibility.
- Improved clarity of comments related to request handling and scheduling logic.
- Ensured comments align with the code functionality and maintain a uniform style.
 2025-10-24 11:57:39 +08:00
Zhou Taichang 582639c77a Merge branch 'main' into feat/hidden-state-input-request-input-processor-patch 
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
 2025-10-24 11:52:45 +08:00
tzhouam 63c72f4c2b Merge branch 'feat/hidden-state-input-request-input-processor-patch' of https://github.com/tzhouam/vllm-omni into feat/hidden-state-input-request-input-processor-patch 2025-10-24 11:40:49 +08:00
tzhouam a97409efef Add dtype mapping method in OmniProcessor 
- Introduced a static method `_dtype_to_name` to map PyTorch data types to their string representations.
- Updated the `dtype_str` assignments in the `OmniProcessor` class to utilize the new mapping method for improved readability and maintainability.
 2025-10-24 11:40:43 +08:00
tzhouam 4cd3b36fa0 Refactor OmniProcessor and update parsing logic 
- Removed redundant imports of `torch` and `numpy` in `OmniProcessor` class.
- Improved comments in `parse_singleton_prompt_omni` function for clarity on token handling.
- Ensured `current_platform` is imported only once for better code organization.
 2025-10-24 11:27:30 +08:00
Hongsheng Liu 4a4c3c1b82 Merge pull request #20 from tzhouam/feat/multimodal-output-processor 
[Engine]Refactor output processing for multimodal capabilities in vLLM-omni
 2025-10-24 11:25:23 +08:00
Hongsheng Liu 40d20978f1 Merge pull request #17 from tzhouam/feat/gpu-omni-model-runner-base 
[Worker]Add OmniGPUModelRunner and OmniModelInputForGPU classes
 2025-10-24 11:23:19 +08:00
Zhou TaichangandCopilot d0423129f8 Apply suggestion from @Copilot 
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
 2025-10-24 11:14:58 +08:00
Zhou TaichangandCopilot b04f114773 Apply suggestion from @Copilot 
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
 2025-10-24 11:14:41 +08:00
Hongsheng Liu 27fbe3511a Merge pull request #16 from tzhouam/feat/diffusion_gpu_worker_and_model_runner 
[Worker]Refactor GPU diffusion model runner and worker
 2025-10-24 11:11:02 +08:00
tzhouam ee4822327b Remove unnecessary numpy imports and update return type in extract_multimodal_outputs method in OmniGPUModelRunner 
- Eliminated redundant numpy import statements to streamline the code.
- Changed the return type of extract_multimodal_outputs from dict to tuple for better clarity on output structure.
 2025-10-24 11:08:42 +08:00
Hongsheng Liu a874cd23fb Merge pull request #15 from tzhouam/feat/ar_gpu_worker_and_model_runner 
[Worker]Feat/ar gpu worker and model runner
 2025-10-24 11:04:30 +08:00
tzhouam 6b196dee24 Enhance error handling in output processing for multimodal tensors 
- Added logging for exceptions during tensor movement to CPU in OmniRequestState and MultimodalOutputProcessor.
- Improved robustness by ensuring the output pipeline continues without crashing on errors.
- Updated comments for clarity on error handling behavior.
 2025-10-24 11:01:32 +08:00
Zhou TaichangandCopilot 9ebc1dd652 Apply suggestion from @Copilot 
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
 2025-10-24 10:41:54 +08:00
Hongsheng Liu 15dbe97750 Merge pull request #13 from Gaohan123/entrypoint_stage 
[Engine] Add entrypoint class and stage management
 2025-10-24 10:38:16 +08:00
tzhouam 634915652d Remove unused code and comments in GPUDiffusionModelRunner for improved clarity 
- Deleted the `has_kv_transfer_group` import as it was not utilized.
- Cleaned up commented-out code related to model methods, focusing on the forward method.
- Added a TODO comment to implement the diffuse method for additional model support.
 2025-10-24 10:34:41 +08:00
Gaohan123 61a215fb54 fix file name 2025-10-24 10:19:35 +08:00
Gaohan123 2c08c04730 fix for reviews of copilot 2025-10-24 10:17:22 +08:00
tzhouam ae3b7504b2 Merge branch 'feat/qwen_2_5_omni_models' of https://github.com/tzhouam/vllm-omni into feat/qwen_2_5_omni_models 2025-10-24 10:02:58 +08:00
tzhouam 52905a0d17 Enhance Qwen2.5-Omni model by defining TALKER_CODEC_EOS_TOKEN_ID and TALKER_CODEC_BOS_TOKEN_ID constants. Update audio generation logic to use these constants for improved clarity and maintainability. Refactor tensor initialization to utilize the talker's hidden size. Update comments for better understanding and consistency. 2025-10-24 10:02:51 +08:00
Zhou TaichangandCopilot 8a6e298243 Apply suggestion from @Copilot 
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
 2025-10-24 09:43:45 +08:00
tzhouam 4acaef3ccb Merge branch 'feat/omni-schedulers' of https://github.com/tzhouam/vllm-omni into feat/omni-schedulers 2025-10-24 09:41:29 +08:00
tzhouam c37cd6f0e3 delete unused imports 2025-10-24 09:41:20 +08:00
tzhouam 28ae542eed match runner, worker name with file name 2025-10-23 14:35:23 +08:00
tzhouam a3a77b6cc4 Qwen2.5-Omni: squash 5 commits (add components, refactor, load thinker embedding from weights, remove unused code) 
Squashed commits:\n- 4e2d812 Add Qwen2.5-Omni model components\n- 351f833 load the thinker embedding from the weights, not rely on local .pt files\n- 2c32533 Refactor Qwen2.5-Omni model files\n- 9163a60 remove unused codes\n- 192bdf8 remove
 2025-10-23 14:24:03 +08:00