anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
162 Commits 235 Branches 31 Tags
befc7fdfd78df00c2e348b72f37bcb04deb730ba
Commit Graph
100 Commits
Author SHA1 Message Date
Jiangyun Zhu 54351f2ac8 [fix] add scheduler.py (#94) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-11-28 14:45:06 +08:00
Jiangyun Zhu 4049f356f2 [Diffusion] Qwen image support (#82) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-11-28 14:22:11 +08:00
Gao Han db81ee9f69 [Docs] optimize and supplement docs system (#86) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-28 11:03:52 +08:00
Baoyuan Qi 38f58064eb [fix] Remove redundant parameter passing (#90) 
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2025-11-27 20:26:51 +08:00
Alicia 6d1bcfd18f [CI]Add version number. (#87) 
Signed-off-by: WANG Cong <115451386+congw729@users.noreply.github.com>
 2025-11-27 17:41:22 +08:00
Gao Han 598b4ddb51 [Feature] support multimodal inputs with multiple requests (#76) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-24 16:02:44 +08:00
Jiangyun Zhu 5707fc78d5 [CI] fix ci and format existing code (#71) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-11-18 20:54:26 +08:00
Gao Han c5a89302f1 [Feature] Support online inference (#64) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-17 14:44:10 +08:00
Baoyuan Qi bf82e18374 [fix] adapt hidden state device for multi-hardware support (#61) 
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2025-11-16 17:31:04 +08:00
Samit 8cafb4294d [Docs] Setup Documentation System and Re-organize Dependencies (#49) 
Signed-off-by: SamitHuang <285365963@qq.com>
 2025-11-15 21:45:43 +08:00
Gao Han a089dd4c4e Merge pull request #51 from tzhouam/feat/multi-request-stream-new 
[Feature] Multi Request Stream for Sync Mode
 2025-11-13 10:14:27 +08:00
tzhouam 26fcae5def Debug and Update doc 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-11 14:49:45 +08:00
tzhouam bc69d8e931 rename util further rename imports 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-11 10:57:15 +08:00
tzhouam 12cc2cd245 rename util 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-11 10:55:57 +08:00
tzhouam edae58b137 modify code based on review suggestions 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-11 10:03:26 +08:00
tzhouam c83d223e91 Squash 7 commits: 36ada0478a 0ef1fefd45 519c140d20 c18fc40f10 5f3f53a39d da385277a2 edefcc244b 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-10 16:42:58 +08:00
tzhouam 8e8783d78a Refactor end-to-end inference script and enhance argument parsing. Added support for loading prompts from a .pt file, introduced new command-line arguments for initialization and output handling, and improved error handling for prompt loading. Removed deprecated files related to previous implementations. 2025-11-10 11:58:47 +08:00
齐保元 d7618b1be4 [fix] support loading model from local path 
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2025-11-09 04:37:19 +08:00
tzhouam c5631ca753 Enhance error handling and logging in pipeline and stage processing. Added detailed logging for prompt loading failures, file writing issues, and stage shutdown processes. Introduced a new logging utility for orchestrator metrics and streamlined stats handling in the PipelinedOmniLLM class. 2025-11-07 18:05:00 +08:00
tzhouam 3c2ad1bf12 debug ar model runner 2025-11-07 16:25:48 +08:00
tzhouam 6a7a27801b Port changes from feat/multi-request-stream (vs main) 2025-11-07 15:48:47 +08:00
Gaohan123 e5adae0a07 fix other reviews 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-06 19:25:55 +08:00
Gaohan123 1aa5ae61cb fix reviews 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-05 23:26:58 +08:00
Gaohan123 e2b803fbe6 fix a bug 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-05 13:05:12 +08:00
Gaohan123 6e1478222e Dependency refractored to vLLM v0.11.0 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-04 22:06:41 +08:00
Gaohan123 d4a364f052 fix huggingface download problem for spk_dict.pt 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-01 16:45:34 +08:00
Gaohan123 6cf73c4e80 Remove unused files and fix some bugs 2025-10-25 18:02:11 +08:00
tzhouam f6158e7576 debug 2025-10-24 18:01:02 +08:00
tzhouam 605d1de013 Update entry points in vllm_omni module 
- Reintroduced AsyncOmniLLM to the main entry points.
- Added import for patch module.
- Cleaned up comments regarding AsyncOmniLLM support.
 2025-10-24 17:54:19 +08:00
tzhouam d45f8a6be7 debug typo 2025-10-24 17:47:49 +08:00
tzhouam 61b344dc39 Add import for OmniModelsRegistry in model executor initialization 2025-10-24 17:44:31 +08:00
tzhouam 3b1324ea39 Merge branch 'main' into end2end_fixup 2025-10-24 17:41:03 +08:00
tzhouam 3265e857eb Refactor OmniLLM and MultimodalOutputProcessor initialization 
- Removed AsyncOmniLLM from the main entry points as it is not supported yet.
- Updated the constructor of MultimodalOutputProcessor to set a default value for engine_core_output_type.
- Adjusted output type handling in various methods to utilize the new default engine_core_output_type for consistency.
 2025-10-24 17:15:34 +08:00
Gaohan123 171f3c1ea7 fix conflicts 2025-10-24 16:49:19 +08:00
Gaohan123 22f36de34e fix error 2025-10-24 16:21:52 +08:00
Gaohan123 eafc7c44c3 fix error for pre-commit 2025-10-24 16:21:50 +08:00
Gaohan123 a04e945166 add precommit 2025-10-24 16:20:58 +08:00
Hongsheng Liu 1bf3424e96 Merge pull request #19 from tzhouam/feat/omni-schedulers 
[Core]Add scheduling components for vLLM-omni
 2025-10-24 14:08:57 +08:00
tzhouam eb006f66e5 Enhance OmniScheduler error handling with logging 
- Added logging for exceptions when wrapping scheduled requests in OmniScheduler.
- Removed unnecessary blank lines for cleaner code structure.
 2025-10-24 12:00:36 +08:00
tzhouam 87b3ae7e70 Merge branch 'feat/omni-schedulers' of https://github.com/tzhouam/vllm-omni into feat/omni-schedulers 2025-10-24 11:57:45 +08:00
tzhouam c23d24ca19 Refactor comments in diffusion_scheduler.py for clarity and consistency 
- Translated comments from Chinese to English for better accessibility.
- Improved clarity of comments related to request handling and scheduling logic.
- Ensured comments align with the code functionality and maintain a uniform style.
 2025-10-24 11:57:39 +08:00
Zhou Taichang 582639c77a Merge branch 'main' into feat/hidden-state-input-request-input-processor-patch 
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
 2025-10-24 11:52:45 +08:00
tzhouam 63c72f4c2b Merge branch 'feat/hidden-state-input-request-input-processor-patch' of https://github.com/tzhouam/vllm-omni into feat/hidden-state-input-request-input-processor-patch 2025-10-24 11:40:49 +08:00
tzhouam a97409efef Add dtype mapping method in OmniProcessor 
- Introduced a static method `_dtype_to_name` to map PyTorch data types to their string representations.
- Updated the `dtype_str` assignments in the `OmniProcessor` class to utilize the new mapping method for improved readability and maintainability.
 2025-10-24 11:40:43 +08:00
tzhouam 4cd3b36fa0 Refactor OmniProcessor and update parsing logic 
- Removed redundant imports of `torch` and `numpy` in `OmniProcessor` class.
- Improved comments in `parse_singleton_prompt_omni` function for clarity on token handling.
- Ensured `current_platform` is imported only once for better code organization.
 2025-10-24 11:27:30 +08:00
Hongsheng Liu 4a4c3c1b82 Merge pull request #20 from tzhouam/feat/multimodal-output-processor 
[Engine]Refactor output processing for multimodal capabilities in vLLM-omni
 2025-10-24 11:25:23 +08:00
Hongsheng Liu 40d20978f1 Merge pull request #17 from tzhouam/feat/gpu-omni-model-runner-base 
[Worker]Add OmniGPUModelRunner and OmniModelInputForGPU classes
 2025-10-24 11:23:19 +08:00
Zhou TaichangandCopilot d0423129f8 Apply suggestion from @Copilot 
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
 2025-10-24 11:14:58 +08:00
Zhou TaichangandCopilot b04f114773 Apply suggestion from @Copilot 
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
 2025-10-24 11:14:41 +08:00
Hongsheng Liu 27fbe3511a Merge pull request #16 from tzhouam/feat/diffusion_gpu_worker_and_model_runner 
[Worker]Refactor GPU diffusion model runner and worker
 2025-10-24 11:11:02 +08:00