anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
260 Commits 235 Branches 31 Tags
31fedcfa2bc421ca44a0197b547d2bcd98680e86
Commit Graph
55 Commits
Author SHA1 Message Date
dengyunyang bd5c81cb7e [BugFix] Failed to print DEBUG level log (#307) 
Signed-off-by: dengyunyang <584797741@qq.com>
 2025-12-15 07:13:36 +00:00
Jiangyun Zhu 9c03b80428 [Misc] move omni_diffusion.py to vllm_omni.entrypoints (#312) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-12-15 12:54:31 +08:00
Prajwal Azjy0516Samit
 		65ca13102d [Feature] teacache integration (#179) 
Signed-off-by: Prajwal A <prajwalanagani@gmail.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Samit <285365963@qq.com>
 2025-12-12 04:29:56 +00:00
LancerandLancer 0548674d30 [Entrypoints] Minor optimization in the orchestrator's final stage determination logic (#275) 
Signed-off-by: Lancer <maruixiang6688@gmai.com>
Co-authored-by: Lancer <maruixiang6688@gmail.com>
 2025-12-11 17:59:00 +08:00
NATURE f995211da0 [Feature] Omni Connector + ray supported (#215) 
Signed-off-by: wzliu <wzliu@connect.hku.hk>
 2025-12-10 20:15:03 +08:00
汪志鹏 71f9f9a9a7 [Bugfix] Remove debug code in AsyncOmni.del to fix resource leak (#260) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2025-12-09 23:23:01 +08:00
WeiQing Chen 9a7f7b7de5 [Bugfix]multi-GPU tp scenarios, devices: "0,1" uses physical IDs instead of logical IDs (#253) 
Signed-off-by: David Chen <530634352@qq.com>
 2025-12-09 21:04:30 +08:00
Gao Han 45dc189d8d [Bugfix] Support passing vllm cli args to online serving in vLLM-Omni (#206) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
Signed-off-by: Gaohan123 <hgaoaf@connect.ust.hk>
 2025-12-08 14:59:53 +08:00
Jiangyun Zhu aa689e6af0 [Core] add clean up method for diffusion engine (#219) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-12-08 03:52:07 +00:00
Mengqing Cao 00b1a9a1a0 [Misc] Temporarily support downloading models from ModelScope by snapshot download (#132) 
Signed-off-by: MengqingCao <cmq0113@163.com>
 2025-12-01 17:13:23 +00:00
Roger Wang c852267a1a [Misc] Update default stage config for qwen3-omni (#124) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-11-30 20:01:20 -08:00
Alicia a33bbcedea [Fix] Resolve comments & update vLLM-Omni name usages. (#122) 
Signed-off-by: WANG Cong <115451386+congw729@users.noreply.github.com>
 2025-12-01 08:05:24 +08:00
Alicia 3427fb4774 [Feat] Rename AsyncOmniLLM -> AsyncOmni (#103) 
Signed-off-by: WANG Cong <115451386+congw729@users.noreply.github.com>
 2025-11-30 22:26:33 +08:00
Gao Han bfd3cad5ee [Bugfix] Fix bug of online serving for qwen2.5-omni (#118) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-30 21:30:17 +08:00
Canlin Guo ec9e04027d [Feat] Add NPU Backend support for vLLM-Omni (#89) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2025-11-30 20:39:06 +08:00
Gao Han 88c00fb271 [Bugfix] Remove mandatory flash-attn dependency and optimzie docs (#113) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-30 18:20:44 +08:00
Zhou Taichang 228f4e5070 [Bug-fix] Fix Bugs in Qwen3/Qwen2.5 Omni Rebased Support (#114) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-30 18:15:10 +08:00
R2-YandRein 0a4e5a8710 [Feature] add support for Qwen3-omni (#55) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Rein <rein@Reins-MacBook-Air.local>
 2025-11-30 04:44:01 +08:00
Roger WangandGao Han a5800cd2c2 [Doc] Misc documentation polishing (#98) 
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: Gao Han <gaohan19@huawei.com>
Co-authored-by: Gao Han <gaohan19@huawei.com>
 2025-11-29 20:42:39 +08:00
Jiangyun Zhu bcc6bd9642 [diffusion] add doc and fix qwen-image (#96) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-11-28 18:07:44 +08:00
Syed Muhammad Bin Asif f4633fe381 [Bugfix] Fix removal of old logs when stats are enabled (#84) 
Signed-off-by: syedmba <syedmba7@connect.hku.hk>
 2025-11-28 15:58:16 +08:00
Jiangyun Zhu 4049f356f2 [Diffusion] Qwen image support (#82) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-11-28 14:22:11 +08:00
Gao Han db81ee9f69 [Docs] optimize and supplement docs system (#86) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-28 11:03:52 +08:00
Baoyuan Qi 38f58064eb [fix] Remove redundant parameter passing (#90) 
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2025-11-27 20:26:51 +08:00
Gao Han 598b4ddb51 [Feature] support multimodal inputs with multiple requests (#76) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-24 16:02:44 +08:00
Jiangyun Zhu 5707fc78d5 [CI] fix ci and format existing code (#71) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-11-18 20:54:26 +08:00
Gao Han c5a89302f1 [Feature] Support online inference (#64) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-17 14:44:10 +08:00
tzhouam 26fcae5def Debug and Update doc 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-11 14:49:45 +08:00
tzhouam bc69d8e931 rename util further rename imports 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-11 10:57:15 +08:00
tzhouam 12cc2cd245 rename util 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-11 10:55:57 +08:00
tzhouam edae58b137 modify code based on review suggestions 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-11 10:03:26 +08:00
tzhouam c83d223e91 Squash 7 commits: 36ada0478a 0ef1fefd45 519c140d20 c18fc40f10 5f3f53a39d da385277a2 edefcc244b 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-10 16:42:58 +08:00
tzhouam 8e8783d78a Refactor end-to-end inference script and enhance argument parsing. Added support for loading prompts from a .pt file, introduced new command-line arguments for initialization and output handling, and improved error handling for prompt loading. Removed deprecated files related to previous implementations. 2025-11-10 11:58:47 +08:00
tzhouam c5631ca753 Enhance error handling and logging in pipeline and stage processing. Added detailed logging for prompt loading failures, file writing issues, and stage shutdown processes. Introduced a new logging utility for orchestrator metrics and streamlined stats handling in the PipelinedOmniLLM class. 2025-11-07 18:05:00 +08:00
tzhouam 6a7a27801b Port changes from feat/multi-request-stream (vs main) 2025-11-07 15:48:47 +08:00
Gaohan123 6e1478222e Dependency refractored to vLLM v0.11.0 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-04 22:06:41 +08:00
Gaohan123 6cf73c4e80 Remove unused files and fix some bugs 2025-10-25 18:02:11 +08:00
tzhouam 3b1324ea39 Merge branch 'main' into end2end_fixup 2025-10-24 17:41:03 +08:00
tzhouam 3265e857eb Refactor OmniLLM and MultimodalOutputProcessor initialization 
- Removed AsyncOmniLLM from the main entry points as it is not supported yet.
- Updated the constructor of MultimodalOutputProcessor to set a default value for engine_core_output_type.
- Adjusted output type handling in various methods to utilize the new default engine_core_output_type for consistency.
 2025-10-24 17:15:34 +08:00
Gaohan123 171f3c1ea7 fix conflicts 2025-10-24 16:49:19 +08:00
Gaohan123 eafc7c44c3 fix error for pre-commit 2025-10-24 16:21:50 +08:00
Gaohan123 a04e945166 add precommit 2025-10-24 16:20:58 +08:00
Gaohan123 61a215fb54 fix file name 2025-10-24 10:19:35 +08:00
Gaohan123 2c08c04730 fix for reviews of copilot 2025-10-24 10:17:22 +08:00
Gaohan123 98a85d42a6 fix for some reviews 2025-10-23 10:26:21 +08:00
Gaohan123 87d6414a26 supplement some imports 2025-10-22 18:14:11 +08:00
Gaohan123 6111c288d0 fix some reviews from gemini 2025-10-22 17:46:10 +08:00
Gaohan123 20cfd15cc6 Add entrypoint class and stage management 2025-10-22 14:37:55 +08:00
hsliu 05d23676ff change omni.py to serve.py and mv to cli 2025-09-29 18:10:37 +08:00
hsliu c330e5e8f3 del unnessearcy files 2025-09-29 17:49:32 +08:00