anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
508 Commits 235 Branches 31 Tags
ed8b03224b0f00a6ccea65d80104f852c4e57acd
Commit Graph
58 Commits
Author SHA1 Message Date
Canlin Guo cadd77262d [NPU] Upgrade to v0.14.0 (#820) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-01-27 16:45:32 +08:00
amy-why-3459Rein YangCHENHongsheng Liu
 		52bf1e2f40 [Feature]Support async computation and communication across stages by chunks (#727) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: CHEN <116010019@link.cuhk.edu.cn>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-26 11:21:24 +08:00
汪志鹏andwzliu 7f821be57b Support Bagel Model (#726) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: wzliu <wzliu@connect.hku.hk>
 2026-01-22 23:09:55 +13:00
Gao Han a54c323952 [Model] Support Qwen3-TTS model series (#895) 
Signed-off-by: Gaohan123 <hgaoaf@connect.ust.hk>
 2026-01-22 16:57:37 +08:00
+13
Zhou TaichangJaredforRealrootJustQJXU MingshiSihyeon JangJiangyun ZhucatcatZiming HuangYuhan LiuAliciaSamitYueqian LinWeiQing ChenPeiqi YinDidan DengHongsheng Liuwangyuwangyu31577汪志鹏John Liu BUAAD!NE$Hroot
 		776c3a794e Dev/rebase 0.14.0 (#847) 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
Signed-off-by: root <root@hk01dgx039.cm.cluster>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: TangPeng <85704592@qq.com>
Signed-off-by: XU Mingshi <91017482+mxuax@users.noreply.github.com>
Signed-off-by: mxuax <mxuax@connect.ust.hk>
Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: iwzbi <wzbi@zju.edu.cn>
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
Signed-off-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com>
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: linyueqian <linyueqian@outlook.com>
Signed-off-by: David Chen <530634352@qq.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: wangyu31577 <wangyu31577@hundsun.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: John Liu BUAA <liukecheng97@gmail.com>
Signed-off-by: Dinesh G <G.Dinesh@ibm.com>
Signed-off-by: gDINESH13 <dinesh13g@gmail.com>
Signed-off-by: root <root@hk01dgx028.cm.cluster>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: JaredforReal <w13431838023@gmail.com>
Co-authored-by: root <root@hk01dgx039.cm.cluster>
Co-authored-by: JustQJ <37905360+JustQJ@users.noreply.github.com>
Co-authored-by: XU Mingshi <91017482+mxuax@users.noreply.github.com>
Co-authored-by: Sihyeon Jang <uneedsihyeon@gmail.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Co-authored-by: catcat <108673086+iwzbi@users.noreply.github.com>
Co-authored-by: Ziming Huang <hzm414167@alibaba-inc.com>
Co-authored-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com>
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com>
Co-authored-by: Samit <285365963@qq.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com>
Co-authored-by: wangyu31577 <wangyu31577@hundsun.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: John Liu BUAA <liukecheng97@gmail.com>
Co-authored-by: D!NE$H <67671800+gDINESH13@users.noreply.github.com>
Co-authored-by: root <root@hk01dgx028.cm.cluster>
 2026-01-21 00:25:25 +08:00
Ziming Huang b5cf8806f0 [BugFix] Fix incorrect mrope positions under cuda graph (#803) 
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
 2026-01-16 03:24:21 +13:00
Ziming Huang 1444e1fc45 [Feature] Support Qwen3 Omni talker mtp batch inference (#722) 
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
 2026-01-14 19:07:31 +08:00
Zhou Taichangandroot 2e13bddf87 [Debug] Clean code in Qwen 3 Omni and add warning for talker temperature. (#688) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: root <root@hk01dgx028.cm.cluster>
 2026-01-12 23:13:45 +13:00
汪志鹏 0ec2da08de [BugFix] Fix assuming all stage model have talker (#730) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2026-01-11 01:36:37 +13:00
Ziming Huang 978e3ad7d4 [Feature] Support Qwen3 Omni talker cudagraph (#669) 
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
 2026-01-09 11:45:58 +08:00
Canlin Guo 46afa046ce [NPU] Support mixed modalities for Qwen3-Omni (#537) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2025-12-31 16:15:20 +08:00
Ziming HuangandHongsheng Liu 1ae0c64fe0 Support Qwen3 Omni thinker cuda graph (#523) 
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2025-12-30 23:46:13 +13:00
Zhou Taichang 59ae174bba [Debug] Debug qwen3 mix modality output empty string (#431) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-12-30 15:32:15 +08:00
Canlin Guo be81443c8a [Bugfix][NPU] Add _model_forward for ModelRunner (#505) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2025-12-28 04:09:34 +08:00
dsinghviandHongsheng Liu df4909630c model forward call logic fixes (#495) 
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2025-12-27 14:22:07 +00:00
Chenguang ZhengyinpeiqiHongsheng Liu
 		595e7c0032 [Core] Supports stage abstraction in the diffusion model (#391) 
Signed-off-by: Chenguang ZHENG <645327136@qq.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Co-authored-by: yinpeiqi <yinpeiqi809@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2025-12-27 01:59:33 +13:00
dsinghviandHongsheng Liu abde501dfa [Misc] Fix type interface mismatch (#478) 
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2025-12-26 15:28:49 +08:00
Canlin GuoandHongsheng Liu fe3d9a6a81 [NPU][Upgrade] Upgrade to v0.12.0 (#447) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2025-12-26 14:10:23 +08:00
Canlin Guo e1f9691120 [Clean] Remove the redundant decoding payloads logic (#404) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2025-12-26 04:48:34 +13:00
Peiqi Yin e9e6b3711c [Core] customize process between stages, remove the need of "request_ids" in model forward, rebase to 0.12.0 (#458) 
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
 2025-12-24 13:26:33 +00:00
Zhou TaichangRein YangIsotr0py
 		8939eb835d [Rebase] Rebase to vllm 0.12.0 (#335) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-12-19 16:42:02 +08:00
Lancer 01549fb953 [Diffusion] fix warning in diffusion stage (#332) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
 2025-12-16 09:07:16 +00:00
Cyrus Leung c091a03d82 [Chore] Bump Python version and apply formatting changes for v0.12 release (#126) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-12-16 15:19:40 +08:00
NATURE f995211da0 [Feature] Omni Connector + ray supported (#215) 
Signed-off-by: wzliu <wzliu@connect.hku.hk>
 2025-12-10 20:15:03 +08:00
Peiqi Yin bbb5a72871 [BugFix] Fix qwen3omni thinker batching. (#207) 
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
 2025-12-08 04:44:13 +00:00
Canlin Guo 6079b14040 [NPU][Refactor] Rename Diffusion* to Generation* (#211) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2025-12-05 08:23:37 +00:00
Zhou TaichangandGao Han c70b167f70 [Engine] Remove Diffusion_XX which duplicates with Generation_XX (#163) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Gao Han <gaohan19@huawei.com>
 2025-12-05 03:04:23 +00:00
Alicia a33bbcedea [Fix] Resolve comments & update vLLM-Omni name usages. (#122) 
Signed-off-by: WANG Cong <115451386+congw729@users.noreply.github.com>
 2025-12-01 08:05:24 +08:00
Canlin Guo ec9e04027d [Feat] Add NPU Backend support for vLLM-Omni (#89) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2025-11-30 20:39:06 +08:00
Zhou Taichang 228f4e5070 [Bug-fix] Fix Bugs in Qwen3/Qwen2.5 Omni Rebased Support (#114) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-30 18:15:10 +08:00
R2-YandRein 0a4e5a8710 [Feature] add support for Qwen3-omni (#55) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Rein <rein@Reins-MacBook-Air.local>
 2025-11-30 04:44:01 +08:00
Roger WangandGao Han a5800cd2c2 [Doc] Misc documentation polishing (#98) 
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: Gao Han <gaohan19@huawei.com>
Co-authored-by: Gao Han <gaohan19@huawei.com>
 2025-11-29 20:42:39 +08:00
Gao Han db81ee9f69 [Docs] optimize and supplement docs system (#86) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-28 11:03:52 +08:00
Gao Han 598b4ddb51 [Feature] support multimodal inputs with multiple requests (#76) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-24 16:02:44 +08:00
Jiangyun Zhu 5707fc78d5 [CI] fix ci and format existing code (#71) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-11-18 20:54:26 +08:00
Gao Han c5a89302f1 [Feature] Support online inference (#64) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-17 14:44:10 +08:00
tzhouam c5631ca753 Enhance error handling and logging in pipeline and stage processing. Added detailed logging for prompt loading failures, file writing issues, and stage shutdown processes. Introduced a new logging utility for orchestrator metrics and streamlined stats handling in the PipelinedOmniLLM class. 2025-11-07 18:05:00 +08:00
tzhouam 3c2ad1bf12 debug ar model runner 2025-11-07 16:25:48 +08:00
tzhouam 6a7a27801b Port changes from feat/multi-request-stream (vs main) 2025-11-07 15:48:47 +08:00
Gaohan123 e5adae0a07 fix other reviews 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-06 19:25:55 +08:00
Gaohan123 1aa5ae61cb fix reviews 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-05 23:26:58 +08:00
Gaohan123 6e1478222e Dependency refractored to vLLM v0.11.0 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-04 22:06:41 +08:00
Gaohan123 171f3c1ea7 fix conflicts 2025-10-24 16:49:19 +08:00
Hongsheng Liu 40d20978f1 Merge pull request #17 from tzhouam/feat/gpu-omni-model-runner-base 
[Worker]Add OmniGPUModelRunner and OmniModelInputForGPU classes
 2025-10-24 11:23:19 +08:00
Hongsheng Liu 27fbe3511a Merge pull request #16 from tzhouam/feat/diffusion_gpu_worker_and_model_runner 
[Worker]Refactor GPU diffusion model runner and worker
 2025-10-24 11:11:02 +08:00
tzhouam ee4822327b Remove unnecessary numpy imports and update return type in extract_multimodal_outputs method in OmniGPUModelRunner 
- Eliminated redundant numpy import statements to streamline the code.
- Changed the return type of extract_multimodal_outputs from dict to tuple for better clarity on output structure.
 2025-10-24 11:08:42 +08:00
tzhouam 634915652d Remove unused code and comments in GPUDiffusionModelRunner for improved clarity 
- Deleted the `has_kv_transfer_group` import as it was not utilized.
- Cleaned up commented-out code related to model methods, focusing on the forward method.
- Added a TODO comment to implement the diffuse method for additional model support.
 2025-10-24 10:34:41 +08:00
tzhouam 28ae542eed match runner, worker name with file name 2025-10-23 14:35:23 +08:00
tzhouam ea16546ae9 Refactor GPU diffusion model classes for clarity and consistency 
- Renamed `DiffusionModelRunner` to `GPUDiffusionModelRunner` for better clarity in class purpose.
- Updated `DiffusionGPUWorker` to utilize the renamed `GPUDiffusionModelRunner`.
- Simplified output handling in `GPUDiffusionModelRunner` by removing unnecessary checks, enhancing code readability.
 2025-10-23 11:04:10 +08:00
tzhouam 4609afa613 Refactor sampled_token_ids assignment in OmniGPUModelRunner 
- Simplified the assignment of sampled_token_ids by removing the conditional check for "model_stage".
- Ensured consistent handling of sampled_token_ids for improved clarity and maintainability.
 2025-10-22 17:42:29 +08:00