anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
593 Commits 235 Branches 31 Tags
9f00a37437ef2dece095b8bd0b3fb5b64f28b59a
Commit Graph
65 Commits
Author SHA1 Message Date
amy-why-3459 5304bc1e16 [Refactor] Refactor async chunk and fix the shape mismatch issue (#1151) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-02-04 00:00:56 +08:00
Zhou Taichangand汪志鹏 70ef548448 Dev/rebase v0.15.0 (#1159) 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-02-03 12:24:45 +08:00
Ziming Huangram16gHongsheng Liu
 		f6cfc0d1e3 [BugFix] Fix Qwen3 Omni talker mtp torch.compile startup error (#1104) 
Signed-off-by: ram16g <anlianfengjie@163.com>
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
Co-authored-by: ram16g <anlianfengjie@163.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-30 23:25:49 +08:00
Ziming HuangandHongsheng Liu a909d36415 [Perf] Qwen3 Omni talker mtp optimization (#1005) 
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-29 21:23:19 +08:00
汪志鹏 741f7e2826 [Core]: KV Cache Transfer Encapsulation (#979) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-01-28 23:12:45 +08:00
Canlin Guo c9392dff01 [Hardware] Support platforms and plugin system (#774) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-01-28 22:12:52 +08:00
Canlin Guo cb48968294 [Bugfix] Modify output to model_runner_output (#1026) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-01-28 18:42:39 +08:00
Canlin Guo cadd77262d [NPU] Upgrade to v0.14.0 (#820) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-01-27 16:45:32 +08:00
amy-why-3459Rein YangCHENHongsheng Liu
 		52bf1e2f40 [Feature]Support async computation and communication across stages by chunks (#727) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: CHEN <116010019@link.cuhk.edu.cn>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-26 11:21:24 +08:00
汪志鹏andwzliu 7f821be57b Support Bagel Model (#726) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: wzliu <wzliu@connect.hku.hk>
 2026-01-22 23:09:55 +13:00
Gao Han a54c323952 [Model] Support Qwen3-TTS model series (#895) 
Signed-off-by: Gaohan123 <hgaoaf@connect.ust.hk>
 2026-01-22 16:57:37 +08:00
+13
Zhou TaichangJaredforRealrootJustQJXU MingshiSihyeon JangJiangyun ZhucatcatZiming HuangYuhan LiuAliciaSamitYueqian LinWeiQing ChenPeiqi YinDidan DengHongsheng Liuwangyuwangyu31577汪志鹏John Liu BUAAD!NE$Hroot
 		776c3a794e Dev/rebase 0.14.0 (#847) 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
Signed-off-by: root <root@hk01dgx039.cm.cluster>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: TangPeng <85704592@qq.com>
Signed-off-by: XU Mingshi <91017482+mxuax@users.noreply.github.com>
Signed-off-by: mxuax <mxuax@connect.ust.hk>
Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: iwzbi <wzbi@zju.edu.cn>
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
Signed-off-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com>
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: linyueqian <linyueqian@outlook.com>
Signed-off-by: David Chen <530634352@qq.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: wangyu31577 <wangyu31577@hundsun.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: John Liu BUAA <liukecheng97@gmail.com>
Signed-off-by: Dinesh G <G.Dinesh@ibm.com>
Signed-off-by: gDINESH13 <dinesh13g@gmail.com>
Signed-off-by: root <root@hk01dgx028.cm.cluster>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: JaredforReal <w13431838023@gmail.com>
Co-authored-by: root <root@hk01dgx039.cm.cluster>
Co-authored-by: JustQJ <37905360+JustQJ@users.noreply.github.com>
Co-authored-by: XU Mingshi <91017482+mxuax@users.noreply.github.com>
Co-authored-by: Sihyeon Jang <uneedsihyeon@gmail.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Co-authored-by: catcat <108673086+iwzbi@users.noreply.github.com>
Co-authored-by: Ziming Huang <hzm414167@alibaba-inc.com>
Co-authored-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com>
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com>
Co-authored-by: Samit <285365963@qq.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com>
Co-authored-by: wangyu31577 <wangyu31577@hundsun.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: John Liu BUAA <liukecheng97@gmail.com>
Co-authored-by: D!NE$H <67671800+gDINESH13@users.noreply.github.com>
Co-authored-by: root <root@hk01dgx028.cm.cluster>
 2026-01-21 00:25:25 +08:00
Ziming Huang b5cf8806f0 [BugFix] Fix incorrect mrope positions under cuda graph (#803) 
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
 2026-01-16 03:24:21 +13:00
Ziming Huang 1444e1fc45 [Feature] Support Qwen3 Omni talker mtp batch inference (#722) 
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
 2026-01-14 19:07:31 +08:00
Zhou Taichangandroot 2e13bddf87 [Debug] Clean code in Qwen 3 Omni and add warning for talker temperature. (#688) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: root <root@hk01dgx028.cm.cluster>
 2026-01-12 23:13:45 +13:00
汪志鹏 0ec2da08de [BugFix] Fix assuming all stage model have talker (#730) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2026-01-11 01:36:37 +13:00
Ziming Huang 978e3ad7d4 [Feature] Support Qwen3 Omni talker cudagraph (#669) 
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
 2026-01-09 11:45:58 +08:00
Canlin Guo 46afa046ce [NPU] Support mixed modalities for Qwen3-Omni (#537) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2025-12-31 16:15:20 +08:00
Ziming HuangandHongsheng Liu 1ae0c64fe0 Support Qwen3 Omni thinker cuda graph (#523) 
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2025-12-30 23:46:13 +13:00
Zhou Taichang 59ae174bba [Debug] Debug qwen3 mix modality output empty string (#431) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-12-30 15:32:15 +08:00
Canlin Guo be81443c8a [Bugfix][NPU] Add _model_forward for ModelRunner (#505) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2025-12-28 04:09:34 +08:00
dsinghviandHongsheng Liu df4909630c model forward call logic fixes (#495) 
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2025-12-27 14:22:07 +00:00
Chenguang ZhengyinpeiqiHongsheng Liu
 		595e7c0032 [Core] Supports stage abstraction in the diffusion model (#391) 
Signed-off-by: Chenguang ZHENG <645327136@qq.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Co-authored-by: yinpeiqi <yinpeiqi809@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2025-12-27 01:59:33 +13:00
dsinghviandHongsheng Liu abde501dfa [Misc] Fix type interface mismatch (#478) 
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2025-12-26 15:28:49 +08:00
Canlin GuoandHongsheng Liu fe3d9a6a81 [NPU][Upgrade] Upgrade to v0.12.0 (#447) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2025-12-26 14:10:23 +08:00
Canlin Guo e1f9691120 [Clean] Remove the redundant decoding payloads logic (#404) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2025-12-26 04:48:34 +13:00
Peiqi Yin e9e6b3711c [Core] customize process between stages, remove the need of "request_ids" in model forward, rebase to 0.12.0 (#458) 
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
 2025-12-24 13:26:33 +00:00
Zhou TaichangRein YangIsotr0py
 		8939eb835d [Rebase] Rebase to vllm 0.12.0 (#335) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-12-19 16:42:02 +08:00
Lancer 01549fb953 [Diffusion] fix warning in diffusion stage (#332) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
 2025-12-16 09:07:16 +00:00
Cyrus Leung c091a03d82 [Chore] Bump Python version and apply formatting changes for v0.12 release (#126) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-12-16 15:19:40 +08:00
NATURE f995211da0 [Feature] Omni Connector + ray supported (#215) 
Signed-off-by: wzliu <wzliu@connect.hku.hk>
 2025-12-10 20:15:03 +08:00
Peiqi Yin bbb5a72871 [BugFix] Fix qwen3omni thinker batching. (#207) 
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
 2025-12-08 04:44:13 +00:00
Canlin Guo 6079b14040 [NPU][Refactor] Rename Diffusion* to Generation* (#211) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2025-12-05 08:23:37 +00:00
Zhou TaichangandGao Han c70b167f70 [Engine] Remove Diffusion_XX which duplicates with Generation_XX (#163) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Gao Han <gaohan19@huawei.com>
 2025-12-05 03:04:23 +00:00
Alicia a33bbcedea [Fix] Resolve comments & update vLLM-Omni name usages. (#122) 
Signed-off-by: WANG Cong <115451386+congw729@users.noreply.github.com>
 2025-12-01 08:05:24 +08:00
Canlin Guo ec9e04027d [Feat] Add NPU Backend support for vLLM-Omni (#89) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2025-11-30 20:39:06 +08:00
Zhou Taichang 228f4e5070 [Bug-fix] Fix Bugs in Qwen3/Qwen2.5 Omni Rebased Support (#114) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-30 18:15:10 +08:00
R2-YandRein 0a4e5a8710 [Feature] add support for Qwen3-omni (#55) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Rein <rein@Reins-MacBook-Air.local>
 2025-11-30 04:44:01 +08:00
Roger WangandGao Han a5800cd2c2 [Doc] Misc documentation polishing (#98) 
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: Gao Han <gaohan19@huawei.com>
Co-authored-by: Gao Han <gaohan19@huawei.com>
 2025-11-29 20:42:39 +08:00
Gao Han db81ee9f69 [Docs] optimize and supplement docs system (#86) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-28 11:03:52 +08:00
Gao Han 598b4ddb51 [Feature] support multimodal inputs with multiple requests (#76) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-24 16:02:44 +08:00
Jiangyun Zhu 5707fc78d5 [CI] fix ci and format existing code (#71) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-11-18 20:54:26 +08:00
Gao Han c5a89302f1 [Feature] Support online inference (#64) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-17 14:44:10 +08:00
tzhouam c5631ca753 Enhance error handling and logging in pipeline and stage processing. Added detailed logging for prompt loading failures, file writing issues, and stage shutdown processes. Introduced a new logging utility for orchestrator metrics and streamlined stats handling in the PipelinedOmniLLM class. 2025-11-07 18:05:00 +08:00
tzhouam 3c2ad1bf12 debug ar model runner 2025-11-07 16:25:48 +08:00
tzhouam 6a7a27801b Port changes from feat/multi-request-stream (vs main) 2025-11-07 15:48:47 +08:00
Gaohan123 e5adae0a07 fix other reviews 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-06 19:25:55 +08:00
Gaohan123 1aa5ae61cb fix reviews 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-05 23:26:58 +08:00
Gaohan123 6e1478222e Dependency refractored to vLLM v0.11.0 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-04 22:06:41 +08:00
Gaohan123 171f3c1ea7 fix conflicts 2025-10-24 16:49:19 +08:00