tzhouam
22e01c416b
Merge remote-tracking branch 'origin/main' into dev/vllm-align
2026-02-25 08:15:55 +00:00
Zhou Taichang
296d995ec2
[CI] Remove overwhelming debug log ( #1463 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-25 12:06:01 +08:00
Canlin Guo and Hongsheng Liu
af11b02081
[Minor][Refactor] Pass seq_token_counts explicitly ( #1425 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-02-24 14:10:00 +08:00
tzhouam
861698f88f
Update Dockerfile for vllm wheel version and enhance SHM connector tests
...
- Updated Dockerfile to use a new vllm wheel version.
- Modified test_put_get_inline to clarify that it always uses SHM and updated metadata assertions.
- Added handling for speculative configuration in GPU model runners to delay clearing connector metadata.
- Refactored payload decoding in OmniGPUModelRunner for better clarity and maintainability.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-24 05:45:48 +00:00
tzhouam
bf69cd94bb
Rebase 2026/02/23
...
Update Dockerfile for vllm wheel version and enhance input processing in Omni models. Added decoding for byte messages in async requests, included new metrics in benchmarks, and improved pause state handling in the scheduler. Refined input and output processing methods for better multimodal support and error handling.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-23 03:36:01 +00:00
Sy03 and Hongsheng Liu
efbe4116ff
[Feature] : Support disaggregated inference pipeline for Qwen3_TTS ( #1161 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-02-21 07:36:44 +08:00
75770c9309
[Rebase] Rebase to vllm v0.16.0 ( #1357 )
...
ComfyUI Integration -- Publish to Comfy registry / Publish Custom Node to registry (push) Failing after 44s
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: ZJY0516 <zhu.jiangyun@foxmail.com >
Co-authored-by: youkaichao <youkaichao@gmail.com >
2026-02-13 18:05:07 +08:00
0b56fba7b3
[Pref] Support Qwen3 Omni code2wav batch infernce with async chunk ( #1246 )
...
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Signed-off-by: Ziming Huang <1520787127@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2026-02-12 11:50:10 +08:00
Zhou Taichang
a854f0b7ff
[Debug] Add trigger to concurrent stage init ( #1274 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-11 14:18:18 +08:00
dsinghvi
59024bee62
Process-Scoped GPU Memory Accounting ( #1204 )
...
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com >
2026-02-08 22:43:02 +08:00
amy-why-3459
5304bc1e16
[Refactor] Refactor async chunk and fix the shape mismatch issue ( #1151 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-02-04 00:00:56 +08:00
Zhou Taichang and 汪志鹏
70ef548448
Dev/rebase v0.15.0 ( #1159 )
...
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
2026-02-03 12:24:45 +08:00
f6cfc0d1e3
[BugFix] Fix Qwen3 Omni talker mtp torch.compile startup error ( #1104 )
...
Signed-off-by: ram16g <anlianfengjie@163.com >
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
Co-authored-by: ram16g <anlianfengjie@163.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-01-30 23:25:49 +08:00
Ziming Huang and Hongsheng Liu
a909d36415
[Perf] Qwen3 Omni talker mtp optimization ( #1005 )
...
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-01-29 21:23:19 +08:00
汪志鹏
741f7e2826
[Core]: KV Cache Transfer Encapsulation ( #979 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com >
2026-01-28 23:12:45 +08:00
Canlin Guo
c9392dff01
[Hardware] Support platforms and plugin system ( #774 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
2026-01-28 22:12:52 +08:00
Canlin Guo
cb48968294
[Bugfix] Modify output to model_runner_output ( #1026 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
2026-01-28 18:42:39 +08:00
Canlin Guo
cadd77262d
[NPU] Upgrade to v0.14.0 ( #820 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
2026-01-27 16:45:32 +08:00
52bf1e2f40
[Feature]Support async computation and communication across stages by chunks ( #727 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
Signed-off-by: Rein Yang <ruiruyang2@gmail.com >
Co-authored-by: Rein Yang <ruiruyang2@gmail.com >
Co-authored-by: CHEN <116010019@link.cuhk.edu.cn >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-01-26 11:21:24 +08:00
汪志鹏 and wzliu
7f821be57b
Support Bagel Model ( #726 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Co-authored-by: wzliu <wzliu@connect.hku.hk >
2026-01-22 23:09:55 +13:00
Gao Han
a54c323952
[Model] Support Qwen3-TTS model series ( #895 )
...
Signed-off-by: Gaohan123 <hgaoaf@connect.ust.hk >
2026-01-22 16:57:37 +08:00
+13
776c3a794e
Dev/rebase 0.14.0 ( #847 )
...
Signed-off-by: JaredforReal <w13431838023@gmail.com >
Signed-off-by: root <root@hk01dgx039.cm.cluster >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
Signed-off-by: TangPeng <85704592@qq.com >
Signed-off-by: XU Mingshi <91017482+mxuax@users.noreply.github.com >
Signed-off-by: mxuax <mxuax@connect.ust.hk >
Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: iwzbi <wzbi@zju.edu.cn >
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
Signed-off-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com >
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com >
Signed-off-by: samithuang <285365963@qq.com >
Signed-off-by: linyueqian <linyueqian@outlook.com >
Signed-off-by: David Chen <530634352@qq.com >
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com >
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Signed-off-by: wangyu31577 <wangyu31577@hundsun.com >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: John Liu BUAA <liukecheng97@gmail.com >
Signed-off-by: Dinesh G <G.Dinesh@ibm.com >
Signed-off-by: gDINESH13 <dinesh13g@gmail.com >
Signed-off-by: root <root@hk01dgx028.cm.cluster >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
Co-authored-by: JaredforReal <w13431838023@gmail.com >
Co-authored-by: root <root@hk01dgx039.cm.cluster >
Co-authored-by: JustQJ <37905360+JustQJ@users.noreply.github.com >
Co-authored-by: XU Mingshi <91017482+mxuax@users.noreply.github.com >
Co-authored-by: Sihyeon Jang <uneedsihyeon@gmail.com >
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com >
Co-authored-by: catcat <108673086+iwzbi@users.noreply.github.com >
Co-authored-by: Ziming Huang <hzm414167@alibaba-inc.com >
Co-authored-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com >
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com >
Co-authored-by: Samit <285365963@qq.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com >
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com >
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com >
Co-authored-by: wangyu31577 <wangyu31577@hundsun.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: John Liu BUAA <liukecheng97@gmail.com >
Co-authored-by: D!NE$H <67671800+gDINESH13@users.noreply.github.com >
Co-authored-by: root <root@hk01dgx028.cm.cluster >
2026-01-21 00:25:25 +08:00
Ziming Huang
b5cf8806f0
[BugFix] Fix incorrect mrope positions under cuda graph ( #803 )
...
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
2026-01-16 03:24:21 +13:00
Ziming Huang
1444e1fc45
[Feature] Support Qwen3 Omni talker mtp batch inference ( #722 )
...
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
2026-01-14 19:07:31 +08:00
Zhou Taichang and root
2e13bddf87
[Debug] Clean code in Qwen 3 Omni and add warning for talker temperature. ( #688 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Co-authored-by: root <root@hk01dgx028.cm.cluster >
2026-01-12 23:13:45 +13:00
汪志鹏
0ec2da08de
[BugFix] Fix assuming all stage model have talker ( #730 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
2026-01-11 01:36:37 +13:00
Ziming Huang
978e3ad7d4
[Feature] Support Qwen3 Omni talker cudagraph ( #669 )
...
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
2026-01-09 11:45:58 +08:00
Canlin Guo
46afa046ce
[NPU] Support mixed modalities for Qwen3-Omni ( #537 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
2025-12-31 16:15:20 +08:00
Ziming Huang and Hongsheng Liu
1ae0c64fe0
Support Qwen3 Omni thinker cuda graph ( #523 )
...
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2025-12-30 23:46:13 +13:00
Zhou Taichang
59ae174bba
[Debug] Debug qwen3 mix modality output empty string ( #431 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2025-12-30 15:32:15 +08:00
Canlin Guo
be81443c8a
[Bugfix][NPU] Add _model_forward for ModelRunner ( #505 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
2025-12-28 04:09:34 +08:00
dsinghvi and Hongsheng Liu
df4909630c
model forward call logic fixes ( #495 )
...
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2025-12-27 14:22:07 +00:00
595e7c0032
[Core] Supports stage abstraction in the diffusion model ( #391 )
...
Signed-off-by: Chenguang ZHENG <645327136@qq.com >
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com >
Co-authored-by: yinpeiqi <yinpeiqi809@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2025-12-27 01:59:33 +13:00
dsinghvi and Hongsheng Liu
abde501dfa
[Misc] Fix type interface mismatch ( #478 )
...
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2025-12-26 15:28:49 +08:00
Canlin Guo and Hongsheng Liu
fe3d9a6a81
[NPU][Upgrade] Upgrade to v0.12.0 ( #447 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2025-12-26 14:10:23 +08:00
Canlin Guo
e1f9691120
[Clean] Remove the redundant decoding payloads logic ( #404 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
2025-12-26 04:48:34 +13:00
Peiqi Yin
e9e6b3711c
[Core] customize process between stages, remove the need of "request_ids" in model forward, rebase to 0.12.0 ( #458 )
...
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com >
2025-12-24 13:26:33 +00:00
8939eb835d
[Rebase] Rebase to vllm 0.12.0 ( #335 )
...
Signed-off-by: Rein Yang <ruiruyang2@gmail.com >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Co-authored-by: Rein Yang <ruiruyang2@gmail.com >
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2025-12-19 16:42:02 +08:00
Lancer
01549fb953
[Diffusion] fix warning in diffusion stage ( #332 )
...
Signed-off-by: Lancer <maruixiang6688@gmail.com >
2025-12-16 09:07:16 +00:00
Cyrus Leung
c091a03d82
[Chore] Bump Python version and apply formatting changes for v0.12 release ( #126 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2025-12-16 15:19:40 +08:00
NATURE
f995211da0
[Feature] Omni Connector + ray supported ( #215 )
...
Signed-off-by: wzliu <wzliu@connect.hku.hk >
2025-12-10 20:15:03 +08:00
Peiqi Yin
bbb5a72871
[BugFix] Fix qwen3omni thinker batching. ( #207 )
...
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com >
2025-12-08 04:44:13 +00:00
Canlin Guo
6079b14040
[NPU][Refactor] Rename Diffusion* to Generation* ( #211 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
2025-12-05 08:23:37 +00:00
Zhou Taichang and Gao Han
c70b167f70
[Engine] Remove Diffusion_XX which duplicates with Generation_XX ( #163 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Co-authored-by: Gao Han <gaohan19@huawei.com >
2025-12-05 03:04:23 +00:00
Alicia
a33bbcedea
[Fix] Resolve comments & update vLLM-Omni name usages. ( #122 )
...
Signed-off-by: WANG Cong <115451386+congw729@users.noreply.github.com >
2025-12-01 08:05:24 +08:00
Canlin Guo
ec9e04027d
[Feat] Add NPU Backend support for vLLM-Omni ( #89 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
2025-11-30 20:39:06 +08:00
Zhou Taichang
228f4e5070
[Bug-fix] Fix Bugs in Qwen3/Qwen2.5 Omni Rebased Support ( #114 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2025-11-30 18:15:10 +08:00
R2-Y and Rein
0a4e5a8710
[Feature] add support for Qwen3-omni ( #55 )
...
Signed-off-by: Rein Yang <ruiruyang2@gmail.com >
Co-authored-by: Rein <rein@Reins-MacBook-Air.local >
2025-11-30 04:44:01 +08:00
Roger Wang and Gao Han
a5800cd2c2
[Doc] Misc documentation polishing ( #98 )
...
Signed-off-by: Roger Wang <hey@rogerw.io >
Signed-off-by: Gao Han <gaohan19@huawei.com >
Co-authored-by: Gao Han <gaohan19@huawei.com >
2025-11-29 20:42:39 +08:00
Gao Han
db81ee9f69
[Docs] optimize and supplement docs system ( #86 )
...
Signed-off-by: Gaohan123 <gaohan19@huawei.com >
2025-11-28 11:03:52 +08:00