Commit Graph
75 Commits
Author SHA1 Message Date
tzhouam 22e01c416b Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-02-25 08:15:55 +00:00
Zhou Taichang 296d995ec2 [CI] Remove overwhelming debug log (#1463)
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-02-25 12:06:01 +08:00
Canlin GuoandHongsheng Liu af11b02081 [Minor][Refactor] Pass seq_token_counts explicitly (#1425)
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-02-24 14:10:00 +08:00
tzhouam 861698f88f Update Dockerfile for vllm wheel version and enhance SHM connector tests
- Updated Dockerfile to use a new vllm wheel version.
- Modified test_put_get_inline to clarify that it always uses SHM and updated metadata assertions.
- Added handling for speculative configuration in GPU model runners to delay clearing connector metadata.
- Refactored payload decoding in OmniGPUModelRunner for better clarity and maintainability.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-02-24 05:45:48 +00:00
tzhouam bf69cd94bb Rebase 2026/02/23
Update Dockerfile for vllm wheel version and enhance input processing in Omni models. Added decoding for byte messages in async requests, included new metrics in benchmarks, and improved pause state handling in the scheduler. Refined input and output processing methods for better multimodal support and error handling.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-02-23 03:36:01 +00:00
Sy03andHongsheng Liu efbe4116ff [Feature] : Support disaggregated inference pipeline for Qwen3_TTS (#1161)
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-02-21 07:36:44 +08:00
75770c9309 [Rebase] Rebase to vllm v0.16.0 (#1357)
ComfyUI Integration -- Publish to Comfy registry / Publish Custom Node to registry (push) Failing after 44s
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: ZJY0516 <zhu.jiangyun@foxmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
2026-02-13 18:05:07 +08:00
0b56fba7b3 [Pref] Support Qwen3 Omni code2wav batch infernce with async chunk (#1246)
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: Ziming Huang <1520787127@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-02-12 11:50:10 +08:00
Zhou Taichang a854f0b7ff [Debug] Add trigger to concurrent stage init (#1274)
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-02-11 14:18:18 +08:00
dsinghvi 59024bee62 Process-Scoped GPU Memory Accounting (#1204)
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
2026-02-08 22:43:02 +08:00
amy-why-3459 5304bc1e16 [Refactor] Refactor async chunk and fix the shape mismatch issue (#1151)
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
2026-02-04 00:00:56 +08:00
Zhou Taichangand汪志鹏 70ef548448 Dev/rebase v0.15.0 (#1159)
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
2026-02-03 12:24:45 +08:00
f6cfc0d1e3 [BugFix] Fix Qwen3 Omni talker mtp torch.compile startup error (#1104)
Signed-off-by: ram16g <anlianfengjie@163.com>
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
Co-authored-by: ram16g <anlianfengjie@163.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-01-30 23:25:49 +08:00
Ziming HuangandHongsheng Liu a909d36415 [Perf] Qwen3 Omni talker mtp optimization (#1005)
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-01-29 21:23:19 +08:00
汪志鹏 741f7e2826 [Core]: KV Cache Transfer Encapsulation (#979)
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
2026-01-28 23:12:45 +08:00
Canlin Guo c9392dff01 [Hardware] Support platforms and plugin system (#774)
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
2026-01-28 22:12:52 +08:00
Canlin Guo cb48968294 [Bugfix] Modify output to model_runner_output (#1026)
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
2026-01-28 18:42:39 +08:00
Canlin Guo cadd77262d [NPU] Upgrade to v0.14.0 (#820)
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
2026-01-27 16:45:32 +08:00
52bf1e2f40 [Feature]Support async computation and communication across stages by chunks (#727)
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: CHEN <116010019@link.cuhk.edu.cn>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-01-26 11:21:24 +08:00
汪志鹏andwzliu 7f821be57b Support Bagel Model (#726)
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: wzliu <wzliu@connect.hku.hk>
2026-01-22 23:09:55 +13:00
Gao Han a54c323952 [Model] Support Qwen3-TTS model series (#895)
Signed-off-by: Gaohan123 <hgaoaf@connect.ust.hk>
2026-01-22 16:57:37 +08:00
+13 776c3a794e Dev/rebase 0.14.0 (#847)
Signed-off-by: JaredforReal <w13431838023@gmail.com>
Signed-off-by: root <root@hk01dgx039.cm.cluster>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: TangPeng <85704592@qq.com>
Signed-off-by: XU Mingshi <91017482+mxuax@users.noreply.github.com>
Signed-off-by: mxuax <mxuax@connect.ust.hk>
Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: iwzbi <wzbi@zju.edu.cn>
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
Signed-off-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com>
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: linyueqian <linyueqian@outlook.com>
Signed-off-by: David Chen <530634352@qq.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: wangyu31577 <wangyu31577@hundsun.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: John Liu BUAA <liukecheng97@gmail.com>
Signed-off-by: Dinesh G <G.Dinesh@ibm.com>
Signed-off-by: gDINESH13 <dinesh13g@gmail.com>
Signed-off-by: root <root@hk01dgx028.cm.cluster>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: JaredforReal <w13431838023@gmail.com>
Co-authored-by: root <root@hk01dgx039.cm.cluster>
Co-authored-by: JustQJ <37905360+JustQJ@users.noreply.github.com>
Co-authored-by: XU Mingshi <91017482+mxuax@users.noreply.github.com>
Co-authored-by: Sihyeon Jang <uneedsihyeon@gmail.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Co-authored-by: catcat <108673086+iwzbi@users.noreply.github.com>
Co-authored-by: Ziming Huang <hzm414167@alibaba-inc.com>
Co-authored-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com>
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com>
Co-authored-by: Samit <285365963@qq.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com>
Co-authored-by: wangyu31577 <wangyu31577@hundsun.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: John Liu BUAA <liukecheng97@gmail.com>
Co-authored-by: D!NE$H <67671800+gDINESH13@users.noreply.github.com>
Co-authored-by: root <root@hk01dgx028.cm.cluster>
2026-01-21 00:25:25 +08:00
Ziming Huang b5cf8806f0 [BugFix] Fix incorrect mrope positions under cuda graph (#803)
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
2026-01-16 03:24:21 +13:00
Ziming Huang 1444e1fc45 [Feature] Support Qwen3 Omni talker mtp batch inference (#722)
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
2026-01-14 19:07:31 +08:00
Zhou Taichangandroot 2e13bddf87 [Debug] Clean code in Qwen 3 Omni and add warning for talker temperature. (#688)
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: root <root@hk01dgx028.cm.cluster>
2026-01-12 23:13:45 +13:00
汪志鹏 0ec2da08de [BugFix] Fix assuming all stage model have talker (#730)
Signed-off-by: princepride <wangzhipeng628@gmail.com>
2026-01-11 01:36:37 +13:00
Ziming Huang 978e3ad7d4 [Feature] Support Qwen3 Omni talker cudagraph (#669)
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
2026-01-09 11:45:58 +08:00
Canlin Guo 46afa046ce [NPU] Support mixed modalities for Qwen3-Omni (#537)
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
2025-12-31 16:15:20 +08:00
Ziming HuangandHongsheng Liu 1ae0c64fe0 Support Qwen3 Omni thinker cuda graph (#523)
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2025-12-30 23:46:13 +13:00
Zhou Taichang 59ae174bba [Debug] Debug qwen3 mix modality output empty string (#431)
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2025-12-30 15:32:15 +08:00
Canlin Guo be81443c8a [Bugfix][NPU] Add _model_forward for ModelRunner (#505)
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
2025-12-28 04:09:34 +08:00
dsinghviandHongsheng Liu df4909630c model forward call logic fixes (#495)
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2025-12-27 14:22:07 +00:00
595e7c0032 [Core] Supports stage abstraction in the diffusion model (#391)
Signed-off-by: Chenguang ZHENG <645327136@qq.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Co-authored-by: yinpeiqi <yinpeiqi809@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2025-12-27 01:59:33 +13:00
dsinghviandHongsheng Liu abde501dfa [Misc] Fix type interface mismatch (#478)
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2025-12-26 15:28:49 +08:00
Canlin GuoandHongsheng Liu fe3d9a6a81 [NPU][Upgrade] Upgrade to v0.12.0 (#447)
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2025-12-26 14:10:23 +08:00
Canlin Guo e1f9691120 [Clean] Remove the redundant decoding payloads logic (#404)
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
2025-12-26 04:48:34 +13:00
Peiqi Yin e9e6b3711c [Core] customize process between stages, remove the need of "request_ids" in model forward, rebase to 0.12.0 (#458)
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
2025-12-24 13:26:33 +00:00
8939eb835d [Rebase] Rebase to vllm 0.12.0 (#335)
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
2025-12-19 16:42:02 +08:00
Lancer 01549fb953 [Diffusion] fix warning in diffusion stage (#332)
Signed-off-by: Lancer <maruixiang6688@gmail.com>
2025-12-16 09:07:16 +00:00
Cyrus Leung c091a03d82 [Chore] Bump Python version and apply formatting changes for v0.12 release (#126)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
2025-12-16 15:19:40 +08:00
NATURE f995211da0 [Feature] Omni Connector + ray supported (#215)
Signed-off-by: wzliu <wzliu@connect.hku.hk>
2025-12-10 20:15:03 +08:00
Peiqi Yin bbb5a72871 [BugFix] Fix qwen3omni thinker batching. (#207)
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
2025-12-08 04:44:13 +00:00
Canlin Guo 6079b14040 [NPU][Refactor] Rename Diffusion* to Generation* (#211)
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
2025-12-05 08:23:37 +00:00
Zhou TaichangandGao Han c70b167f70 [Engine] Remove Diffusion_XX which duplicates with Generation_XX (#163)
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Gao Han <gaohan19@huawei.com>
2025-12-05 03:04:23 +00:00
Alicia a33bbcedea [Fix] Resolve comments & update vLLM-Omni name usages. (#122)
Signed-off-by: WANG Cong <115451386+congw729@users.noreply.github.com>
2025-12-01 08:05:24 +08:00
Canlin Guo ec9e04027d [Feat] Add NPU Backend support for vLLM-Omni (#89)
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
2025-11-30 20:39:06 +08:00
Zhou Taichang 228f4e5070 [Bug-fix] Fix Bugs in Qwen3/Qwen2.5 Omni Rebased Support (#114)
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2025-11-30 18:15:10 +08:00
R2-YandRein 0a4e5a8710 [Feature] add support for Qwen3-omni (#55)
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Rein <rein@Reins-MacBook-Air.local>
2025-11-30 04:44:01 +08:00
Roger WangandGao Han a5800cd2c2 [Doc] Misc documentation polishing (#98)
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: Gao Han <gaohan19@huawei.com>
Co-authored-by: Gao Han <gaohan19@huawei.com>
2025-11-29 20:42:39 +08:00
Gao Han db81ee9f69 [Docs] optimize and supplement docs system (#86)
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
2025-11-28 11:03:52 +08:00