tzhouam
3a508d7b7a
rebase: align vllm-omni with vLLM e9163b536e72
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-03-16 02:32:39 +00:00
tzhouam
62c8695dfd
Update GPUGenerationModelRunner to clarify KV connector metadata handling during draft model runs
...
- Changed variable name to better reflect its purpose in managing KV connector metadata.
- Updated comments for improved clarity regarding the handling of speculative configurations.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-03-13 03:16:33 +00:00
tzhouam
e17fb0e97e
Refactor OpenAI serving components and enhance multimodal handling
...
- Updated import for OpenAIServingEmbedding in api_server.py.
- Enhanced omni_init_app_state to initialize renderer for engine_client.
- Added handle_oov_mm_token parameter to multiple model classes for better multimodal token handling.
- Improved comments for clarity in various model files.
- Adjusted GPUARModelRunner to ensure proper handling of late interaction runner attributes.
Signed-off-by: [Your Name] <[Your Email]>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-03-13 01:55:17 +00:00
tzhouam
08a2673c66
rebase: align vllm-omni with vLLM 84e436ed1c94
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-03-11 07:29:52 +00:00
Junhong Liu
2ee4a07adb
[Refactor][Perf] Qwen3-omni: code predictor with re-prefill + SDPA and eliminate decode hot-path CPU round-trips ( #1758 )
...
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com >
2026-03-11 07:52:21 +08:00
汪志鹏
7271116973
[Bagel]: Support multistage img2img ( #1669 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com >
2026-03-09 22:23:50 +08:00
89b527e533
Dev/rebase v0170 ( #1639 )
...
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com >
2026-03-09 17:29:38 +08:00
1612948062
add support for MammothModa2 model ( #336 )
...
Signed-off-by: HonestDeng <2958906959@qq.com >
Signed-off-by: iwzbi <iwzbi@zju.edu.cn >
Signed-off-by: iwzbi <wzbi@zju.edu.cn >
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com >
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: catcat <108673086+iwzbi@users.noreply.github.com >
Co-authored-by: iwzbi <iwzbi@zju.edu.cn >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: dsinghvi <divyanshsinghvi@gmail.com >
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com >
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
2026-03-04 22:52:56 +08:00
Yueqian Lin
a3240fd484
[Bugfix] Fix transformers 5.x compat issues in online TTS serving ( #1536 )
...
Signed-off-by: linyueqian <linyueqian@outlook.com >
2026-03-04 01:28:32 +08:00
Junhong Liu
742d845b8e
Make chunk_size and left_context_size configurable via YAML for async chunking ( #1423 )
...
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com >
Signed-off-by: Junhong Liu <ljh_lbj@163.com >
2026-03-03 22:06:52 +08:00
rein yang
f082f6eedb
[Bugfix] fix kernel error for qwen3-omni ( #1602 )
...
Signed-off-by: Rein Yang <ruiruyang2@gmail.com >
2026-03-02 18:12:30 +08:00
dsinghvi and Zhou Taichang
68764cc4f4
[Refactor]: Phase1 for rebasing_additional_info ( #1394 )
...
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com >
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
Co-authored-by: Zhou Taichang <tzhouam@connect.ust.hk >
2026-03-02 10:19:41 +08:00
+13
b19c227fb3
[New Model]: XiaomiMiMo/MiMo-Audio-7B-Instruct support ( #750 )
...
Signed-off-by: wangyu31577 <wangyu31577@hundsun.com >
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com >
Signed-off-by: hsliu <liuhongsheng4@huawei.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Signed-off-by: GG-li <3226868735@qq.com >
Signed-off-by: Sihao Li <111170255+GG-li@users.noreply.github.com >
Signed-off-by: XU Mingshi <91017482+mxuax@users.noreply.github.com >
Signed-off-by: mxuax <mxuax@connect.ust.hk >
Signed-off-by: Baoyuan Qi <qibaoyuan@126.com >
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Signed-off-by: wuzhongjian <wuzhongjian_yewu@cmss.chinamobile.com >
Signed-off-by: dongbo910220 <1275604947@qq.com >
Signed-off-by: dongbo910220 <32610838+dongbo910220@users.noreply.github.com >
Signed-off-by: Jiangyun Zhu <riverclouds.zhu@qq.com >
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com >
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com >
Signed-off-by: baoyuan qi <qibaoyuan@126.com >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: Prajwal A <prajwalanagani@gmail.com >
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com >
Signed-off-by: 丁宁 <nndding@gmail.com >
Signed-off-by: SHIJIN ZHANG <75300765+Dovis01@users.noreply.github.com >
Signed-off-by: dingning<dingning7@xiaomi.com >
Signed-off-by: dingning <dingning7@xiaomi.com >
Signed-off-by: dingning <dingning@xiaomi.com >
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com >
Co-authored-by: wangyu31577 <wangyu31577@hundsun.com >
Co-authored-by: Zhang Shijin <zhangshijin@xiaomi.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Sihao Li <111170255+GG-li@users.noreply.github.com >
Co-authored-by: XU Mingshi <91017482+mxuax@users.noreply.github.com >
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com >
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Co-authored-by: JohnJan <wuzhongjian_yewu@cmss.chinamobile.com >
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com >
Co-authored-by: dongbo910220 <32610838+dongbo910220@users.noreply.github.com >
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com >
Co-authored-by: Junhong Liu <ljh_lbj@163.com >
Co-authored-by: TJian <tunjian.tan@embeddedllm.com >
Co-authored-by: shijin zhang <zsj1364226740@gmail.com >
Co-authored-by: Zhou Taichang <tzhouam@connect.ust.hk >
Co-authored-by: root <root@hk01dgx028.cm.cluster >
Co-authored-by: Prajwal A <34590600+LawJarp-A@users.noreply.github.com >
Co-authored-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com >
Co-authored-by: dingning <dingning7@xiaomi.com >
Co-authored-by: ning ding <nndding@gmail.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2026-02-27 11:19:49 +08:00
4146095623
[Perf] GLM Image ( #920 )
...
Signed-off-by: JaredforReal <w13431838023@gmail.com >
Signed-off-by: Jared Wen <w13431838023@gmail.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-02-27 10:29:14 +08:00
amy-why-3459
66c3e04263
[Performance]Qwen3-Omni performance optimization ( #1378 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-02-26 21:21:36 +08:00
SHIJIN ZHANG
466a7f7a62
[Bugfix]: Fix merging updated additional information to ensure dict type ( #1296 )
...
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com >
2026-02-26 00:06:49 +08:00
Zhou Taichang
296d995ec2
[CI] Remove overwhelming debug log ( #1463 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-25 12:06:01 +08:00
Canlin Guo and Hongsheng Liu
af11b02081
[Minor][Refactor] Pass seq_token_counts explicitly ( #1425 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-02-24 14:10:00 +08:00
Sy03 and Hongsheng Liu
efbe4116ff
[Feature] : Support disaggregated inference pipeline for Qwen3_TTS ( #1161 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-02-21 07:36:44 +08:00
75770c9309
[Rebase] Rebase to vllm v0.16.0 ( #1357 )
...
ComfyUI Integration -- Publish to Comfy registry / Publish Custom Node to registry (push) Failing after 44s
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: ZJY0516 <zhu.jiangyun@foxmail.com >
Co-authored-by: youkaichao <youkaichao@gmail.com >
2026-02-13 18:05:07 +08:00
0b56fba7b3
[Pref] Support Qwen3 Omni code2wav batch infernce with async chunk ( #1246 )
...
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Signed-off-by: Ziming Huang <1520787127@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2026-02-12 11:50:10 +08:00
Zhou Taichang
a854f0b7ff
[Debug] Add trigger to concurrent stage init ( #1274 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-11 14:18:18 +08:00
dsinghvi
59024bee62
Process-Scoped GPU Memory Accounting ( #1204 )
...
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com >
2026-02-08 22:43:02 +08:00
amy-why-3459
5304bc1e16
[Refactor] Refactor async chunk and fix the shape mismatch issue ( #1151 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-02-04 00:00:56 +08:00
Zhou Taichang and 汪志鹏
70ef548448
Dev/rebase v0.15.0 ( #1159 )
...
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
2026-02-03 12:24:45 +08:00
f6cfc0d1e3
[BugFix] Fix Qwen3 Omni talker mtp torch.compile startup error ( #1104 )
...
Signed-off-by: ram16g <anlianfengjie@163.com >
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
Co-authored-by: ram16g <anlianfengjie@163.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-01-30 23:25:49 +08:00
Ziming Huang and Hongsheng Liu
a909d36415
[Perf] Qwen3 Omni talker mtp optimization ( #1005 )
...
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-01-29 21:23:19 +08:00
汪志鹏
741f7e2826
[Core]: KV Cache Transfer Encapsulation ( #979 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com >
2026-01-28 23:12:45 +08:00
Canlin Guo
c9392dff01
[Hardware] Support platforms and plugin system ( #774 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
2026-01-28 22:12:52 +08:00
Canlin Guo
cb48968294
[Bugfix] Modify output to model_runner_output ( #1026 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
2026-01-28 18:42:39 +08:00
Canlin Guo
cadd77262d
[NPU] Upgrade to v0.14.0 ( #820 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
2026-01-27 16:45:32 +08:00
52bf1e2f40
[Feature]Support async computation and communication across stages by chunks ( #727 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
Signed-off-by: Rein Yang <ruiruyang2@gmail.com >
Co-authored-by: Rein Yang <ruiruyang2@gmail.com >
Co-authored-by: CHEN <116010019@link.cuhk.edu.cn >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-01-26 11:21:24 +08:00
汪志鹏 and wzliu
7f821be57b
Support Bagel Model ( #726 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Co-authored-by: wzliu <wzliu@connect.hku.hk >
2026-01-22 23:09:55 +13:00
Gao Han
a54c323952
[Model] Support Qwen3-TTS model series ( #895 )
...
Signed-off-by: Gaohan123 <hgaoaf@connect.ust.hk >
2026-01-22 16:57:37 +08:00
+13
776c3a794e
Dev/rebase 0.14.0 ( #847 )
...
Signed-off-by: JaredforReal <w13431838023@gmail.com >
Signed-off-by: root <root@hk01dgx039.cm.cluster >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
Signed-off-by: TangPeng <85704592@qq.com >
Signed-off-by: XU Mingshi <91017482+mxuax@users.noreply.github.com >
Signed-off-by: mxuax <mxuax@connect.ust.hk >
Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: iwzbi <wzbi@zju.edu.cn >
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
Signed-off-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com >
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com >
Signed-off-by: samithuang <285365963@qq.com >
Signed-off-by: linyueqian <linyueqian@outlook.com >
Signed-off-by: David Chen <530634352@qq.com >
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com >
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Signed-off-by: wangyu31577 <wangyu31577@hundsun.com >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: John Liu BUAA <liukecheng97@gmail.com >
Signed-off-by: Dinesh G <G.Dinesh@ibm.com >
Signed-off-by: gDINESH13 <dinesh13g@gmail.com >
Signed-off-by: root <root@hk01dgx028.cm.cluster >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
Co-authored-by: JaredforReal <w13431838023@gmail.com >
Co-authored-by: root <root@hk01dgx039.cm.cluster >
Co-authored-by: JustQJ <37905360+JustQJ@users.noreply.github.com >
Co-authored-by: XU Mingshi <91017482+mxuax@users.noreply.github.com >
Co-authored-by: Sihyeon Jang <uneedsihyeon@gmail.com >
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com >
Co-authored-by: catcat <108673086+iwzbi@users.noreply.github.com >
Co-authored-by: Ziming Huang <hzm414167@alibaba-inc.com >
Co-authored-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com >
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com >
Co-authored-by: Samit <285365963@qq.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com >
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com >
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com >
Co-authored-by: wangyu31577 <wangyu31577@hundsun.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: John Liu BUAA <liukecheng97@gmail.com >
Co-authored-by: D!NE$H <67671800+gDINESH13@users.noreply.github.com >
Co-authored-by: root <root@hk01dgx028.cm.cluster >
2026-01-21 00:25:25 +08:00
Ziming Huang
b5cf8806f0
[BugFix] Fix incorrect mrope positions under cuda graph ( #803 )
...
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
2026-01-16 03:24:21 +13:00
Ziming Huang
1444e1fc45
[Feature] Support Qwen3 Omni talker mtp batch inference ( #722 )
...
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
2026-01-14 19:07:31 +08:00
Zhou Taichang and root
2e13bddf87
[Debug] Clean code in Qwen 3 Omni and add warning for talker temperature. ( #688 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Co-authored-by: root <root@hk01dgx028.cm.cluster >
2026-01-12 23:13:45 +13:00
汪志鹏
0ec2da08de
[BugFix] Fix assuming all stage model have talker ( #730 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
2026-01-11 01:36:37 +13:00
Ziming Huang
978e3ad7d4
[Feature] Support Qwen3 Omni talker cudagraph ( #669 )
...
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
2026-01-09 11:45:58 +08:00
Canlin Guo
46afa046ce
[NPU] Support mixed modalities for Qwen3-Omni ( #537 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
2025-12-31 16:15:20 +08:00
Ziming Huang and Hongsheng Liu
1ae0c64fe0
Support Qwen3 Omni thinker cuda graph ( #523 )
...
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2025-12-30 23:46:13 +13:00
Zhou Taichang
59ae174bba
[Debug] Debug qwen3 mix modality output empty string ( #431 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2025-12-30 15:32:15 +08:00
Canlin Guo
be81443c8a
[Bugfix][NPU] Add _model_forward for ModelRunner ( #505 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
2025-12-28 04:09:34 +08:00
dsinghvi and Hongsheng Liu
df4909630c
model forward call logic fixes ( #495 )
...
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2025-12-27 14:22:07 +00:00
595e7c0032
[Core] Supports stage abstraction in the diffusion model ( #391 )
...
Signed-off-by: Chenguang ZHENG <645327136@qq.com >
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com >
Co-authored-by: yinpeiqi <yinpeiqi809@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2025-12-27 01:59:33 +13:00
dsinghvi and Hongsheng Liu
abde501dfa
[Misc] Fix type interface mismatch ( #478 )
...
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2025-12-26 15:28:49 +08:00
Canlin Guo and Hongsheng Liu
fe3d9a6a81
[NPU][Upgrade] Upgrade to v0.12.0 ( #447 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2025-12-26 14:10:23 +08:00
Canlin Guo
e1f9691120
[Clean] Remove the redundant decoding payloads logic ( #404 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
2025-12-26 04:48:34 +13:00
Peiqi Yin
e9e6b3711c
[Core] customize process between stages, remove the need of "request_ids" in model forward, rebase to 0.12.0 ( #458 )
...
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com >
2025-12-24 13:26:33 +00:00
tzhouam