anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
873 Commits 235 Branches 31 Tags
3e539fcb6a017fa9d8ac86f4b8d6eb852ff0dde4
Commit Graph
138 Commits
Author SHA1 Message Date
tzhouam 3e539fcb6a Merge remote-tracking branch 'origin/main' into dev/vllm-align 
Made-with: Cursor

# Conflicts:
#	vllm_omni/entrypoints/openai/serving_chat.py
#	vllm_omni/worker/gpu_model_runner.py

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 02:57:27 +00:00
Baoyuan Qi f371f7bf05 [bugfix] Fix unexpected argument 'is_finished' in function llm2code2wav_async_chunk of mimo-audio (#1570) 
Signed-off-by: Baoyuan Qi <qibaoyuan@126.com>
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2026-03-02 21:49:06 +08:00
rein yang f082f6eedb [Bugfix] fix kernel error for qwen3-omni (#1602) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
 2026-03-02 18:12:30 +08:00
汪志鹏andCursor e37a89ffcb [Feature]: Support cfg kv-cache transfer in multi-stage (#1422) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-03-02 05:06:28 +00:00
dsinghviandZhou Taichang 68764cc4f4 [Refactor]: Phase1 for rebasing_additional_info (#1394) 
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com>
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: Zhou Taichang <tzhouam@connect.ust.hk>
 2026-03-02 10:19:41 +08:00
tzhouam 26744bcb35 Merge origin/main into dev/vllm-align 
Resolved conflicts:
- vllm_omni/inputs/preprocess.py: took origin/main's elif mm_processor_kwargs branch, fixed undefined mm_uuids reference
- vllm_omni/model_executor/models/qwen3_tts/qwen3_tts_code_predictor_vllm.py: took origin/main's NPU support

Made-with: Cursor

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-02 02:10:32 +00:00
汪志鹏 3d9fa8d53f [BugFix]: fix a lot of bug (#1565) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2026-02-28 16:26:56 +08:00
wangyu cd2234a1bf [CI] Modify some CI test cases to run on L4 environment to reduce H100 resource usage. (#1543) 
Signed-off-by: yenuo26 <410167048@qq.com>
Signed-off-by: wangyu <53896905+yenuo26@users.noreply.github.com>
 2026-02-28 14:41:51 +08:00
Canlin Guo c81266703a [NPU][Bugfix] Align GPU side and recover qwen3-tts (#1564) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-02-28 12:46:13 +08:00
Zhou Taichang fe0c02ee78 [Debug] Merge vllm pull 35368 (#1534) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-28 10:39:25 +08:00
Junhong Liu 42d86af1f1 Fix no embed text spk tokens (#1540) 
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
 2026-02-28 09:33:58 +08:00
Sy03 0cc3481acf [Feat] : Support Async chunk cleanup (#1087) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-02-27 14:39:33 +08:00
+13
Baoyuan Qiwangyuwangyu31577Zhang ShijinHongsheng LiuSihao LiXU MingshiCanlin GuoIsotr0pyJohnJanWeiQing Chendongbo910220Jiangyun ZhuJunhong LiuTJianshijin zhangZhou TaichangrootPrajwal AShijin Zhangdingningning dingCopilot
 		b19c227fb3 [New Model]: XiaomiMiMo/MiMo-Audio-7B-Instruct support (#750) 
Signed-off-by: wangyu31577 <wangyu31577@hundsun.com>
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
Signed-off-by: hsliu <liuhongsheng4@huawei.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: GG-li <3226868735@qq.com>
Signed-off-by: Sihao Li <111170255+GG-li@users.noreply.github.com>
Signed-off-by: XU Mingshi <91017482+mxuax@users.noreply.github.com>
Signed-off-by: mxuax <mxuax@connect.ust.hk>
Signed-off-by: Baoyuan Qi <qibaoyuan@126.com>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: wuzhongjian <wuzhongjian_yewu@cmss.chinamobile.com>
Signed-off-by: dongbo910220 <1275604947@qq.com>
Signed-off-by: dongbo910220 <32610838+dongbo910220@users.noreply.github.com>
Signed-off-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Signed-off-by: baoyuan qi <qibaoyuan@126.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Prajwal A <prajwalanagani@gmail.com>
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com>
Signed-off-by: 丁宁 <nndding@gmail.com>
Signed-off-by: SHIJIN ZHANG <75300765+Dovis01@users.noreply.github.com>
Signed-off-by: dingning<dingning7@xiaomi.com>
Signed-off-by: dingning <dingning7@xiaomi.com>
Signed-off-by: dingning <dingning@xiaomi.com>
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com>
Co-authored-by: wangyu31577 <wangyu31577@hundsun.com>
Co-authored-by: Zhang Shijin <zhangshijin@xiaomi.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Sihao Li <111170255+GG-li@users.noreply.github.com>
Co-authored-by: XU Mingshi <91017482+mxuax@users.noreply.github.com>
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: JohnJan <wuzhongjian_yewu@cmss.chinamobile.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Co-authored-by: dongbo910220 <32610838+dongbo910220@users.noreply.github.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Co-authored-by: Junhong Liu <ljh_lbj@163.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
Co-authored-by: shijin zhang <zsj1364226740@gmail.com>
Co-authored-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: root <root@hk01dgx028.cm.cluster>
Co-authored-by: Prajwal A <34590600+LawJarp-A@users.noreply.github.com>
Co-authored-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com>
Co-authored-by: dingning <dingning7@xiaomi.com>
Co-authored-by: ning ding <nndding@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2026-02-27 11:19:49 +08:00
Xu Luandxulusjb c9b31a4c1a feat(qwen3-tts): Add CUDA Graph support for speech tokenizer decoder (#1205) 
Signed-off-by: xulusjb <fdukeshik@gmail.com>
Co-authored-by: xulusjb <fdukeshik@gmail.com>
 2026-02-27 10:48:25 +08:00
Jared Wen汪志鹏Hongsheng Liu
 		4146095623 [Perf] GLM Image (#920) 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
Signed-off-by: Jared Wen <w13431838023@gmail.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-27 10:29:14 +08:00
amy-why-3459Hongsheng LiuCanlin Guo
 		e1337c89c7 [BugFix] Restore talker's config (#1524) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Canlin Guo <961750412@qq.com>
 2026-02-27 06:44:11 +08:00
amy-why-3459 66c3e04263 [Performance]Qwen3-Omni performance optimization (#1378) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-02-26 21:21:36 +08:00
Juan Pablo ZuluagaGao HanHongsheng Liu
 		66457c39fa [Bugfix][Qwen3TTS] (#1289) 
Signed-off-by: pablo <juanz9312@gmail.com>
Co-authored-by: Gao Han <gaohan19@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-26 18:58:35 +08:00
tzhouam 8d0c46487e Merge origin/main into dev/vllm-align 
Resolve conflicts:
- docker/Dockerfile.ci: Keep wheel install section, update hash to 6042e66cd5
- vllm_omni/platforms/npu/worker/npu_model_runner.py: Accept main's refactoring

Made-with: Cursor

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-26 08:19:53 +00:00
Yuanheng Zhao 904b97d5bd [Fix][Chore] Qwen3-TTS Modeling Minor Code Sanity Improvements (#1482) 
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
 2026-02-26 14:39:08 +08:00
Canlin Guo 22c8cde328 [NPU] Upgrade to v0.16.0 (#1375) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-02-26 14:26:50 +08:00
Sangchun Ha 315daeba8c [Bugfix] Fix case-sensitive task_type matching in Qwen3TTSModelForGeneration (#1455) 
Signed-off-by: Sangchun Ha <seomk9896@gmail.com>
 2026-02-26 10:27:13 +08:00
Zhou Taichang 8e950daf6a [Debug] Multi-Request for Qwen 3 Omni use_audio_in_video (#1433) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-26 09:26:25 +08:00
Yanick Schraner 97f4c5b1e7 [Bugfix] Add TTS configuration options (#1177) 
Signed-off-by: Yanick Schraner <yanick.schraner@bs.ch>
 2026-02-26 00:14:44 +08:00
tzhouam 22e01c416b Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-02-25 08:15:55 +00:00
Juan Pablo Zuluagaandpablo 4de077ea62 [Qwen3TTS][Feat] Code2Wav batched decoding (#1426) 
Signed-off-by: pablo <pablo@agigo.ai>
Co-authored-by: pablo <pablo@agigo.ai>
 2026-02-25 11:16:46 +13:00
Canlin GuoandHongsheng Liu af11b02081 [Minor][Refactor] Pass seq_token_counts explicitly (#1425) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-24 14:10:00 +08:00
tzhouam e22ae62396 Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-02-24 03:23:20 +00:00
tzhouam 9070f15f52 debug qwen tts 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-24 03:17:11 +00:00
NATURE 1589931fa6 [Feature] vLLM-Omni RDMA connector (#1019) 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
 2026-02-24 11:13:52 +08:00
tzhouam bf69cd94bb Rebase 2026/02/23 
Update Dockerfile for vllm wheel version and enhance input processing in Omni models. Added decoding for byte messages in async requests, included new metrics in benchmarks, and improved pause state handling in the scheduler. Refined input and output processing methods for better multimodal support and error handling.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-23 03:36:01 +00:00
Sy03andHongsheng Liu efbe4116ff [Feature] : Support disaggregated inference pipeline for Qwen3_TTS (#1161) 
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-21 07:36:44 +08:00
Zhou Taichang汪志鹏Hongsheng LiuIsotr0pyZJY0516youkaichao
 		75770c9309 [Rebase] Rebase to vllm v0.16.0 (#1357)
ComfyUI Integration -- Publish to Comfy registry / Publish Custom Node to registry (push) Failing after 44s
Details 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: ZJY0516 <zhu.jiangyun@foxmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2026-02-13 18:05:07 +08:00
rein yangandHongsheng Liu 36a79717c6 change qwen3-omni open cudagraph by default (#1352) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-12 20:49:34 +08:00
ChenWenjingBhanu068Gao Han
 		ab1acb27ff [Feature]async scheduling to overlap chunk IO and compute (#951) 
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn>
Co-authored-by: Bhanu068 <voutharoja.bhanu06@gmail.com>
Co-authored-by: Gao Han <gaohan19@huawei.com>
 2026-02-12 16:05:02 +08:00
Ziming HuangHongsheng LiuCopilot
 		0b56fba7b3 [Pref] Support Qwen3 Omni code2wav batch infernce with async chunk (#1246) 
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: Ziming Huang <1520787127@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2026-02-12 11:50:10 +08:00
Ekagra Ranjan dd302b63c4 [Bugfix][Qwen3-TTS] Fix task type (#1317) 
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
 2026-02-11 15:49:15 +08:00
rein yang 963b64e7ae [Bugfix] fix precision issues of qwen3-omni when enable async_chunk without system prompt (#1288) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
 2026-02-11 10:03:41 +08:00
zhou zhuoxinandCopilot 109bb97b2c [Fix]Ensure HuggingFace downloads complete before initialization. (#1213) 
Signed-off-by: zhou zhuoxin <zhouzhuoxin1508@outlook.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2026-02-10 16:05:12 +08:00
Yueqian Lin 53a44c56fe [Bugfix] Fix padding bug in 12Hz tokenizer ConvTranspose1d decode (#1241) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-02-08 00:35:49 +08:00
Canlin Guo 285d71b26a [Refactor] Reuse upstream Qwen3MoeSparseMoeBlock (#1202) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-02-06 01:06:50 +00:00
Zhou Taichang f1960d8dfd [Feature] Enable use_audio_in_video for Qwen 3 Omni Online (#1198) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-04 20:48:25 +08:00
amy-why-3459 5304bc1e16 [Refactor] Refactor async chunk and fix the shape mismatch issue (#1151) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-02-04 00:00:56 +08:00
Zhou Taichangand汪志鹏 70ef548448 Dev/rebase v0.15.0 (#1159) 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-02-03 12:24:45 +08:00
汪志鹏andHongsheng Liu de0a8b057b [CI]: Bagel E2E Smoked Test (#1074) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-31 22:08:22 +13:00
Markus / Mark 425952e7d3 [BugFix] Fix Qwen3 TTS 0.6B profile run hang (#995) (#1082) 2026-01-31 08:00:03 +08:00
Ziming Huangram16gHongsheng Liu
 		f6cfc0d1e3 [BugFix] Fix Qwen3 Omni talker mtp torch.compile startup error (#1104) 
Signed-off-by: ram16g <anlianfengjie@163.com>
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
Co-authored-by: ram16g <anlianfengjie@163.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-30 23:25:49 +08:00
amy-why-3459andHongsheng Liu e6b9d52905 [BugFix] Modify max_tokens and modify the log and fix #1103 (#1097) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-31 01:50:23 +13:00
Ding Zuhao 0e07eb6ff9 [Doc][Bagel] Add BAGEL-7B-MoT documentation and edit the default stage configuration (#987) 
Signed-off-by: Ding Zuhao <e1583181@u.nus.edu>
Signed-off-by: jzz <e1583181@u.nus.edu>
 2026-01-30 12:56:10 +13:00
Fanli LinandCopilot bfbf3e5b69 [Feat] Support XPU Backend in vLLM-Omni (#191) 
Signed-off-by: Fanli Lin <fanli.lin@intel.com>
Signed-off-by: Fanli Lin <fanli0116@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2026-01-30 04:33:05 +13:00