anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
1,317 Commits 235 Branches 31 Tags
b5ddff72429b2bf2485bfc0e98d2c28b4ec80310
Commit Graph
201 Commits
Author SHA1 Message Date
Zhou Taichang e3b6254090 debug Qwen TTS (#902) 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-01-22 15:28:56 +00:00
汪志鹏andwzliu 7f821be57b Support Bagel Model (#726) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: wzliu <wzliu@connect.hku.hk>
 2026-01-22 23:09:55 +13:00
Gao Han a54c323952 [Model] Support Qwen3-TTS model series (#895) 
Signed-off-by: Gaohan123 <hgaoaf@connect.ust.hk>
 2026-01-22 16:57:37 +08:00
John Liu BUAA c3894585b5 [Perf] Fuse Q/K/V Linear with QKVParallelLinear in Qwen2.5Omni DiTAttention (#884) 
Signed-off-by: John Liu BUAA <liukecheng97@gmail.com>
 2026-01-21 09:24:05 +00:00
+13
Zhou TaichangJaredforRealrootJustQJXU MingshiSihyeon JangJiangyun ZhucatcatZiming HuangYuhan LiuAliciaSamitYueqian LinWeiQing ChenPeiqi YinDidan DengHongsheng Liuwangyuwangyu31577汪志鹏John Liu BUAAD!NE$Hroot
 		776c3a794e Dev/rebase 0.14.0 (#847) 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
Signed-off-by: root <root@hk01dgx039.cm.cluster>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: TangPeng <85704592@qq.com>
Signed-off-by: XU Mingshi <91017482+mxuax@users.noreply.github.com>
Signed-off-by: mxuax <mxuax@connect.ust.hk>
Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: iwzbi <wzbi@zju.edu.cn>
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
Signed-off-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com>
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: linyueqian <linyueqian@outlook.com>
Signed-off-by: David Chen <530634352@qq.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: wangyu31577 <wangyu31577@hundsun.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: John Liu BUAA <liukecheng97@gmail.com>
Signed-off-by: Dinesh G <G.Dinesh@ibm.com>
Signed-off-by: gDINESH13 <dinesh13g@gmail.com>
Signed-off-by: root <root@hk01dgx028.cm.cluster>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: JaredforReal <w13431838023@gmail.com>
Co-authored-by: root <root@hk01dgx039.cm.cluster>
Co-authored-by: JustQJ <37905360+JustQJ@users.noreply.github.com>
Co-authored-by: XU Mingshi <91017482+mxuax@users.noreply.github.com>
Co-authored-by: Sihyeon Jang <uneedsihyeon@gmail.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Co-authored-by: catcat <108673086+iwzbi@users.noreply.github.com>
Co-authored-by: Ziming Huang <hzm414167@alibaba-inc.com>
Co-authored-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com>
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com>
Co-authored-by: Samit <285365963@qq.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com>
Co-authored-by: wangyu31577 <wangyu31577@hundsun.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: John Liu BUAA <liukecheng97@gmail.com>
Co-authored-by: D!NE$H <67671800+gDINESH13@users.noreply.github.com>
Co-authored-by: root <root@hk01dgx028.cm.cluster>
 2026-01-21 00:25:25 +08:00
John Liu BUAA bb24e07a7a [Perf] Optimize the Qwen2.5-Omni Model thinker-to-talker-proj with nn.Linear (#825) 
Signed-off-by: John Liu BUAA <liukecheng97@gmail.com>
 2026-01-17 19:47:11 +08:00
Peiqi Yin 35f994e255 [BugFix] Qwen2.5-omni supress end token and won't stop (#773) 
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
 2026-01-16 00:41:59 +08:00
Ziming Huang 1444e1fc45 [Feature] Support Qwen3 Omni talker mtp batch inference (#722) 
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
 2026-01-14 19:07:31 +08:00
Zhou Taichangandroot 2e13bddf87 [Debug] Clean code in Qwen 3 Omni and add warning for talker temperature. (#688) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: root <root@hk01dgx028.cm.cluster>
 2026-01-12 23:13:45 +13:00
Junhong Liu d0836d8459 [Bugfix] Fix multi-audio input shape alignment for Qwen3-Omni Thinker (#697) 
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
 2026-01-12 13:16:29 +08:00
Canlin Guo dde96c2468 [Perf][Qwen3-Omni] Fuse QKV linear and gate_up proj (#734) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-01-11 11:06:24 +08:00
Canlin GuoandIsotr0py 9c2f7467a0 [Perf] Use vLLM's SharedFusedMoE in Qwen3-Omni (#560) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-01-10 04:30:06 +00:00
Ziming Huang 978e3ad7d4 [Feature] Support Qwen3 Omni talker cudagraph (#669) 
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
 2026-01-09 11:45:58 +08:00
Ziming Huang cddcc400d3 [BugFix] Fix stage engine outputs mismatch bug in online batching (#691) 
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
 2026-01-09 04:01:57 +13:00
dengyunyang b60030092c [BugFix] token2wav code out of range (#655) 
Signed-off-by: dengyunyang <584797741@qq.com>
 2026-01-06 11:05:08 +08:00
Dong WandHongsheng Liu c1ebb60747 [Bugfix]Fix Qwen 2.5 omni hardcoded max_mel_frames to resolve shape mismatch (#543) 
Signed-off-by: Dong Wang <dongw2019@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-01 17:12:03 +13:00
Gao Han 9c7de5373a [Feature]Basic version of supporting streaming output (#367) 
Signed-off-by: Gaohan123 <hgaoaf@connect.ust.hk>
 2026-01-01 03:02:04 +13:00
Ziming HuangandHongsheng Liu 1ae0c64fe0 Support Qwen3 Omni thinker cuda graph (#523) 
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2025-12-30 23:46:13 +13:00
Zhou Taichang 59ae174bba [Debug] Debug qwen3 mix modality output empty string (#431) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-12-30 15:32:15 +08:00
Canlin GuoandHongsheng Liu 52d15c4c04 [NPU][Model] Support Qwen3-Omni on NPU (#484) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2025-12-27 00:05:55 +13:00
Peiqi Yin e9e6b3711c [Core] customize process between stages, remove the need of "request_ids" in model forward, rebase to 0.12.0 (#458) 
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
 2025-12-24 13:26:33 +00:00
Peiqi Yin da7f25e14b [BugFix] Qwen3-Omni / Qwen2.5-Omni thinker batching with audio input (#451) 
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
 2025-12-24 18:43:00 +08:00
Zhou TaichangRein YangIsotr0py
 		8939eb835d [Rebase] Rebase to vllm 0.12.0 (#335) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-12-19 16:42:02 +08:00
Cyrus Leung c091a03d82 [Chore] Bump Python version and apply formatting changes for v0.12 release (#126) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-12-16 15:19:40 +08:00
rein yang baf1086d55 [CI] Add Qwen3-omni offline UT (#216) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
 2025-12-08 18:11:39 +08:00
Peiqi Yin bbb5a72871 [BugFix] Fix qwen3omni thinker batching. (#207) 
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
 2025-12-08 04:44:13 +00:00
汪志鹏 f3c69df419 [FixBug]online serving fails for high-resolution videos (#198) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2025-12-05 02:29:01 +00:00
Roger Wang 7391db62d1 [Doc] Cleanup non-english comments (#125) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-11-30 16:45:30 -08:00
Canlin Guo ec9e04027d [Feat] Add NPU Backend support for vLLM-Omni (#89) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2025-11-30 20:39:06 +08:00
Gao Han 88c00fb271 [Bugfix] Remove mandatory flash-attn dependency and optimzie docs (#113) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-30 18:20:44 +08:00
Roger Wang 22cd976980 [Bugfix] Fix special token nothink naming. (#107) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-11-29 16:07:26 -08:00
R2-YandRein 0a4e5a8710 [Feature] add support for Qwen3-omni (#55) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Rein <rein@Reins-MacBook-Air.local>
 2025-11-30 04:44:01 +08:00
Gao Han 598b4ddb51 [Feature] support multimodal inputs with multiple requests (#76) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-24 16:02:44 +08:00
Jiangyun Zhu 5707fc78d5 [CI] fix ci and format existing code (#71) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-11-18 20:54:26 +08:00
Gao Han c5a89302f1 [Feature] Support online inference (#64) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-17 14:44:10 +08:00
Samit 8cafb4294d [Docs] Setup Documentation System and Re-organize Dependencies (#49) 
Signed-off-by: SamitHuang <285365963@qq.com>
 2025-11-15 21:45:43 +08:00
Gao Han a089dd4c4e Merge pull request #51 from tzhouam/feat/multi-request-stream-new 
[Feature] Multi Request Stream for Sync Mode
 2025-11-13 10:14:27 +08:00
tzhouam edae58b137 modify code based on review suggestions 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-11 10:03:26 +08:00
齐保元 d7618b1be4 [fix] support loading model from local path 
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2025-11-09 04:37:19 +08:00
tzhouam 6a7a27801b Port changes from feat/multi-request-stream (vs main) 2025-11-07 15:48:47 +08:00
Gaohan123 e2b803fbe6 fix a bug 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-05 13:05:12 +08:00
Gaohan123 6e1478222e Dependency refractored to vLLM v0.11.0 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-04 22:06:41 +08:00
Gaohan123 d4a364f052 fix huggingface download problem for spk_dict.pt 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-01 16:45:34 +08:00
Gaohan123 6cf73c4e80 Remove unused files and fix some bugs 2025-10-25 18:02:11 +08:00
tzhouam d45f8a6be7 debug typo 2025-10-24 17:47:49 +08:00
tzhouam 61b344dc39 Add import for OmniModelsRegistry in model executor initialization 2025-10-24 17:44:31 +08:00
Gaohan123 171f3c1ea7 fix conflicts 2025-10-24 16:49:19 +08:00
Gaohan123 22f36de34e fix error 2025-10-24 16:21:52 +08:00
Gaohan123 eafc7c44c3 fix error for pre-commit 2025-10-24 16:21:50 +08:00
tzhouam 52905a0d17 Enhance Qwen2.5-Omni model by defining TALKER_CODEC_EOS_TOKEN_ID and TALKER_CODEC_BOS_TOKEN_ID constants. Update audio generation logic to use these constants for improved clarity and maintainability. Refactor tensor initialization to utilize the talker's hidden size. Update comments for better understanding and consistency. 2025-10-24 10:02:51 +08:00