anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
934 Commits 235 Branches 31 Tags
b7a36d16d45b0a9da839abd9f2d3a7cfe35ba9ef
Commit Graph
125 Commits
Author SHA1 Message Date
tzhouam 722ad6bbae rebase: align vllm-omni with vLLM 0a0a1a198be8 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-17 03:26:42 +00:00
tzhouam 0eccb77c4f Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-03-17 02:48:50 +00:00
rongfu.leng 4036cd547c [Misc] removed qwen3_tts.py as it is out-dated (#1926) 
Signed-off-by: rongfu.leng <lenronfu@gmail.com>
 2026-03-16 13:09:04 -04:00
tzhouam a26e568dfc Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-03-16 07:30:43 +00:00
Canlin Guo 9718a9e369 [NPU] Upgrade to v0.17.0 (#1890) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-03-16 14:42:59 +08:00
tzhouam 3a508d7b7a rebase: align vllm-omni with vLLM e9163b536e72 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-16 02:32:39 +00:00
tzhouam 97899c6c98 Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-03-13 19:03:57 +00:00
汪志鹏 e0b71056ba [BugFix]: Fix bagel online inference bug (#1804) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2026-03-13 08:58:44 +00:00
tzhouam d3db476f2d Merge branch 'main' into dev/vllm-align 2026-03-13 06:51:22 +00:00
tzhouam e17fb0e97e Refactor OpenAI serving components and enhance multimodal handling 
- Updated import for OpenAIServingEmbedding in api_server.py.
- Enhanced omni_init_app_state to initialize renderer for engine_client.
- Added handle_oov_mm_token parameter to multiple model classes for better multimodal token handling.
- Improved comments for clarity in various model files.
- Adjusted GPUARModelRunner to ensure proper handling of late interaction runner attributes.

Signed-off-by: [Your Name] <[Your Email]>

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-13 01:55:17 +00:00
Yueqian Lin 366b3368ea Add Fish Speech S2 Pro support with online serving and voice cloning (#1798) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-12 16:20:52 -04:00
usberkeleyprinceprideHongsheng Liu
 		b3cced310a [bugfix] Add Interleaved 2D Rotary Embedding for HunyuanImage3 (#1784) 
Signed-off-by: Bradley <bradley.b.pitt@gmail.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-03-12 21:33:03 +08:00
tzhouam 11b6c16007 rebase: align vllm-omni with vLLM 84e436ed1c94 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-11 19:24:44 +00:00
tzhouam 08a2673c66 rebase: align vllm-omni with vLLM 84e436ed1c94 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-11 07:29:52 +00:00
Junhong Liu 2ee4a07adb [Refactor][Perf] Qwen3-omni: code predictor with re-prefill + SDPA and eliminate decode hot-path CPU round-trips (#1758) 
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
 2026-03-11 07:52:21 +08:00
Juan Pablo Zuluagaandpablo 1d0a97f0d1 [feat][Qwen3TTS] Simple dynamic TTFA based on Code2Wav load (#1714) 
Signed-off-by: pablo <pablo@agigo.ai>
Co-authored-by: pablo <pablo@agigo.ai>
 2026-03-10 21:02:33 +00:00
Baoyuan Qi 2fdc647ed7 [BUGFIX] Add compatibility for mimo-audio with vLLM 0.17.0 (#1752) 
Signed-off-by: Baoyuan Qi <qibaoyuan@xiaomi.com>
 2026-03-10 20:57:26 +00:00
Sy03 761eff930a [Fix][Qwen3-TTS] Preserve ref_code decoder context for Base ICL (#1731) 2026-03-09 13:36:27 -04:00
汪志鹏 7271116973 [Bagel]: Support multistage img2img (#1669) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-03-09 22:23:50 +08:00
Zhou Taichang汪志鹏Hongsheng LiuCyrus Leung
 		89b527e533 Dev/rebase v0170 (#1639) 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-03-09 17:29:38 +08:00
zhaotyerlinyueqianYueqian Lin
 		41964cb90a feat(tts): add voice upload API for Qwen3-TTS (#1201) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
Co-authored-by: linyueqian <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-03-09 11:02:11 +08:00
Yueqian Lin a9f694b08a [Bugfix] Use upstream MediaConnector for ref_audio resolution (#1661) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-09 01:38:26 +00:00
Roman Koshkin 0e432e524c [Bugfix] (qwen3_tts): enable batched offline inference by fixing tens… (#1417) 
Signed-off-by: Roman Koshkin <roman.koshkin@sbintuitions.co.jp>
 2026-03-09 09:17:10 +08:00
Baoyuan Qi 32fd084ae2 [MiMo-Audio] Bugfix tp lg than 1 (#1688) 
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2026-03-06 14:45:08 +08:00
Sy03 90ec499bd5 [Refactor][Perf] Qwen3-TTS: re-prefill Code Predictor with torch.compile + enable Code2Wav decoder CUDA Graph (#1617) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-03-06 18:21:55 +13:00
Du BinlinyueqianYueqian Lin
 		fd51841230 perf: replace per-element .item() GPU syncs with batch .tolist() in TTS code predictor (#1614) 
Signed-off-by: dubin555 <dubin555@gmail.com>
Signed-off-by: linyueqian <linyueqian@outlook.com>
Co-authored-by: linyueqian <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-03-06 12:22:01 +08:00
Jack Leegoogle-labs-jules[bot]JackLeeHal
 		c0b714a8ac [Feature] Support flexible task_type configuration for Qwen3-TTS models (#1197) 
Signed-off-by: Jack Lee <695697442@qq.com>
Signed-off-by: jackleehal <695697442@qq.com>
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Co-authored-by: JackLeeHal <15664755+JackLeeHal@users.noreply.github.com>
 2026-03-05 13:39:09 -05:00
zhanqiuhuandYueqian Lin e643aaec15 [Bugfix] Fix all-silence TTS output: use float32 for speech tokenizer decoder (#1664) 
Signed-off-by: Zhanqiu Hu <zh338@cornell.edu>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-03-06 00:04:26 +08:00
Ziming Huang 070ea0ddc3 [Optimize][Qwen3-Omni] Reduce inter-packet latency in async chunk (#1656) 
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
 2026-03-05 22:26:30 +08:00
dsinghviandHongsheng Liu b63e7c7c9a [Model] Fun cosy voice3-0.5-b-2512 (#498) 
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-03-04 19:54:59 -05:00
ShiCheng DengcatcatiwzbiHongsheng LiudsinghviCanlin GuoPeiqi Yin汪志鹏
 		1612948062 add support for MammothModa2 model (#336) 
Signed-off-by: HonestDeng <2958906959@qq.com>
Signed-off-by: iwzbi <iwzbi@zju.edu.cn>
Signed-off-by: iwzbi <wzbi@zju.edu.cn>
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: catcat <108673086+iwzbi@users.noreply.github.com>
Co-authored-by: iwzbi <iwzbi@zju.edu.cn>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: dsinghvi <divyanshsinghvi@gmail.com>
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-03-04 22:52:56 +08:00
usberkeleyandprincepride 84338494d2 [Model] Add Hunyuan Image3 AR Support (#759) 
Signed-off-by: Bradley <bradley.b.pitt@gmail.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: princepride <wangzhipeng628@gmail.com>
 2026-03-04 21:49:38 +08:00
Yueqian Lin a3240fd484 [Bugfix] Fix transformers 5.x compat issues in online TTS serving (#1536) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-04 01:28:32 +08:00
Junhong Liu 742d845b8e Make chunk_size and left_context_size configurable via YAML for async chunking (#1423) 
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
Signed-off-by: Junhong Liu <ljh_lbj@163.com>
 2026-03-03 22:06:52 +08:00
zhanqiuhu 80c89e00cb [Bugfix] Fix Qwen3-TTS code predictor crash due to missing vLLM config context (#1619) 
Signed-off-by: Zhanqiu <zh338@cornell.edu>
 2026-03-03 11:02:13 +08:00
dsinghviandZhou Taichang 68764cc4f4 [Refactor]: Phase1 for rebasing_additional_info (#1394) 
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com>
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: Zhou Taichang <tzhouam@connect.ust.hk>
 2026-03-02 10:19:41 +08:00
汪志鹏 3d9fa8d53f [BugFix]: fix a lot of bug (#1565) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2026-02-28 16:26:56 +08:00
Canlin Guo c81266703a [NPU][Bugfix] Align GPU side and recover qwen3-tts (#1564) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-02-28 12:46:13 +08:00
Zhou Taichang fe0c02ee78 [Debug] Merge vllm pull 35368 (#1534) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-28 10:39:25 +08:00
Junhong Liu 42d86af1f1 Fix no embed text spk tokens (#1540) 
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
 2026-02-28 09:33:58 +08:00
+13
Baoyuan Qiwangyuwangyu31577Zhang ShijinHongsheng LiuSihao LiXU MingshiCanlin GuoIsotr0pyJohnJanWeiQing Chendongbo910220Jiangyun ZhuJunhong LiuTJianshijin zhangZhou TaichangrootPrajwal AShijin Zhangdingningning dingCopilot
 		b19c227fb3 [New Model]: XiaomiMiMo/MiMo-Audio-7B-Instruct support (#750) 
Signed-off-by: wangyu31577 <wangyu31577@hundsun.com>
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
Signed-off-by: hsliu <liuhongsheng4@huawei.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: GG-li <3226868735@qq.com>
Signed-off-by: Sihao Li <111170255+GG-li@users.noreply.github.com>
Signed-off-by: XU Mingshi <91017482+mxuax@users.noreply.github.com>
Signed-off-by: mxuax <mxuax@connect.ust.hk>
Signed-off-by: Baoyuan Qi <qibaoyuan@126.com>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: wuzhongjian <wuzhongjian_yewu@cmss.chinamobile.com>
Signed-off-by: dongbo910220 <1275604947@qq.com>
Signed-off-by: dongbo910220 <32610838+dongbo910220@users.noreply.github.com>
Signed-off-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Signed-off-by: baoyuan qi <qibaoyuan@126.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Prajwal A <prajwalanagani@gmail.com>
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com>
Signed-off-by: 丁宁 <nndding@gmail.com>
Signed-off-by: SHIJIN ZHANG <75300765+Dovis01@users.noreply.github.com>
Signed-off-by: dingning<dingning7@xiaomi.com>
Signed-off-by: dingning <dingning7@xiaomi.com>
Signed-off-by: dingning <dingning@xiaomi.com>
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com>
Co-authored-by: wangyu31577 <wangyu31577@hundsun.com>
Co-authored-by: Zhang Shijin <zhangshijin@xiaomi.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Sihao Li <111170255+GG-li@users.noreply.github.com>
Co-authored-by: XU Mingshi <91017482+mxuax@users.noreply.github.com>
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: JohnJan <wuzhongjian_yewu@cmss.chinamobile.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Co-authored-by: dongbo910220 <32610838+dongbo910220@users.noreply.github.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Co-authored-by: Junhong Liu <ljh_lbj@163.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
Co-authored-by: shijin zhang <zsj1364226740@gmail.com>
Co-authored-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: root <root@hk01dgx028.cm.cluster>
Co-authored-by: Prajwal A <34590600+LawJarp-A@users.noreply.github.com>
Co-authored-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com>
Co-authored-by: dingning <dingning7@xiaomi.com>
Co-authored-by: ning ding <nndding@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2026-02-27 11:19:49 +08:00
Xu Luandxulusjb c9b31a4c1a feat(qwen3-tts): Add CUDA Graph support for speech tokenizer decoder (#1205) 
Signed-off-by: xulusjb <fdukeshik@gmail.com>
Co-authored-by: xulusjb <fdukeshik@gmail.com>
 2026-02-27 10:48:25 +08:00
Jared Wen汪志鹏Hongsheng Liu
 		4146095623 [Perf] GLM Image (#920) 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
Signed-off-by: Jared Wen <w13431838023@gmail.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-27 10:29:14 +08:00
amy-why-3459Hongsheng LiuCanlin Guo
 		e1337c89c7 [BugFix] Restore talker's config (#1524) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Canlin Guo <961750412@qq.com>
 2026-02-27 06:44:11 +08:00
amy-why-3459 66c3e04263 [Performance]Qwen3-Omni performance optimization (#1378) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-02-26 21:21:36 +08:00
Juan Pablo ZuluagaGao HanHongsheng Liu
 		66457c39fa [Bugfix][Qwen3TTS] (#1289) 
Signed-off-by: pablo <juanz9312@gmail.com>
Co-authored-by: Gao Han <gaohan19@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-26 18:58:35 +08:00
Yuanheng Zhao 904b97d5bd [Fix][Chore] Qwen3-TTS Modeling Minor Code Sanity Improvements (#1482) 
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
 2026-02-26 14:39:08 +08:00
Canlin Guo 22c8cde328 [NPU] Upgrade to v0.16.0 (#1375) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-02-26 14:26:50 +08:00
Sangchun Ha 315daeba8c [Bugfix] Fix case-sensitive task_type matching in Qwen3TTSModelForGeneration (#1455) 
Signed-off-by: Sangchun Ha <seomk9896@gmail.com>
 2026-02-26 10:27:13 +08:00
Zhou Taichang 8e950daf6a [Debug] Multi-Request for Qwen 3 Omni use_audio_in_video (#1433) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-26 09:26:25 +08:00