tzhouam
034d579d77
Update vLLM precompiled wheel version and refactor various components for improved functionality and error handling. Changes include modifying the get_name method in DiffusionQuantizationConfig to use class-level attributes, enhancing metadata handling in StageEngineCoreClient, and adjusting the AsyncOmni class constructor for better parameter management. Additionally, improved error handling in omni_snapshot_download and optimized initial chunk size computation in talker2code2wav_async_chunk for dynamic load adaptation.
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-03-19 05:14:34 +00:00
tzhouam
342a209924
Merge remote-tracking branch 'origin/main' into dev/vllm-align
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-03-19 02:06:26 +00:00
fc5e8f825e
[Optim][Qwen3TTS] big boost model throughput+latency high concurrency ( #1852 )
...
Signed-off-by: pablo <pablo@agigo.ai >
Signed-off-by: JuanPZuluaga <juanz9312@gmail.com >
Co-authored-by: pablo <pablo@agigo.ai >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-03-18 12:22:05 -04:00
Taichang Zhou
dc6f2fb81f
Enhance cu_seqlens computation in Qwen3Omni_VisionTransformer by switching to tensor operations for improved performance and compatibility. Added error handling to fallback on a vectorized implementation if necessary. This update aligns with upstream changes and optimizes the handling of sequence lengths during attention execution.
2026-03-18 23:27:55 +08:00
Taichang Zhou
7a68bd37b7
Refactor cu_seqlens handling in Qwen3Omni_VisionTransformer to align with upstream vLLM changes. The sequence length helper now retains cu_seqlens in numpy format and converts it for attention execution, improving compatibility and performance.
2026-03-18 23:23:41 +08:00
3158e9d28f
[Entrypoint][Refactor] vLLM-Omni Entrypoint Refactoring ( #1908 )
...
Signed-off-by: yinpe <11810305@mail.sustech.edu.cn >
Signed-off-by: Peiqi Yin <yinpeiqi809@gmail.com >
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com >
Signed-off-by: wuhang <whlbx@hotmail.com >
Signed-off-by: linyueqian <linyueqian@outlook.com >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: lishunyang <lishunyang12@163.com >
Signed-off-by: Chenguang ZHENG <645327136@qq.com >
Signed-off-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com >
Co-authored-by: yinpe <11810305@mail.sustech.edu.cn >
Co-authored-by: Peiqi Yin <yinpeiqi809@gmail.com >
Co-authored-by: wuhang <whlbx@hotmail.com >
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com >
Co-authored-by: linyueqian <linyueqian@outlook.com >
Co-authored-by: princepride <wangzhipeng628@gmail.com >
Co-authored-by: lishunyang <lishunyang12@163.com >
2026-03-18 17:33:53 +08:00
Yueqian Lin
fe786f1a19
Fix Base voice clone streaming quality and stop-token crash ( #1945 )
...
Signed-off-by: linyueqian <linyueqian@outlook.com >
2026-03-18 06:54:12 +08:00
Taichang Zhou
8a31e7865d
Increase timeout for Diffusion Model Test and Diffusion Images API LoRA E2E in Buildkite configurations to 30 minutes. Also, introduce a new timeout constant for diffusion initialization in relevant test files to enhance stability during testing. Update OmniServer initialization to utilize the new timeout settings.
2026-03-17 23:43:46 +08:00
tzhouam
722ad6bbae
rebase: align vllm-omni with vLLM 0a0a1a198be8
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-03-17 03:26:42 +00:00
tzhouam
0eccb77c4f
Merge remote-tracking branch 'origin/main' into dev/vllm-align
2026-03-17 02:48:50 +00:00
rongfu.leng
4036cd547c
[Misc] removed qwen3_tts.py as it is out-dated ( #1926 )
...
Signed-off-by: rongfu.leng <lenronfu@gmail.com >
2026-03-16 13:09:04 -04:00
tzhouam
a26e568dfc
Merge remote-tracking branch 'origin/main' into dev/vllm-align
2026-03-16 07:30:43 +00:00
Canlin Guo
9718a9e369
[NPU] Upgrade to v0.17.0 ( #1890 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
2026-03-16 14:42:59 +08:00
tzhouam
3a508d7b7a
rebase: align vllm-omni with vLLM e9163b536e72
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-03-16 02:32:39 +00:00
tzhouam
97899c6c98
Merge remote-tracking branch 'origin/main' into dev/vllm-align
2026-03-13 19:03:57 +00:00
汪志鹏
e0b71056ba
[BugFix]: Fix bagel online inference bug ( #1804 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
2026-03-13 08:58:44 +00:00
tzhouam
d3db476f2d
Merge branch 'main' into dev/vllm-align
2026-03-13 06:51:22 +00:00
tzhouam
e17fb0e97e
Refactor OpenAI serving components and enhance multimodal handling
...
- Updated import for OpenAIServingEmbedding in api_server.py.
- Enhanced omni_init_app_state to initialize renderer for engine_client.
- Added handle_oov_mm_token parameter to multiple model classes for better multimodal token handling.
- Improved comments for clarity in various model files.
- Adjusted GPUARModelRunner to ensure proper handling of late interaction runner attributes.
Signed-off-by: [Your Name] <[Your Email]>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-03-13 01:55:17 +00:00
Yueqian Lin
366b3368ea
Add Fish Speech S2 Pro support with online serving and voice cloning ( #1798 )
...
Signed-off-by: linyueqian <linyueqian@outlook.com >
2026-03-12 16:20:52 -04:00
b3cced310a
[bugfix] Add Interleaved 2D Rotary Embedding for HunyuanImage3 ( #1784 )
...
Signed-off-by: Bradley <bradley.b.pitt@gmail.com >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: princepride <wangzhipeng628@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-03-12 21:33:03 +08:00
tzhouam
11b6c16007
rebase: align vllm-omni with vLLM 84e436ed1c94
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-03-11 19:24:44 +00:00
tzhouam
08a2673c66
rebase: align vllm-omni with vLLM 84e436ed1c94
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-03-11 07:29:52 +00:00
Junhong Liu
2ee4a07adb
[Refactor][Perf] Qwen3-omni: code predictor with re-prefill + SDPA and eliminate decode hot-path CPU round-trips ( #1758 )
...
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com >
2026-03-11 07:52:21 +08:00
Juan Pablo Zuluaga and pablo
1d0a97f0d1
[feat][Qwen3TTS] Simple dynamic TTFA based on Code2Wav load ( #1714 )
...
Signed-off-by: pablo <pablo@agigo.ai >
Co-authored-by: pablo <pablo@agigo.ai >
2026-03-10 21:02:33 +00:00
Baoyuan Qi
2fdc647ed7
[BUGFIX] Add compatibility for mimo-audio with vLLM 0.17.0 ( #1752 )
...
Signed-off-by: Baoyuan Qi <qibaoyuan@xiaomi.com >
2026-03-10 20:57:26 +00:00
Sy03
761eff930a
[Fix][Qwen3-TTS] Preserve ref_code decoder context for Base ICL ( #1731 )
2026-03-09 13:36:27 -04:00
汪志鹏
7271116973
[Bagel]: Support multistage img2img ( #1669 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com >
2026-03-09 22:23:50 +08:00
89b527e533
Dev/rebase v0170 ( #1639 )
...
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com >
2026-03-09 17:29:38 +08:00
41964cb90a
feat(tts): add voice upload API for Qwen3-TTS ( #1201 )
...
Signed-off-by: linyueqian <linyueqian@outlook.com >
Co-authored-by: linyueqian <linyueqian@outlook.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-03-09 11:02:11 +08:00
Yueqian Lin
a9f694b08a
[Bugfix] Use upstream MediaConnector for ref_audio resolution ( #1661 )
...
Signed-off-by: linyueqian <linyueqian@outlook.com >
2026-03-09 01:38:26 +00:00
Roman Koshkin
0e432e524c
[Bugfix] (qwen3_tts): enable batched offline inference by fixing tens… ( #1417 )
...
Signed-off-by: Roman Koshkin <roman.koshkin@sbintuitions.co.jp >
2026-03-09 09:17:10 +08:00
Baoyuan Qi
32fd084ae2
[MiMo-Audio] Bugfix tp lg than 1 ( #1688 )
...
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com >
2026-03-06 14:45:08 +08:00
Sy03
90ec499bd5
[Refactor][Perf] Qwen3-TTS: re-prefill Code Predictor with torch.compile + enable Code2Wav decoder CUDA Graph ( #1617 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
2026-03-06 18:21:55 +13:00
fd51841230
perf: replace per-element .item() GPU syncs with batch .tolist() in TTS code predictor ( #1614 )
...
Signed-off-by: dubin555 <dubin555@gmail.com >
Signed-off-by: linyueqian <linyueqian@outlook.com >
Co-authored-by: linyueqian <linyueqian@outlook.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-03-06 12:22:01 +08:00
c0b714a8ac
[Feature] Support flexible task_type configuration for Qwen3-TTS models ( #1197 )
...
Signed-off-by: Jack Lee <695697442@qq.com >
Signed-off-by: jackleehal <695697442@qq.com >
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Co-authored-by: JackLeeHal <15664755+JackLeeHal@users.noreply.github.com >
2026-03-05 13:39:09 -05:00
zhanqiuhu and Yueqian Lin
e643aaec15
[Bugfix] Fix all-silence TTS output: use float32 for speech tokenizer decoder ( #1664 )
...
Signed-off-by: Zhanqiu Hu <zh338@cornell.edu >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-03-06 00:04:26 +08:00
Ziming Huang
070ea0ddc3
[Optimize][Qwen3-Omni] Reduce inter-packet latency in async chunk ( #1656 )
...
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
2026-03-05 22:26:30 +08:00
dsinghvi and Hongsheng Liu
b63e7c7c9a
[Model] Fun cosy voice3-0.5-b-2512 ( #498 )
...
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com >
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-03-04 19:54:59 -05:00
1612948062
add support for MammothModa2 model ( #336 )
...
Signed-off-by: HonestDeng <2958906959@qq.com >
Signed-off-by: iwzbi <iwzbi@zju.edu.cn >
Signed-off-by: iwzbi <wzbi@zju.edu.cn >
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com >
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: catcat <108673086+iwzbi@users.noreply.github.com >
Co-authored-by: iwzbi <iwzbi@zju.edu.cn >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: dsinghvi <divyanshsinghvi@gmail.com >
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com >
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
2026-03-04 22:52:56 +08:00
usberkeley and princepride
84338494d2
[Model] Add Hunyuan Image3 AR Support ( #759 )
...
Signed-off-by: Bradley <bradley.b.pitt@gmail.com >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: princepride <wangzhipeng628@gmail.com >
2026-03-04 21:49:38 +08:00
Yueqian Lin
a3240fd484
[Bugfix] Fix transformers 5.x compat issues in online TTS serving ( #1536 )
...
Signed-off-by: linyueqian <linyueqian@outlook.com >
2026-03-04 01:28:32 +08:00
Junhong Liu
742d845b8e
Make chunk_size and left_context_size configurable via YAML for async chunking ( #1423 )
...
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com >
Signed-off-by: Junhong Liu <ljh_lbj@163.com >
2026-03-03 22:06:52 +08:00
zhanqiuhu
80c89e00cb
[Bugfix] Fix Qwen3-TTS code predictor crash due to missing vLLM config context ( #1619 )
...
Signed-off-by: Zhanqiu <zh338@cornell.edu >
2026-03-03 11:02:13 +08:00
dsinghvi and Zhou Taichang
68764cc4f4
[Refactor]: Phase1 for rebasing_additional_info ( #1394 )
...
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com >
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
Co-authored-by: Zhou Taichang <tzhouam@connect.ust.hk >
2026-03-02 10:19:41 +08:00
汪志鹏
3d9fa8d53f
[BugFix]: fix a lot of bug ( #1565 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
2026-02-28 16:26:56 +08:00
Canlin Guo
c81266703a
[NPU][Bugfix] Align GPU side and recover qwen3-tts ( #1564 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
2026-02-28 12:46:13 +08:00
Zhou Taichang
fe0c02ee78
[Debug] Merge vllm pull 35368 ( #1534 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-02-28 10:39:25 +08:00
Junhong Liu
42d86af1f1
Fix no embed text spk tokens ( #1540 )
...
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com >
2026-02-28 09:33:58 +08:00
+13
b19c227fb3
[New Model]: XiaomiMiMo/MiMo-Audio-7B-Instruct support ( #750 )
...
Signed-off-by: wangyu31577 <wangyu31577@hundsun.com >
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com >
Signed-off-by: hsliu <liuhongsheng4@huawei.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Signed-off-by: GG-li <3226868735@qq.com >
Signed-off-by: Sihao Li <111170255+GG-li@users.noreply.github.com >
Signed-off-by: XU Mingshi <91017482+mxuax@users.noreply.github.com >
Signed-off-by: mxuax <mxuax@connect.ust.hk >
Signed-off-by: Baoyuan Qi <qibaoyuan@126.com >
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Signed-off-by: wuzhongjian <wuzhongjian_yewu@cmss.chinamobile.com >
Signed-off-by: dongbo910220 <1275604947@qq.com >
Signed-off-by: dongbo910220 <32610838+dongbo910220@users.noreply.github.com >
Signed-off-by: Jiangyun Zhu <riverclouds.zhu@qq.com >
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com >
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com >
Signed-off-by: baoyuan qi <qibaoyuan@126.com >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: Prajwal A <prajwalanagani@gmail.com >
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com >
Signed-off-by: 丁宁 <nndding@gmail.com >
Signed-off-by: SHIJIN ZHANG <75300765+Dovis01@users.noreply.github.com >
Signed-off-by: dingning<dingning7@xiaomi.com >
Signed-off-by: dingning <dingning7@xiaomi.com >
Signed-off-by: dingning <dingning@xiaomi.com >
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com >
Co-authored-by: wangyu31577 <wangyu31577@hundsun.com >
Co-authored-by: Zhang Shijin <zhangshijin@xiaomi.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Sihao Li <111170255+GG-li@users.noreply.github.com >
Co-authored-by: XU Mingshi <91017482+mxuax@users.noreply.github.com >
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com >
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Co-authored-by: JohnJan <wuzhongjian_yewu@cmss.chinamobile.com >
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com >
Co-authored-by: dongbo910220 <32610838+dongbo910220@users.noreply.github.com >
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com >
Co-authored-by: Junhong Liu <ljh_lbj@163.com >
Co-authored-by: TJian <tunjian.tan@embeddedllm.com >
Co-authored-by: shijin zhang <zsj1364226740@gmail.com >
Co-authored-by: Zhou Taichang <tzhouam@connect.ust.hk >
Co-authored-by: root <root@hk01dgx028.cm.cluster >
Co-authored-by: Prajwal A <34590600+LawJarp-A@users.noreply.github.com >
Co-authored-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com >
Co-authored-by: dingning <dingning7@xiaomi.com >
Co-authored-by: ning ding <nndding@gmail.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2026-02-27 11:19:49 +08:00
Xu Lu and xulusjb
c9b31a4c1a
feat(qwen3-tts): Add CUDA Graph support for speech tokenizer decoder ( #1205 )
...
Signed-off-by: xulusjb <fdukeshik@gmail.com >
Co-authored-by: xulusjb <fdukeshik@gmail.com >
2026-02-27 10:48:25 +08:00
tzhouam