Zhou Taichang
c27f1c4a00
Merge branch 'main' into dev/rebase-v0.18.0
2026-03-21 19:13:12 +08:00
Juan Pablo Zuluaga and JuanPZuluaga
8e12007ac1
[BugFix][Qwen3TTS] CodePredictor CudaGraph Pool ( #2059 )
...
Signed-off-by: JuanPZuluaga <juanz9312@gmal.com >
Co-authored-by: JuanPZuluaga <juanz9312@gmal.com >
2026-03-21 19:11:34 +08:00
Zhou Taichang
e80548767b
Merge branch 'main' into dev/rebase-v0.18.0
2026-03-21 15:56:29 +08:00
Sy03 and sy0307
072647eeb2
[Perf] Improve Fish Speech S2 Pro inference performance ( #1859 )
...
Signed-off-by: sy0307 <sy0307@users.noreply.github.com >
Signed-off-by: Sy03 <1370724210@qq.com >
Co-authored-by: sy0307 <sy0307@users.noreply.github.com >
2026-03-21 01:15:02 -04:00
Zhou Taichang
869a593483
Merge branch 'main' into dev/rebase-v0.18.0
2026-03-20 19:55:26 +08:00
Junhong Liu
6901ba426e
[Fixbug][Perf] Qwen3-omni: code predictor with re-prefill + SDPA and eliminate decode hot-path CPU round-trips ( #2012 )
...
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com >
2026-03-20 19:33:52 +08:00
Zhou Taichang
66c31a3502
Merge branch 'main' into dev/rebase-v0.18.0
...
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
2026-03-20 16:42:17 +08:00
Zhou Taichang
05d8de813e
[Bugfix] Remove duplicated config keyword max batch size ( #1851 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
2026-03-20 16:13:24 +08:00
Juan Pablo Zuluaga and JuanPZuluaga
b4342fb487
[Optim][Qwen3TTS][CodePredictor] support torch.compile with reduce-overhead and dynamic False ( #1913 )
...
Signed-off-by: JuanPZuluaga <juanz9312@gmal.com >
Signed-off-by: JuanPZuluaga <juanz9312@gmail.com >
Co-authored-by: JuanPZuluaga <juanz9312@gmal.com >
2026-03-20 12:12:12 +08:00
81a90d24ef
[Feat][Qwen3TTS][Code2wav] triton SnakeBeta and Cuda Graph ( #1797 )
...
Signed-off-by: pablo <pablo@agigo.ai >
Signed-off-by: JuanPZuluaga <juanz9312@gmal.com >
Co-authored-by: pablo <pablo@agigo.ai >
Co-authored-by: JuanPZuluaga <juanz9312@gmal.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-03-20 12:11:37 +08:00
tzhouam
6ff31eb6ec
Update rope parameters in Qwen3OmniMoeTalker to include rope_theta for improved configuration handling.
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-03-20 03:42:07 +00:00
tzhouam
3c52432100
Merge branch 'main' into dev/vllm-align
2026-03-20 01:38:43 +00:00
Chen-Yo Sun
e9fe78e28c
[Model] Add Voxtral TTS model ( #1803 )
...
Signed-off-by: Chen-Yo Sun <chenyo.sun@mistral.ai >
2026-03-19 16:02:11 -04:00
tzhouam
367ce6328c
Merge remote-tracking branch 'remotes/origin/main' into dev/vllm-align
2026-03-19 09:38:17 +00:00
rein yang
60a8c01a96
[Bugfix] revert PR#1758 which introduced the accuracy problem of qwen3-omni ( #2009 )
...
Signed-off-by: Rein Yang <ruiruyang2@gmail.com >
2026-03-19 17:34:01 +08:00
Chendi.Xue and Hongsheng Liu
ba5a66b15b
[XPU] update bagel modeling to remove cuda hardcode, add xpu stage_config ( #1931 )
...
Signed-off-by: Chendi Xue <chendi.xue@intel.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-03-19 16:55:16 +08:00
tzhouam
37e055ed9e
Merge remote-tracking branch 'remotes/origin/main' into dev/vllm-align
2026-03-19 08:49:35 +00:00
tzhouam
f83df010ef
Remove redundant assignment of rope_theta in Qwen3OmniMoeTalker initialization for cleaner configuration setup.
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-03-19 08:35:56 +00:00
Dom Brown
31c8cb22cf
[Perf] [Qwen3-TTS] Keep audio_codes and last_talker_hidden on GPU to eliminate per-step sync stalls ( #1985 )
...
Signed-off-by: Dom Brown <3886319+DomBrown@users.noreply.github.com >
2026-03-19 13:50:26 +08:00
tzhouam
034d579d77
Update vLLM precompiled wheel version and refactor various components for improved functionality and error handling. Changes include modifying the get_name method in DiffusionQuantizationConfig to use class-level attributes, enhancing metadata handling in StageEngineCoreClient, and adjusting the AsyncOmni class constructor for better parameter management. Additionally, improved error handling in omni_snapshot_download and optimized initial chunk size computation in talker2code2wav_async_chunk for dynamic load adaptation.
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-03-19 05:14:34 +00:00
tzhouam
342a209924
Merge remote-tracking branch 'origin/main' into dev/vllm-align
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-03-19 02:06:26 +00:00
fc5e8f825e
[Optim][Qwen3TTS] big boost model throughput+latency high concurrency ( #1852 )
...
Signed-off-by: pablo <pablo@agigo.ai >
Signed-off-by: JuanPZuluaga <juanz9312@gmail.com >
Co-authored-by: pablo <pablo@agigo.ai >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-03-18 12:22:05 -04:00
Taichang Zhou
dc6f2fb81f
Enhance cu_seqlens computation in Qwen3Omni_VisionTransformer by switching to tensor operations for improved performance and compatibility. Added error handling to fallback on a vectorized implementation if necessary. This update aligns with upstream changes and optimizes the handling of sequence lengths during attention execution.
2026-03-18 23:27:55 +08:00
Taichang Zhou
7a68bd37b7
Refactor cu_seqlens handling in Qwen3Omni_VisionTransformer to align with upstream vLLM changes. The sequence length helper now retains cu_seqlens in numpy format and converts it for attention execution, improving compatibility and performance.
2026-03-18 23:23:41 +08:00
3158e9d28f
[Entrypoint][Refactor] vLLM-Omni Entrypoint Refactoring ( #1908 )
...
Signed-off-by: yinpe <11810305@mail.sustech.edu.cn >
Signed-off-by: Peiqi Yin <yinpeiqi809@gmail.com >
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com >
Signed-off-by: wuhang <whlbx@hotmail.com >
Signed-off-by: linyueqian <linyueqian@outlook.com >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: lishunyang <lishunyang12@163.com >
Signed-off-by: Chenguang ZHENG <645327136@qq.com >
Signed-off-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com >
Co-authored-by: yinpe <11810305@mail.sustech.edu.cn >
Co-authored-by: Peiqi Yin <yinpeiqi809@gmail.com >
Co-authored-by: wuhang <whlbx@hotmail.com >
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com >
Co-authored-by: linyueqian <linyueqian@outlook.com >
Co-authored-by: princepride <wangzhipeng628@gmail.com >
Co-authored-by: lishunyang <lishunyang12@163.com >
2026-03-18 17:33:53 +08:00
Yueqian Lin
fe786f1a19
Fix Base voice clone streaming quality and stop-token crash ( #1945 )
...
Signed-off-by: linyueqian <linyueqian@outlook.com >
2026-03-18 06:54:12 +08:00
Taichang Zhou
8a31e7865d
Increase timeout for Diffusion Model Test and Diffusion Images API LoRA E2E in Buildkite configurations to 30 minutes. Also, introduce a new timeout constant for diffusion initialization in relevant test files to enhance stability during testing. Update OmniServer initialization to utilize the new timeout settings.
2026-03-17 23:43:46 +08:00
tzhouam
722ad6bbae
rebase: align vllm-omni with vLLM 0a0a1a198be8
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-03-17 03:26:42 +00:00
tzhouam
0eccb77c4f
Merge remote-tracking branch 'origin/main' into dev/vllm-align
2026-03-17 02:48:50 +00:00
rongfu.leng
4036cd547c
[Misc] removed qwen3_tts.py as it is out-dated ( #1926 )
...
Signed-off-by: rongfu.leng <lenronfu@gmail.com >
2026-03-16 13:09:04 -04:00
tzhouam
a26e568dfc
Merge remote-tracking branch 'origin/main' into dev/vllm-align
2026-03-16 07:30:43 +00:00
Canlin Guo
9718a9e369
[NPU] Upgrade to v0.17.0 ( #1890 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
2026-03-16 14:42:59 +08:00
tzhouam
3a508d7b7a
rebase: align vllm-omni with vLLM e9163b536e72
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-03-16 02:32:39 +00:00
tzhouam
97899c6c98
Merge remote-tracking branch 'origin/main' into dev/vllm-align
2026-03-13 19:03:57 +00:00
汪志鹏
e0b71056ba
[BugFix]: Fix bagel online inference bug ( #1804 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
2026-03-13 08:58:44 +00:00
tzhouam
d3db476f2d
Merge branch 'main' into dev/vllm-align
2026-03-13 06:51:22 +00:00
tzhouam
e17fb0e97e
Refactor OpenAI serving components and enhance multimodal handling
...
- Updated import for OpenAIServingEmbedding in api_server.py.
- Enhanced omni_init_app_state to initialize renderer for engine_client.
- Added handle_oov_mm_token parameter to multiple model classes for better multimodal token handling.
- Improved comments for clarity in various model files.
- Adjusted GPUARModelRunner to ensure proper handling of late interaction runner attributes.
Signed-off-by: [Your Name] <[Your Email]>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-03-13 01:55:17 +00:00
Yueqian Lin
366b3368ea
Add Fish Speech S2 Pro support with online serving and voice cloning ( #1798 )
...
Signed-off-by: linyueqian <linyueqian@outlook.com >
2026-03-12 16:20:52 -04:00
b3cced310a
[bugfix] Add Interleaved 2D Rotary Embedding for HunyuanImage3 ( #1784 )
...
Signed-off-by: Bradley <bradley.b.pitt@gmail.com >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: princepride <wangzhipeng628@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-03-12 21:33:03 +08:00
tzhouam
11b6c16007
rebase: align vllm-omni with vLLM 84e436ed1c94
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-03-11 19:24:44 +00:00
tzhouam
08a2673c66
rebase: align vllm-omni with vLLM 84e436ed1c94
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-03-11 07:29:52 +00:00
Junhong Liu
2ee4a07adb
[Refactor][Perf] Qwen3-omni: code predictor with re-prefill + SDPA and eliminate decode hot-path CPU round-trips ( #1758 )
...
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com >
2026-03-11 07:52:21 +08:00
Juan Pablo Zuluaga and pablo
1d0a97f0d1
[feat][Qwen3TTS] Simple dynamic TTFA based on Code2Wav load ( #1714 )
...
Signed-off-by: pablo <pablo@agigo.ai >
Co-authored-by: pablo <pablo@agigo.ai >
2026-03-10 21:02:33 +00:00
Baoyuan Qi
2fdc647ed7
[BUGFIX] Add compatibility for mimo-audio with vLLM 0.17.0 ( #1752 )
...
Signed-off-by: Baoyuan Qi <qibaoyuan@xiaomi.com >
2026-03-10 20:57:26 +00:00
Sy03
761eff930a
[Fix][Qwen3-TTS] Preserve ref_code decoder context for Base ICL ( #1731 )
2026-03-09 13:36:27 -04:00
汪志鹏
7271116973
[Bagel]: Support multistage img2img ( #1669 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com >
2026-03-09 22:23:50 +08:00
89b527e533
Dev/rebase v0170 ( #1639 )
...
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com >
2026-03-09 17:29:38 +08:00
41964cb90a
feat(tts): add voice upload API for Qwen3-TTS ( #1201 )
...
Signed-off-by: linyueqian <linyueqian@outlook.com >
Co-authored-by: linyueqian <linyueqian@outlook.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-03-09 11:02:11 +08:00
Yueqian Lin
a9f694b08a
[Bugfix] Use upstream MediaConnector for ref_audio resolution ( #1661 )
...
Signed-off-by: linyueqian <linyueqian@outlook.com >
2026-03-09 01:38:26 +00:00
Roman Koshkin
0e432e524c
[Bugfix] (qwen3_tts): enable batched offline inference by fixing tens… ( #1417 )
...
Signed-off-by: Roman Koshkin <roman.koshkin@sbintuitions.co.jp >
2026-03-09 09:17:10 +08:00
Zhou Taichang