Taichang Zhou and Claude Opus 4.8
9678ae58cf
[MR V2] Migrate to Model Runner V2
...
Squashed migration rebased onto current main. Collapses the 46-commit
dev/migrate-MR-v2 history (10 merge commits + superseded pre-rebase work)
into a single commit, so the diff reflects only the genuine MR-V2 feature
(worker_v2/ runners + model_states, omni schedulers, stage engine pipeline,
pooling_output serialization bridge). Content already merged to main
separately (step_audio2, tts_adapters, npu/_310p, etc.) drops out.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-06-20 07:19:40 +00:00
cead615288
[Bugfix][Qwen3-Omni] Fix Code2Wav CUDA-graph output length surplus ( #4466 ) ( #4525 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Sy03 <1370724210@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-06-19 23:58:45 -07:00
yangyonggit and Yueqian Lin
1228a109f0
[Bugfix] MOSS-TTS: fix cross-request audio corruption under batching ( #4415 )
...
Signed-off-by: leo.yang <leo.yang.engineer@gmail.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-06-18 18:11:51 +00:00
akshatvishu
d8a72be206
fix(models): remove invalid sampling_metadata parameter from logits_processor ( #4456 )
...
Signed-off-by: akshatvishu <akshatnayak197@gmail.com >
2026-06-16 14:41:40 +08:00
9bc8705c98
[New Model] Step-Audio2 ( #464 )
...
Signed-off-by: linyueqian <linyueqian@outlook.com >
Signed-off-by: wuli666 <djjpro975@gmail.com >
Co-authored-by: linyueqian <linyueqian@outlook.com >
Co-authored-by: wuli666 <djjpro975@gmail.com >
2026-06-13 01:00:33 +00:00
1733831f78
[Bugfix] Fix cross-request codes.ref leak in Qwen3-TTS make_omni_output ( #4373 )
...
Signed-off-by: jjw <m@jjw.in >
Co-authored-by: jjw <m@jjw.in >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-06-13 00:41:49 +00:00
yangyonggit
a3834e32cf
feat(moss-tts): add CUDA Graph support for codec decoder ( #4157 )
...
Signed-off-by: leo.yang <leo.yang.engineer@gmail.com >
2026-06-11 22:17:31 +00:00
akshatvishu and Hongsheng Liu
2fa4c7a4a9
[Model] Add Ming-omni-tts dense 0.5B pipeline ( #2906 )
...
Signed-off-by: akshatvishu <akshatnayak197@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-06-10 20:11:04 +08:00
Yuekai Zhang and Claude Opus 4.8
f1bf544208
[Perf][TTS] Optimize cosyvoice TTFP and throughput using TensorRT ( #4168 )
...
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-06-10 16:42:34 +08:00
c33ca60208
[Perf][VoxCPM2] Optimize VoxCPM2 high-concurrency decode throughput. ( #3882 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk >
2026-06-03 23:08:32 +08:00
2b7249fbb3
[Model]Support MiniCPM-o 4.5 ( #3642 )
...
Signed-off-by: tc-mb <tianchi_cai@icloud.com >
Co-authored-by: GKangaroo <1095103651@qq.com >
Co-authored-by: GKangaroo <gqx24@mails.tsinghua.edu.cn >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-06-01 22:12:24 +08:00
boatman and sphinxkkkbc
2dcfcc0519
[Feat]Support Nonverbal Tags in OmniVoice ( #3968 )
...
Signed-off-by: sphinxkkkbc <binchengkang8@gmail.com >
Co-authored-by: sphinxkkkbc <binchengkang8@gmail.com >
2026-06-01 15:51:05 +08:00
d4c139508f
[Feature] Add precomputed custom voices and Qwen3-TTS ref-context cache ( #3492 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Dan250124 <416947747@qq.com >
Co-authored-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-05-31 17:51:24 +00:00
Viacheslav Klimkov and Yueqian Lin
914e92cf80
[Refactor][Qwen3-TTS] Extract reusable prompt-embeds builder and make tts_pad_embed a persistent buffer ( #3992 )
...
Signed-off-by: Viacheslav Klimkov <vklimkov@nvidia.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-05-30 16:11:00 +08:00
nperraud
9fdb3bcb89
[Bugfix][Model] Qwen3-TTS: don't collapse 2D ref_code list when estimating prompt length ( #3940 )
...
Signed-off-by: nperraud <nathanael.perraudin@gmail.com >
2026-05-29 09:18:25 -06:00
wuli666 and wuli666
8935439428
[Bugfix] Fix Triton SnakeBeta kernel for bf16/fp16 inputs ( #3472 )
...
Signed-off-by: wuli666 <djjpro975@gmail.com >
Co-authored-by: wuli666 <djjpro975@gmail.com >
2026-05-28 17:10:36 +00:00
Nick Cao and Claude
c20afebce4
[Refactor][Qwen3-TTS] Construct speech tokenizer encoder natively ( #3360 )
...
Signed-off-by: Nick Cao <ncao@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-05-28 16:20:49 +08:00
Liang Lv
b72ffb82bb
[AutoRound] Support GLM-Image W4A16 quantization model ( #3059 )
...
Signed-off-by: lvliang-intel <liang1.lv@intel.com >
2026-05-28 07:19:06 +00:00
NATURE
3a347b10b0
[3/5][core]refactor communication layer: PR 3 of 5, all other models in non async mode ( #3719 )
...
Signed-off-by: natureofnature <wzliu@connect.hku.hk >
2026-05-28 14:06:41 +08:00
Sy03 and Yueqian Lin
d37912b071
[TTS][Perf] Optimize Fish Speech S2 Pro high-concurrency serving ( #3773 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-05-27 13:25:49 -05:00
Yueqian Lin and Rein Yang
e996723bc9
[Bugfix] Fix Qwen3-TTS Stage 0 prefix-caching correctness ( #3665 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Signed-off-by: Rein Yang <ruiruyang2@gmail.com >
Co-authored-by: Rein Yang <ruiruyang2@gmail.com >
the ci failure case is irrelavent to this PR and occurs rarely
2026-05-25 23:09:38 +08:00
BeatSeat and Yueqian Lin
28ce618f0e
[Model] Add GLM-TTS text-to-speech model support ( #3141 )
...
Signed-off-by: BeatSeat <wendavid552@gmail.com >
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-05-24 05:42:26 +08:00
e7644daa7f
[Perf] [OmniVoice] Triton kernel fusion + CUDA Graph acceleration ( #3336 )
...
Signed-off-by: univa-HARRY <harry@univa.co.kr >
Signed-off-by: Harry <harry@univa.co.kr >
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-05-22 13:12:47 +00:00
Zeng Chuang and Claude Opus 4.7
f0af3ab200
[BugFix] Fix prefer_model_sampler token history in async scheduling ( #3681 )
...
Signed-off-by: zengchuang <zengchuang3@huawei.com >
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com >
2026-05-22 14:29:46 +08:00
baonudesifeizhai
076c34abfb
（Phase 2）Add ModelOpt mixed FP8/NVFP4 support for image generation ( #3570 )
...
Signed-off-by: roG0d <baonudesifeizhai@gmail.com >
Signed-off-by: baonudesifeizhai <85092850+baonudesifeizhai@users.noreply.github.com >
2026-05-22 10:39:10 +08:00
Siyuan Kong
7d9d8aee79
[Bugfix] Fix qwen2_5_omni weight loading ( #3598 )
...
Signed-off-by: ksiyuan <ksiyuan@umich.edu >
2026-05-21 16:43:33 +08:00
Sy03 and amy-why-3459
ab59673f21
[Bugfix][Qwen3-Omni] Handle short Code2Wav chunk outputs ( #3687 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-05-18 18:59:23 +00:00
c99df1ebd9
[TTS][Perf] Optimize Qwen3-TTS high-concurrency serving ( #3662 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com >
2026-05-17 16:58:59 -04:00
+20
eb4e60ee64
[Rebase] Rebase to vllm v0.21.0 ( #3530 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
Signed-off-by: NumberWan <wantszkin2003@gmail.com >
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com >
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com >
Signed-off-by: Dnoob <dxpouo@gmail.com >
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com >
Signed-off-by: rein yang <ruiruyang2@gmail.com >
Signed-off-by: Nick Cao <ncao@redhat.com >
Signed-off-by: zhumingjue <zhumingjue@huawei.com >
Signed-off-by: Ricardo Noriega De Soto <rnoriega@redhat.com >
Signed-off-by: lyj-jjj <liuyingjun5@huawei.com >
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
Signed-off-by: wangyu <410167048@qq.com >
Signed-off-by: weizhoublue <weizhoublue@github.com >
Signed-off-by: weizhou.lan@daocloud.io <weizhou.lan@daocloud.io >
Signed-off-by: dengyunyang <584797741@qq.com >
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com >
Signed-off-by: David Chen <530634352@qq.com >
Signed-off-by: Jie Liu <33612777+keeper-jie@users.noreply.github.com >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: natureofnature <wzliu@connect.hku.hk >
Signed-off-by: bjf-frz <frz123db@gmail.com >
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
Signed-off-by: KexiongYu <yukexiong1@huawei.com >
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com >
Signed-off-by: Zeyu Huang | 黃澤宇 <11222265+fhfuih@users.noreply.github.com >
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com >
Co-authored-by: NumberWan <wantszkin2003@gmail.com >
Co-authored-by: dsinghvi <divyanshsinghvi@gmail.com >
Co-authored-by: Dnoob <dxpouo@gmail.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: knlnguyen1802 <knlnguyen1802@gmail.com >
Co-authored-by: Samit <285365963@qq.com >
Co-authored-by: rein yang <73573651+R2-Y@users.noreply.github.com >
Co-authored-by: Nick Cao <ncao@redhat.com >
Co-authored-by: zhumingjue138 <zhumingjue@huawei.com >
Co-authored-by: Ricardo Noriega <rnoriega@redhat.com >
Co-authored-by: lyj-jjj <liuyingjun5@huawei.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: gcanlin <canlinguosdu@gmail.com >
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com >
Co-authored-by: weizhoublue <45163302+weizhoublue@users.noreply.github.com >
Co-authored-by: weizhoublue <weizhoublue@github.com >
Co-authored-by: dengyunyang <584797741@qq.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com >
Co-authored-by: Jie Liu <33612777+keeper-jie@users.noreply.github.com >
Co-authored-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: NATURE <wzliu@connect.hku.hk >
Co-authored-by: bjf-frz <frz123db@gmail.com >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
Co-authored-by: Y. Fisher <yukexiong1@huawei.com >
Co-authored-by: Zeyu Huang | 黃澤宇 <11222265+fhfuih@users.noreply.github.com >
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com >
2026-05-16 18:18:43 +08:00
Nick Cao and Claude Opus 4.6
5313cf6d48
[Bugfix] Fix omni processing test for non-multimodal talker stage ( #3559 )
...
Signed-off-by: Nick Cao <ncao@redhat.com >
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-05-13 10:08:23 +08:00
dsinghvi
e40076e872
[Refactor] msgspec standardisation for data entry key names and improved type checks ( #3149 )
...
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com >
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com >
2026-05-12 14:30:13 +08:00
e1088026fa
[Performance] Improve MiMo-Audio tokenizer decoding performance ( #2183 )
...
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com >
Co-authored-by: Jialong Liu <88185941+Galleons2029@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-05-11 10:05:46 +08:00
Sy03
2c73cf3522
[Perf] Fix Qwen3-TTS latency regression ( #3485 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
2026-05-10 23:07:38 +00:00
wangyu and Huang, Zeyu
6f2ad7b403
[Test] Unify L2/L3 test layout, Buildkite steps, and test helpers ( #2556 )
...
Signed-off-by: wangyu <410167048@qq.com >
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com >
Signed-off-by: wangyu <53896905+yenuo26@users.noreply.github.com >
Co-authored-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com >
2026-05-08 14:12:23 +08:00
Nick Cao
e6466cf0e6
[Refactor] Replace and ban a few torch.cuda functions in favor of torch.accelerator replacements. ( #3365 )
...
Signed-off-by: Nick Cao <ncao@redhat.com >
2026-05-07 23:23:59 +08:00
Juan Pablo Zuluaga and Yueqian Lin
19f8f42822
[Feat][Qwen3-Omni] Add CUDA graph support for Code2Wav decoder ( #2376 )
...
Signed-off-by: JuanPZuluaga <juanz9312@gmail.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-05-07 06:16:03 +08:00
Sy03
2b85474c23
[Bugfix] Propagate seed to Qwen3-TTS Fast AR sampler ( #3350 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
2026-05-05 14:57:55 -04:00
Nick Cao and Claude
fde38ca54f
[Refactor][Qwen3-TTS] Construct Code2Wav decoder natively ( #2341 )
...
Signed-off-by: Nick Cao <ncao@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-05-05 15:26:30 +00:00
Juan Pablo Zuluaga and JuanPZuluaga
b64ab05d09
[TTS][SpeakerCacheManager] A global speaker cache manager for Voice Cloning ( #2630 )
...
Signed-off-by: JuanPZuluaga <juanz9312@gmal.com >
Co-authored-by: JuanPZuluaga <juanz9312@gmal.com >
2026-05-05 10:53:45 -04:00
Yueqian Lin
703e31fc47
[Docs] Consolidate per-model TTS examples into a single hub ( #3234 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
2026-05-05 08:41:29 +12:00
Zhou Taichang and Yuanheng Zhao
7f2b1e9386
[Rebase] Rebase to vllm 0.20.0 ( #3232 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com >
Co-authored-by: Yuanheng Zhao <54058983+yuanheng-zhao@users.noreply.github.com >
2026-04-30 09:56:59 +12:00
2390350533
[Refactor] Standardize data entry key names to {type}.{qualifier} format ( #1829 )
...
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com >
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Co-authored-by: Zhou Taichang <tzhouam@connect.ust.hk >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-04-25 18:15:38 +08:00
3a32cd6780
[Model] Ming-flash-omni-2.0 Omni-Speech and TTS ( #2890 )
...
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com >
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com >
Signed-off-by: LHXuuu <xulianhao.xlh@antgroup.com >
Co-authored-by: LHXuuu <xulianhao.xlh@antgroup.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-04-23 12:04:52 +08:00
Chen-Yo Sun and Yueqian Lin
8e1bbc9ab5
[Feature][Voxtral TTS] Support per-model extra sampling params & add cfg_alpha for Voxtral TTS ( #2338 )
...
Signed-off-by: Chen-Yo Sun <chenyo.sun@mistral.ai >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Yueqian Lin <linyueqian@outlook.com >
2026-04-22 03:49:50 +00:00
Nick Cao
339a6da28c
[Bugfix] treewide: drop references to librosa ( #2996 )
...
Signed-off-by: Nick Cao <ncao@redhat.com >
2026-04-21 16:04:15 +00:00
Jared Wen and Hongsheng Liu
7e28eda929
[Bugfix] Fix GLM-Image output dimensions and image edit pipeline ( #2320 )
...
Signed-off-by: JaredforReal <w13431838023@gmail.com >
Signed-off-by: Jared Wen <w13431838023@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-04-20 19:24:06 +08:00
wangyu
8a9add1c81
[CI] Restructure vLLM-Omni Test Layout, Fixture Scope, and Support Modules ( #2620 )
...
Signed-off-by: wangyu <410167048@qq.com >
2026-04-20 14:23:42 +08:00
Sy03
26edc7f2fb
[Bugfix][VoxCPM2]: Fix vectorized_gather OOB under concurrent prefill+decode batches ( #2903 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
2026-04-19 20:39:14 +12:00
Juan Pablo Zuluaga and Hongsheng Liu
82f8c93343
[Feat][Qwen3-Omni] Shared code predictor module for Qwen3-TTS and Qwen3-Omni ( #2375 )
...
Signed-off-by: JuanPZuluaga <juanz9312@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-04-15 15:14:57 +08:00
Yiyang "Ian" Liu and Copilot
53a9cf49a6
fix: do not apply FP8 quant config to vision/audio encoders for pre-quantized checkpoints ( #2702 )
...
Signed-off-by: Yiyang Liu <37043548+ianliuy@users.noreply.github.com >
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com >
2026-04-14 13:52:32 +00:00
