Sy03
df8137e858
Address Qwen3-TTS payload split review comments
...
Signed-off-by: Sy03 <1370724210@qq.com >
2026-07-04 16:21:02 +08:00
Yueqian Lin
ede3dddfdf
fix(serving): scope no-async-chunk FINAL_ONLY streaming to qwen3_tts
...
async_chunk defaults to False in OmniModelConfig, so coercing every
async_chunk=False deploy to FINAL_ONLY for streaming requests would
silently disable incremental delivery for non-Qwen3 models that stream
DELTA-style today. Gate the coercion on _tts_model_type == "qwen3_tts"
(the model this PR validated) and pin the non-qwen3 behavior with a test.
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
2026-07-03 22:57:27 -07:00
Sy03
c523026a8e
Fix Qwen3-TTS no-async-chunk payload splitting
...
Signed-off-by: Sy03 <1370724210@qq.com >
2026-07-04 12:35:08 +08:00
LHXuuu
991fb7408b
[BugFix][Ming-TTS] Improve Seed-TTS WER accuracy ( #4859 )
...
Signed-off-by: LHXuuu <xulianhao.xlh@antgroup.com >
2026-07-03 12:15:03 +02:00
Canlin Guo
a8b2bf11d2
[Feature] Support MOSS-TTS-Local-v1.5 Streaming ( #4804 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
2026-07-03 16:00:30 +08:00
faceless void
11f6a0b72e
Make speech streaming default to SSE ( #4679 )
...
Signed-off-by: syd520zy <529477025@qq.com >
2026-06-30 02:17:24 +00:00
Yueqian Lin
92f7208559
[Bugfix] CosyVoice3: wrap ref_text in instruction template ( #4644 ) ( #4756 )
2026-06-27 14:38:33 -07:00
Yueqian Lin and Sy03
b66cd024b9
[Bugfix] Fix /v1/audio/speech usage token accounting for Qwen3-TTS ( #4646 ) ( #4673 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Sy03 <1370724210@qq.com >
2026-06-27 06:13:31 +08:00
0b04f2a0d6
[Model] Add Ming-omni-tts MoE 16.8B-A3B + CFM CUDAGraph ( #4341 )
...
Signed-off-by: LHXuuu <xulianhao.xlh@antgroup.com >
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com >
Co-authored-by: Yuanheng Zhao <54058983+yuanheng-zhao@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-06-25 22:11:35 +08:00
Canlin Guo
2e6c83c89c
[Model] Support MOSS-TTS-local-v1.5 ( #4664 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
2026-06-25 15:43:03 +08:00
BeatSeat
044240d8eb
[Model] Add IndexTTS2 text-to-speech support ( #3838 )
...
Signed-off-by: BeatSeat <wendavid552@gmail.com >
Signed-off-by: David <david@podcast.ai >
Signed-off-by: wendavid552 <wendavid552@gmail.com >
Signed-off-by: WenDavid <davidzhangcf@foxmail.com >
2026-06-22 19:24:01 +00:00
+2
ec4ec0f604
Add SSE stream_format support for audio speech streaming ( #4490 )
...
Signed-off-by: syd520zy <529477025@qq.com >
Signed-off-by: faceless void <100186390+syd520zy@users.noreply.github.com >
Signed-off-by: Nick Cao <ncao@redhat.com >
Signed-off-by: Chendi Xue <chendi.xue@intel.com >
Signed-off-by: kTorp <Kristoffer.Torp@amd.com >
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Signed-off-by: Hyoseop Song <crad_on25@naver.com >
Signed-off-by: Hyoseop Song <crad_on25@naver.com >
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
Co-authored-by: Nick Cao <ncao@redhat.com >
Co-authored-by: Chendi.Xue <chendi.xue@intel.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Kristoffer <kristoffer.torp@amd.com >
Co-authored-by: Junhong Liu <ljh_lbj@163.com >
Co-authored-by: lishunyang <lishunyang12@163.com >
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com >
Co-authored-by: Hyoseop Song <crad_on25@naver.com >
Co-authored-by: Zhou Taichang <tzhouam@connect.ust.hk >
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-06-22 06:42:25 +00:00
ad07f98b73
[Feat] Streaming TTS word-level timestamps via forced aligner ( #4034 )
...
Signed-off-by: wjinxu <1299461899@qq.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com >
2026-06-22 04:51:45 +00:00
Yueqian Lin
208932210f
[Chore][TTS] Reference recipes.vllm.ai recipes, remove dead code, and fix stale references ( #4567 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
2026-06-19 15:05:26 +08:00
Yueqian Lin
d75a6433b4
[Refactor] TTS serving adapter framework + migrate TTS models (excl. ming_flash_omni) ( #4330 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
2026-06-13 04:22:54 +00:00
9bc8705c98
[New Model] Step-Audio2 ( #464 )
...
Signed-off-by: linyueqian <linyueqian@outlook.com >
Signed-off-by: wuli666 <djjpro975@gmail.com >
Co-authored-by: linyueqian <linyueqian@outlook.com >
Co-authored-by: wuli666 <djjpro975@gmail.com >
2026-06-13 01:00:33 +00:00
152343d5fb
Feat: non_streaming_mode for Qwen3-TTS Base Models During Online Inference ( #4198 )
...
Signed-off-by: nagisa-kun <1434936049@qq.com >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Sy03 <1370724210@qq.com >
2026-06-11 22:15:05 +00:00
e27fd46ab9
[Perf] Optimize Higgs Audio v3 serving ( #4204 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-06-11 13:29:31 -07:00
akshatvishu and Hongsheng Liu
2fa4c7a4a9
[Model] Add Ming-omni-tts dense 0.5B pipeline ( #2906 )
...
Signed-off-by: akshatvishu <akshatnayak197@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-06-10 20:11:04 +08:00
Optance
d50bf1aa42
[Bugfix] Fix Fish Speech prefix cache collision from missing cache_salt ( #4008 )
...
Signed-off-by: nagelanping <xiao.yi.ming@foxmail.com >
2026-06-10 17:11:01 +08:00
Yuekai Zhang and Claude Opus 4.8
f1bf544208
[Perf][TTS] Optimize cosyvoice TTFP and throughput using TensorRT ( #4168 )
...
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-06-10 16:42:34 +08:00
Lokesh
176b00538c
[Model] Support languages added by fine-tuned Qwen3-TTS checkpoints ( #4210 )
...
Signed-off-by: lokesh <lokesh.v9911@gmail.com >
2026-06-07 10:10:40 -06:00
Yueqian Lin
fce0f51089
[Perf][Higgs-Audio-V3] LRU cache for voice-clone ref-audio encode ( #4200 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
2026-06-07 03:09:16 +00:00
Yuekai Zhang and Yueqian Lin
82e74ce972
[TTS][New Model] support bosonai/higgs-audio-v3-tts-4b ( #4169 )
...
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Yueqian Lin <linyueqian@outlook.com >
2026-06-06 15:59:12 +00:00
Zhang Jian and Canlin Guo
b550709bba
[TTS][Model] support MOSS-TTS series ( #3420 )
...
Signed-off-by: Zhang <jianmusings@gmail.com >
Signed-off-by: Zhang Jian <jianmusings@gmail.com >
Signed-off-by: jian <jianmusings@gmail.com >
Signed-off-by: Canlin Guo <961750412@qq.com >
Co-authored-by: Canlin Guo <961750412@qq.com >
2026-06-02 15:46:01 +08:00
d4c139508f
[Feature] Add precomputed custom voices and Qwen3-TTS ref-context cache ( #3492 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Dan250124 <416947747@qq.com >
Co-authored-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-05-31 17:51:24 +00:00
Viacheslav Klimkov and Yueqian Lin
914e92cf80
[Refactor][Qwen3-TTS] Extract reusable prompt-embeds builder and make tts_pad_embed a persistent buffer ( #3992 )
...
Signed-off-by: Viacheslav Klimkov <vklimkov@nvidia.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-05-30 16:11:00 +08:00
boatman and sphinxkkkbc
4d9262a6eb
[Feat]Support voice clone for omnivoice in online serving & add seed parameter for reproducible ( #3668 )
...
Signed-off-by: sphinxkkkbc <binchengkang8@gmail.com >
Co-authored-by: sphinxkkkbc <binchengkang8@gmail.com >
2026-05-29 07:42:08 +00:00
8f45e68bcd
[Higgs-Audio] bosonai/higgs-audio-v2-generation-3B-base TTS model support ( #3762 )
...
Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com >
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-05-25 23:16:07 +08:00
Yueqian Lin and Rein Yang
e996723bc9
[Bugfix] Fix Qwen3-TTS Stage 0 prefix-caching correctness ( #3665 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Signed-off-by: Rein Yang <ruiruyang2@gmail.com >
Co-authored-by: Rein Yang <ruiruyang2@gmail.com >
the ci failure case is irrelavent to this PR and occurs rarely
2026-05-25 23:09:38 +08:00
BeatSeat and Yueqian Lin
28ce618f0e
[Model] Add GLM-TTS text-to-speech model support ( #3141 )
...
Signed-off-by: BeatSeat <wendavid552@gmail.com >
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-05-24 05:42:26 +08:00
Yueqian Lin
fc8486cf20
[Doc][TTS] CosyVoice3 online docs + residual TTS yaml cleanup + remove VoxCPM v1 ( #3748 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
2026-05-20 15:24:06 +08:00
Nick Cao and Claude
d4e90bbba8
[Bugfix][TTS] Only populate voice_name for uploaded voices without inline ref_audio ( #3523 )
...
Signed-off-by: Nick Cao <ncao@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-05-20 04:36:59 +00:00
+20
eb4e60ee64
[Rebase] Rebase to vllm v0.21.0 ( #3530 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
Signed-off-by: NumberWan <wantszkin2003@gmail.com >
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com >
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com >
Signed-off-by: Dnoob <dxpouo@gmail.com >
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com >
Signed-off-by: rein yang <ruiruyang2@gmail.com >
Signed-off-by: Nick Cao <ncao@redhat.com >
Signed-off-by: zhumingjue <zhumingjue@huawei.com >
Signed-off-by: Ricardo Noriega De Soto <rnoriega@redhat.com >
Signed-off-by: lyj-jjj <liuyingjun5@huawei.com >
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
Signed-off-by: wangyu <410167048@qq.com >
Signed-off-by: weizhoublue <weizhoublue@github.com >
Signed-off-by: weizhou.lan@daocloud.io <weizhou.lan@daocloud.io >
Signed-off-by: dengyunyang <584797741@qq.com >
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com >
Signed-off-by: David Chen <530634352@qq.com >
Signed-off-by: Jie Liu <33612777+keeper-jie@users.noreply.github.com >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: natureofnature <wzliu@connect.hku.hk >
Signed-off-by: bjf-frz <frz123db@gmail.com >
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
Signed-off-by: KexiongYu <yukexiong1@huawei.com >
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com >
Signed-off-by: Zeyu Huang | 黃澤宇 <11222265+fhfuih@users.noreply.github.com >
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com >
Co-authored-by: NumberWan <wantszkin2003@gmail.com >
Co-authored-by: dsinghvi <divyanshsinghvi@gmail.com >
Co-authored-by: Dnoob <dxpouo@gmail.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: knlnguyen1802 <knlnguyen1802@gmail.com >
Co-authored-by: Samit <285365963@qq.com >
Co-authored-by: rein yang <73573651+R2-Y@users.noreply.github.com >
Co-authored-by: Nick Cao <ncao@redhat.com >
Co-authored-by: zhumingjue138 <zhumingjue@huawei.com >
Co-authored-by: Ricardo Noriega <rnoriega@redhat.com >
Co-authored-by: lyj-jjj <liuyingjun5@huawei.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: gcanlin <canlinguosdu@gmail.com >
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com >
Co-authored-by: weizhoublue <45163302+weizhoublue@users.noreply.github.com >
Co-authored-by: weizhoublue <weizhoublue@github.com >
Co-authored-by: dengyunyang <584797741@qq.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com >
Co-authored-by: Jie Liu <33612777+keeper-jie@users.noreply.github.com >
Co-authored-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: NATURE <wzliu@connect.hku.hk >
Co-authored-by: bjf-frz <frz123db@gmail.com >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
Co-authored-by: Y. Fisher <yukexiong1@huawei.com >
Co-authored-by: Zeyu Huang | 黃澤宇 <11222265+fhfuih@users.noreply.github.com >
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com >
2026-05-16 18:18:43 +08:00
d7ea5d5979
[New Model] Add support for tencent/Covo-Audio-Chat ( #2293 )
...
Signed-off-by: Dnoob <dxpouo@gmail.com >
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-05-12 14:39:58 +08:00
Saad Al-Tohamy and Gao Han
fe72d078ca
[FIX] Ensure extra_params are correctly merged into sampling params in _create_diffusion_speech() ( #3320 )
...
Signed-off-by: saadaltohamy <saad_altohamy@yahoo.com >
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk >
2026-05-12 01:10:04 -04:00
Dan
c481ccee2b
[Perf] Optimize VoxCPM2 first-request latency via startup warmup ( #3424 )
...
Signed-off-by: Dan250124 <416947747@qq.com >
2026-05-09 06:44:19 +08:00
Sy03
2b85474c23
[Bugfix] Propagate seed to Qwen3-TTS Fast AR sampler ( #3350 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
2026-05-05 14:57:55 -04:00
Juan Pablo Zuluaga and JuanPZuluaga
b64ab05d09
[TTS][SpeakerCacheManager] A global speaker cache manager for Voice Cloning ( #2630 )
...
Signed-off-by: JuanPZuluaga <juanz9312@gmal.com >
Co-authored-by: JuanPZuluaga <juanz9312@gmal.com >
2026-05-05 10:53:45 -04:00
Nick Cao and Claude
5fc0bfe01e
[Cleanup] Use tokens_input() for TTS prompt construction ( #3227 )
...
Signed-off-by: Nick Cao <ncao@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-05-04 10:17:25 -04:00
GuoSheng Feng and Hongsheng Liu
5dabbb2f18
[Bugfix] Map Qwen3-TTS max_new_tokens to max_tokens ( #3217 )
...
Signed-off-by: sfiisf <sfiisf@163.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-05-03 04:36:15 +00:00
Yueqian Lin and Hongsheng Liu
7491b67446
[Bugfix][MOSS-TTS-Nano] Drop fictional voice presets, require ref_audio ( #3192 )
...
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-05-03 16:35:35 +12:00
Jason
38d440109b
[Bugfix] Validate Qwen3-TTS speaker embedding dimensions before engine execution ( #3191 )
...
Signed-off-by: Jiahui Sun <jhsun2020@gmail.com >
2026-05-01 18:39:49 +00:00
Zhou Taichang and Yuanheng Zhao
7f2b1e9386
[Rebase] Rebase to vllm 0.20.0 ( #3232 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com >
Co-authored-by: Yuanheng Zhao <54058983+yuanheng-zhao@users.noreply.github.com >
2026-04-30 09:56:59 +12:00
4c2bea782e
Fix "Add online serving to Stable Audio Diffusion and introduce v1/audio/generate endpoint" ( #1794 )
...
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Co-authored-by: Yueqian Lin <linyueqian@outlook.com >
2026-04-28 18:06:57 +00:00
Hongsheng Liu
7a8b428bdb
Revert "[Doc][Frontend][Model][VoxCPM2] Support instructions and per-request cfg_value" ( #3204 )
2026-04-28 14:29:49 +08:00
Alfian Firmansyah and Hongsheng Liu
1069a144a8
[Doc][Frontend][Model][VoxCPM2] Support instructions and per-request cfg_value ( #3118 )
...
Signed-off-by: gnomefin <alfian@uselevers.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-04-28 11:24:04 +08:00
Yueqian Lin and Hongsheng Liu
afa2b09684
feat: add MOSS-TTS-Nano single-stage TTS support ( #2753 )
...
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-04-27 10:26:26 +08:00
62c86c9d62
[Bugfix] Add seed support to TTS API for deterministic Fish Speech voice generation ( #2624 )
...
Signed-off-by: Yiyang Liu <yiyangliu@microsoft.com >
Signed-off-by: Yiyang Liu <37043548+ianliuy@users.noreply.github.com >
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-04-26 19:15:30 +00:00
e03ab82dd2
[Refactor/Bugfix] Use Delta Messages for Streaming ( #2911 )
...
Signed-off-by: Alex Brooks <albrooks@redhat.com >
Signed-off-by: Sy03 <1370724210@qq.com >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Co-authored-by: Sy03 <1370724210@qq.com >
Co-authored-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-04-26 15:11:53 +12:00