Yueqian Lin
74522ca3ec
Merge branch 'main' into feat/moss-tts-nano
...
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-04-24 11:00:13 -04:00
3a32cd6780
[Model] Ming-flash-omni-2.0 Omni-Speech and TTS ( #2890 )
...
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com >
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com >
Signed-off-by: LHXuuu <xulianhao.xlh@antgroup.com >
Co-authored-by: LHXuuu <xulianhao.xlh@antgroup.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-04-23 12:04:52 +08:00
Chen-Yo Sun and Yueqian Lin
8e1bbc9ab5
[Feature][Voxtral TTS] Support per-model extra sampling params & add cfg_alpha for Voxtral TTS ( #2338 )
...
Signed-off-by: Chen-Yo Sun <chenyo.sun@mistral.ai >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Yueqian Lin <linyueqian@outlook.com >
2026-04-22 03:49:50 +00:00
Yueqian Lin
26bbe7b527
fix(moss_tts_nano): address PR review feedback
...
- Wire ref_audio through /v1/audio/speech: _build_moss_tts_params is now
async, resolves ref_audio via MediaConnector and passes the waveform as
prompt_audio_array so the model can materialise a temp WAV for upstream
inference_stream (previously voice cloning via the REST endpoint was a
no-op). Serving now also requires ref_text when ref_audio is provided.
- Fix per-request EOS in batched decode: replace the shared
_ar_emit_stop_token bool with a _ar_last_chunk_flags list so
compute_logits emits EOS per row; finished requests no longer wait for
the slowest peer in a max_num_seqs=4 batch.
- Snapshot and restore CPU+CUDA RNG state around torch.manual_seed to
limit global-state bleed; add comment noting deterministic output
under concurrent batching is best-effort (upstream inference_stream
uses the global RNG).
- Align _DEFAULT_MODE with the offline example and tests ("voice_clone").
- Clean up outdated "buffer first" comment in _create_stream_gen; document
single-threaded AR-worker assumption for _stream_gens; add one-liner
explaining the load_weights iterator drain.
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
2026-04-21 17:58:50 -04:00
Yueqian Lin
3bd77d3f8f
Merge branch 'main' into feat/moss-tts-nano
...
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-04-21 17:50:21 -04:00
52b53367ef
[Enhancement] Engine runtime errors ( #2426 )
...
Signed-off-by: Daniel Huang <daniel1.huang@intel.com >
Signed-off-by: Daniel Huang <pilotflyer824@gmail.com >
Signed-off-by: bjf-frz <frz123db@gmail.com >
Co-authored-by: Chenguang Zheng <645327136@qq.com >
Co-authored-by: Zhou Taichang <tzhouam@connect.ust.hk >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: bjf-frz <frz123db@gmail.com >
2026-04-21 14:46:05 +08:00
Sy03
a683b1dd75
[Bugfix][VoxCPM2] Fix voice-clone decode loop by padding prefill prompt ( #2894 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
2026-04-18 14:04:44 +00:00
Sy03
7d64a7c996
[BugFix][VoxCPM2]: split multichar Chinese tokens to match training tokenization ( #2832 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
2026-04-16 10:07:48 -05:00
4bf4c63147
Add voxcpm model support. ( #2467 )
...
Signed-off-by: Celeste-jq <591998922@qq.com >
Signed-off-by: lyj-jjj <liuyingjun5@huawei.com >
Signed-off-by: IsleOfDawnlight <stellamou@qq.com >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Celeste-jq <591998922@qq.com >
Co-authored-by: lyj-jjj <liuyingjun5@huawei.com >
Co-authored-by: Yueqian Lin <linyueqian@outlook.com >
2026-04-15 15:04:58 +08:00
Yueqian Lin
fc81b8504a
feat(moss-tts-nano): add online serving, Gradio demo, and fix /v1/audio/speech integration
...
- Register moss_tts_nano as TTS model stage in serving_speech.py
(model detection, validation, prompt building)
- Fix registry mod_relname: moss_tts_nano -> modeling_moss_tts_nano
- Fix stage config: is_comprehension=true (required for generate task)
- Fix default mode: voice_clone -> continuation (built-in presets)
- Add compute_logits stub for VllmModelForTextGeneration protocol
- Remove unused _sentinel nn.Parameter
- Add Gradio demo with AudioWorklet streaming player (48kHz stereo)
- Add online/offline serving docs and launch scripts
TODO: Single-stage generation models don't support true streaming
(progressive audio chunks). Current TTFP = full generation time.
Needs multi-step scheduling support in GPUGenerationModelRunner.
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
2026-04-14 13:06:44 -04:00
Bvicii
4fb078a031
[Bugfix] Reject /v1/audio/speech for Qwen omni models ( #2763 )
...
Signed-off-by: Bvicii <yizhanhuang2002@gmail.com >
2026-04-14 12:53:06 +00:00
14f7910900
[Perf]: Speedup VoxCPM2 TTS performance and Support PagedAttention ( #2690 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-04-14 09:16:47 +12:00
Juan Pablo Zuluaga
f7e8df9dbd
[TTS][OmniVoice] Add voice cloning support for OmniVoice TTS ( #2676 )
...
Signed-off-by: JuanPZuluaga <juanz9312@gmail.com >
2026-04-11 12:14:03 -04:00
teith and Yueqian Lin
001f2e32e2
[BUG FIX]: prevent EngineCore crash when Qwen TTS Base task is missing ref_text ( #2203 )
...
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-04-11 02:46:38 +00:00
Yueqian Lin
94232436bb
[Feat][FishSpeech] Cache DAC-encoded ref audio for voice cloning ( #2609 )
...
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-04-10 10:34:03 +08:00
Sy03
694be6f2e1
[Bugfix] Fix Fish Speech voice clone FileNotFoundError on multi-GPU ( #2606 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
2026-04-09 20:34:34 +00:00
vveerrgg and vveerrgg
2c6c07c4f6
[TTS] Add missing _generate_pcm_chunks for OmniOpenAIServingSpeech streaming ( #2569 )
...
Signed-off-by: Yueqian Lin <pandaleefree@gmail.com >
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Co-authored-by: vveerrgg <vveerrgg@users.noreply.github.com >
2026-04-08 22:20:33 +00:00
Markus / Mark and marksverdhai
feefdaee9f
[Bugfix] Accept 'speaker' as alias for 'voice' in TTS speech API ( #2424 )
...
Signed-off-by: marksverdhei <marksverdhei@hotmail.com >
Signed-off-by: marksverdhai <249650165+marksverdhai@users.noreply.github.com >
Co-authored-by: marksverdhai <249650165+marksverdhai@users.noreply.github.com >
2026-04-07 15:07:01 +00:00
Bvicii
badbe8eb0f
[RFC] Offload blocking TTS/speech ops to thread pool to unblock event loop ( #2511 )
...
Signed-off-by: Bvicii <yizhanhuang2002@gmail.com >
2026-04-07 00:17:04 -04:00
indevn and linyueqian
6fc38e0467
[Model][Core] Enable async_chunk streaming pipeline for CosyVoice3 ( #1703 )
...
Signed-off-by: linyueqian <linyueqian@outlook.com >
Signed-off-by: indevn <indevn@outlook.com >
Co-authored-by: linyueqian <linyueqian@outlook.com >
2026-04-05 06:11:33 +00:00
Zhou Taichang
2804a85e58
[Rebase] Rebase to vllm v0.19.0 ( #2475 )
2026-04-04 15:03:32 +08:00
Yueqian Lin
4c031580cf
[CosyVoice3] Add online serving support, fix stage config, and add CI tests ( #2431 )
...
Signed-off-by: linyueqian <linyueqian@outlook.com >
2026-04-04 14:55:15 +13:00
+11
f50c5a413f
[Qwen3TTS] [TTS] [Feat] Refactor voice cache manager ( #2108 )
...
Signed-off-by: JuanPZuluaga <juanz9312@gmal.com >
Signed-off-by: yiliu30 <yi4.liu@intel.com >
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
Signed-off-by: Binh Tang <tangbinhna@gmail.com >
Signed-off-by: Binh Tang <binht@netflix.com >
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com >
Signed-off-by: Rein Yang <ruiruyang2@gmail.com >
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn >
Signed-off-by: vraiti <vraiti@redhat.com >
Signed-off-by: Songrui625 <songrui625@gmail.com >
Signed-off-by: Lidang Jiang <lidangjiang@gmail.com >
Signed-off-by: Lidang-Jiang <lidangjiang@gmail.com >
Signed-off-by: Alex Brooks <albrooks@redhat.com >
Co-authored-by: JuanPZuluaga <juanz9312@gmal.com >
Co-authored-by: Yi Liu <yi4.liu@intel.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com >
Co-authored-by: Binh Tang <tangbinhna@gmail.com >
Co-authored-by: Binh Tang <binht@netflix.com >
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com >
Co-authored-by: rein yang <73573651+R2-Y@users.noreply.github.com >
Co-authored-by: zhumingjue138 <zhumingjue@huawei.com >
Co-authored-by: ChenWenjing <54166744+Shirley125@users.noreply.github.com >
Co-authored-by: vraiti <vraiti@redhat.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: Sy03 <1370724210@qq.com >
Co-authored-by: chickeyton <ngton2014@gmail.com >
Co-authored-by: Jerry Song <46962917+Songrui625@users.noreply.github.com >
Co-authored-by: Lidang Jiang <119769478+Lidang-Jiang@users.noreply.github.com >
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com >
Co-authored-by: Alex Brooks <albrooks@redhat.com >
Co-authored-by: linyueqian <linyueqian@outlook.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-04-03 16:51:58 -04:00
Yueqian Lin
0e83ebe1d4
[OmniVoice] Add two-stage TTS serving support ( #2463 )
...
Signed-off-by: linyueqian <linyueqian@outlook.com >
2026-04-03 20:41:20 +00:00
Sy03
d9912eb6e9
[Bugfix] Fix Fish Speech S2 Pro prompt handling for truncated audio & emotion tag ( #2268 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
2026-03-28 07:00:00 +08:00
Juan Pablo Zuluaga and Yueqian Lin
574ec99ef4
[Qwen3TTS][ServingSpeech] Bugfix/voice upload and add optional ref_text ( #2046 )
...
Signed-off-by: JuanPZuluaga <juanz9312@gmail.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-03-26 14:26:52 +00:00
29936a3edd
[Frontend] Speaker embedding support for speech and voices APIs ( #1227 )
...
Signed-off-by: marksverdhei <marksverdhei@hotmail.com >
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Signed-off-by: linyueqian <linyueqian@outlook.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Co-authored-by: linyueqian <linyueqian@outlook.com >
2026-03-25 19:02:48 +00:00
Sy03
fc32da7624
[Perf] [TTS] Improve Fish Speech S2 Pro voice cloning TTFP ( #2145 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
2026-03-25 11:41:10 -04:00
Sy03
44770f9f20
[Fix] Qwen3 TTS audio handling for long ref_audio ( #2104 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
Signed-off-by: Sy0307 <1370724210@qq.com >
2026-03-24 20:02:52 +00:00
dsinghvi
e7e05a55cf
[Feature] Speech batch entrypoint ( #1701 )
...
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com >
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com >
2026-03-24 10:52:01 -04:00
rein yang
3cdf28828e
[Feature] support to change the speaker of qwen3-omni ( #1963 )
...
Signed-off-by: Rein Yang <ruiruyang2@gmail.com >
2026-03-24 13:53:24 +08:00
Yueqian Lin
15f9d60dd4
[Bugfix] Fix Fish Speech and CosyVoice3 online serving - missing is_comprehension and broken model detection ( #2058 )
...
Signed-off-by: linyueqian <linyueqian@outlook.com >
2026-03-21 22:18:10 +08:00
Chen-Yo Sun
e9fe78e28c
[Model] Add Voxtral TTS model ( #1803 )
...
Signed-off-by: Chen-Yo Sun <chenyo.sun@mistral.ai >
2026-03-19 16:02:11 -04:00
3158e9d28f
[Entrypoint][Refactor] vLLM-Omni Entrypoint Refactoring ( #1908 )
...
Signed-off-by: yinpe <11810305@mail.sustech.edu.cn >
Signed-off-by: Peiqi Yin <yinpeiqi809@gmail.com >
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com >
Signed-off-by: wuhang <whlbx@hotmail.com >
Signed-off-by: linyueqian <linyueqian@outlook.com >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: lishunyang <lishunyang12@163.com >
Signed-off-by: Chenguang ZHENG <645327136@qq.com >
Signed-off-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com >
Co-authored-by: yinpe <11810305@mail.sustech.edu.cn >
Co-authored-by: Peiqi Yin <yinpeiqi809@gmail.com >
Co-authored-by: wuhang <whlbx@hotmail.com >
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com >
Co-authored-by: linyueqian <linyueqian@outlook.com >
Co-authored-by: princepride <wangzhipeng628@gmail.com >
Co-authored-by: lishunyang <lishunyang12@163.com >
2026-03-18 17:33:53 +08:00
Yueqian Lin
366b3368ea
Add Fish Speech S2 Pro support with online serving and voice cloning ( #1798 )
...
Signed-off-by: linyueqian <linyueqian@outlook.com >
2026-03-12 16:20:52 -04:00
rongfu.leng
7c5159037f
[Feat] add wav response_format when stream is true in /v1/audio/speec… ( #1819 )
...
Signed-off-by: rongfu.leng <lenronfu@gmail.com >
2026-03-12 17:45:13 +08:00
5d28b6a40c
[Feat][Qwen3-TTS] Support streaming audio output for websocket ( #1719 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Signed-off-by: linyueqian <linyueqian@outlook.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Co-authored-by: linyueqian <linyueqian@outlook.com >
2026-03-11 23:48:41 -04:00
Hongsheng Liu
4027534dbf
Revert "Add online serving to Stable Audio Diffusion and introduce v1/audio/generate endpoint" ( #1789 )
2026-03-10 22:40:48 +08:00
Ekagra Ranjan
7a560492a2
Add online serving to Stable Audio Diffusion and introduce v1/audio/generate endpoint ( #1255 )
2026-03-09 13:36:46 -04:00
Sy03
761eff930a
[Fix][Qwen3-TTS] Preserve ref_code decoder context for Base ICL ( #1731 )
2026-03-09 13:36:27 -04:00
41964cb90a
feat(tts): add voice upload API for Qwen3-TTS ( #1201 )
...
Signed-off-by: linyueqian <linyueqian@outlook.com >
Co-authored-by: linyueqian <linyueqian@outlook.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-03-09 11:02:11 +08:00
Yueqian Lin
a9f694b08a
[Bugfix] Use upstream MediaConnector for ref_audio resolution ( #1661 )
...
Signed-off-by: linyueqian <linyueqian@outlook.com >
2026-03-09 01:38:26 +00:00
Canlin Guo
846167b237
[1/N][Refactor] Clean up dead code in output processor ( #1579 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
2026-03-06 14:45:53 +08:00
Juan Pablo Zuluaga and pablo
6a45efb287
[Feat][Qwen3TTS] reduce TTFA with flexible initial phase ( #1583 )
...
Signed-off-by: pablo <pablo@agigo.ai >
Co-authored-by: pablo <pablo@agigo.ai >
2026-03-05 23:14:41 +08:00
Yueqian Lin
35bd3ff7d2
[Bugfix] Add TTS request validation to prevent engine crashes ( #1641 )
...
Signed-off-by: linyueqian <linyueqian@outlook.com >
2026-03-04 14:27:02 +08:00
Yueqian Lin
37775ffe60
fix(qwen3-tts): fix Base ICL voice clone producing corrupted audio ( #1554 )
...
Signed-off-by: linyueqian <linyueqian@outlook.com >
2026-02-28 11:49:47 +08:00
03c01e0415
[Qwen3TTS][Feat] Streaming output ( #1438 )
...
Signed-off-by: pablo <pablo@agigo.ai >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: pablo <pablo@agigo.ai >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-02-26 21:02:35 +08:00
Yanick Schraner
97f4c5b1e7
[Bugfix] Add TTS configuration options ( #1177 )
...
Signed-off-by: Yanick Schraner <yanick.schraner@bs.ch >
2026-02-26 00:14:44 +08:00
Sy03 and Hongsheng Liu
efbe4116ff
[Feature] : Support disaggregated inference pipeline for Qwen3_TTS ( #1161 )
...
Signed-off-by: Sy03 <1370724210@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-02-21 07:36:44 +08:00
Zhou Taichang and 汪志鹏
70ef548448
Dev/rebase v0.15.0 ( #1159 )
...
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
2026-02-03 12:24:45 +08:00