[Feature] timestamps for TTS via a forced-aligner pooling (#4795)

Signed-off-by: wjinxu <1299461899@qq.com>
Co-authored-by: xurui <xurui1@modelbest.onaliyun.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
authored and GitHub committed 2026-08-12 05:33:34 +00:00
1 parent a15b5b4bcd
commit 9b0df3909a
28 files changed
+850 -627

No files matched your search

@@ -372,18 +372,10 @@ python examples/offline_inference/text_to_speech/qwen3_tts/end2end.py \
Streaming requires `async_chunk: true` in the stage config.
### Word Timestamps
Generate a WAV offline and a JSON sidecar with word-level timestamps from
`Qwen/Qwen3-ForcedAligner-0.6B`:
```bash
python examples/offline_inference/text_to_speech/qwen3_tts/word_timestamps.py \
--model Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice \
--forced-aligner Qwen/Qwen3-ForcedAligner-0.6B \
--text "Hello world." \
--output-dir /tmp/qwen3_tts_timestamps
```
The script writes `qwen3_tts_word_timestamps.wav` and
`qwen3_tts_word_timestamps.json`. On machines without a local CUDA toolkit,
set `VLLM_USE_FLASHINFER_SAMPLER=0` to avoid FlashInfer sampler JIT.
Word-level timestamps are currently a serving-path feature: launch
`vllm-omni serve` with `--forced-aligner` and request `word_timestamps`
(see `examples/online_serving/text_to_speech/README.md`). An offline
example is not provided in this release.
### Batched decoding
The Code2Wav stage supports batched decoding through the SpeechTokenizer. Pass multiple prompts via `--txt-prompts` and set `--batch-size` accordingly. To raise `max_num_seqs` on either stage, point `--stage-configs-path` at a stage configs YAML with the desired values (see `vllm_omni/model_executor/stage_configs/` for templates):