[Feat] Streaming TTS word-level timestamps via forced aligner (#4034)
Signed-off-by: wjinxu <1299461899@qq.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
17 files changed
+2162
-20
No files matched your search
@@ -504,6 +504,35 @@ python qwen3_tts/streaming_speech_client.py --text "Hello world. How are you? I
|
||||
python qwen3_tts/streaming_speech_client.py --text "..." --simulate-stt --stt-delay 0.1
|
||||
```
|
||||
|
||||
To receive word-level timestamps, launch the server with a forced aligner:
|
||||
```bash
|
||||
vllm-omni serve Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice \
|
||||
--omni \
|
||||
--deploy-config vllm_omni/deploy/qwen3_tts.yaml \
|
||||
--trust-remote-code \
|
||||
--forced-aligner Qwen/Qwen3-ForcedAligner-0.6B
|
||||
```
|
||||
Then request PCM JSON sidecar chunks:
|
||||
```bash
|
||||
python qwen3_tts/streaming_speech_client.py \
|
||||
--text "Hello world. How are you?" \
|
||||
--stream-audio \
|
||||
--response-format pcm \
|
||||
--word-timestamps
|
||||
```
|
||||
The client writes one PCM file per sentence and a matching
|
||||
`sentence_XXX_timestamps.json` sidecar.
|
||||
|
||||
To *see* the alignment instead of reading a JSON sidecar, run the
|
||||
word-timestamp Gradio demo (server must be launched with `--forced-aligner`):
|
||||
```bash
|
||||
python qwen3_tts/word_timestamps_demo.py --api-base http://localhost:8091
|
||||
```
|
||||
Each sentence's audio plays in an `<audio>` element while its text is rendered
|
||||
as inline word spans; the current word highlights as `audio.currentTime`
|
||||
crosses each `start_ms`. The **Stop (barge-in)** button cuts playback and
|
||||
reports the last-spoken word, useful for the voice-agent barge-in case.
|
||||
|
||||
### Gradio demos
|
||||
```bash
|
||||
./qwen3_tts/run_gradio_demo.sh # CustomVoice (default)
|
||||
|
||||
Reference in new issue
Block a user