[Feature] Flush the speech WebSocket on input.done without closing it (#5517)

Signed-off-by: iancarrasco-b10 <ian.carrasco@baseten.co>
This commit is contained in:
iancarrasco-b10 authored and GitHub committed 2026-07-28 23:46:19 +00:00
1 parent 0feb92bdfa
commit 23bb4395bb
8 files changed
+688 -302

No files matched your search

@@ -422,12 +422,19 @@ curl -X POST http://localhost:8091/v1/audio/speech \
Raw PCM streaming requires `stream_format="audio"`, `response_format="pcm"`, and `async_chunk: true` on the stage config (default in `qwen3_tts.yaml`). `speed` is not supported when streaming.
### Streaming WebSocket
The `/v1/audio/speech/stream` endpoint accepts text incrementally, splits it at sentence boundaries, and emits one PCM stream per sentence:
The `/v1/audio/speech/stream` endpoint accepts text incrementally and synthesizes the buffered text as one continuous request on `input.done`:
```bash
python qwen3_tts/streaming_speech_client.py --text "Hello world. How are you? I am fine."
python qwen3_tts/streaming_speech_client.py --text "..." --simulate-stt --stt-delay 0.1
```
`input.done` flushes without closing, so repeating `--text` synthesizes several utterances over one connection:
```bash
python qwen3_tts/streaming_speech_client.py \
--text "First utterance." \
--text "Second utterance, same connection."
```
### Gradio demos
```bash
./qwen3_tts/run_gradio_demo.sh # CustomVoice (default)