[Feat][Qwen3TTS] reduce TTFA with flexible initial phase (#1583)

Signed-off-by: pablo <pablo@agigo.ai>
Co-authored-by: pablo <pablo@agigo.ai>
This commit is contained in:
Juan Pablo Zuluagaandpablo authored and GitHub committed 2026-03-05 23:14:41 +08:00
1 parent 070ea0ddc3
commit 6a45efb287
14 files changed
+202 -44

No files matched your search

+1 -1
View File
@@ -19,7 +19,7 @@ The `async_chunk` feature enables asynchronous, chunked processing of data acros
For qwen3-omni:
- **Thinker → Talker**: Per decode step (typically chunk_size=1)
- **Talker → Code2Wav**: Accumulated to code2wav chunk_size(default=25, currently only support default, will support chunk_size soon) before sending
- **Talker → Code2Wav**: Accumulated to `codec_chunk_frames` (default=25) before sending. Set `initial_codec_chunk_frames` to emit smaller chunks during the initial phase for reduced TTFA
- **Code2Wav**: Streaming decode with code2wav chunk_size
With `async_chunk`:
+1
View File
@@ -96,6 +96,7 @@ Content-Type: application/json
| `language` | string | "Auto" | Language (see supported languages below) |
| `instructions` | string | "" | Voice style/emotion instructions |
| `max_new_tokens` | integer | 2048 | Maximum tokens to generate |
| `initial_codec_chunk_frames` | integer | null | Initial chunk size for reduced TTFA (overrides stage config) |
**Supported languages:** Auto, Chinese, English, Japanese, Korean, German, French, Russian, Portuguese, Spanish, Italian
@@ -98,7 +98,8 @@ Add `--streaming` to stream audio chunks progressively via `AsyncOmni` (requires
python end2end.py --query-type CustomVoice --streaming --output-dir /tmp/out_stream
```
Each 25-frame Code2Wav chunk is logged as it arrives. The final WAV file is written once generation
Each Code2Wav chunk is logged as it arrives (default 25 frames; configurable via `codec_chunk_frames`
and `initial_codec_chunk_frames` in the stage config). The final WAV file is written once generation
completes. This demonstrates that audio data is available progressively rather than only at the end.
> **Note:** Streaming uses `AsyncOmni` internally. The non-streaming path (`Omni`) is unchanged.
@@ -268,7 +268,7 @@ Returns binary audio data with appropriate `Content-Type` header (e.g., `audio/w
## Streaming
Set `stream=true` with `response_format="pcm"` to receive raw PCM audio chunks as they are decoded
(one chunk per 25-frame Code2Wav window):
(one chunk per Code2Wav window, default 25 frames; configurable in the stage config):
```bash
curl -X POST http://localhost:8091/v1/audio/speech \