[Feat][Qwen3TTS] reduce TTFA with flexible initial phase (#1583)
Signed-off-by: pablo <pablo@agigo.ai> Co-authored-by: pablo <pablo@agigo.ai>
This commit is contained in:
1 parent
070ea0ddc3
commit
6a45efb287
14 files changed
+202
-44
No files matched your search
@@ -19,7 +19,7 @@ The `async_chunk` feature enables asynchronous, chunked processing of data acros
|
||||
|
||||
For qwen3-omni:
|
||||
- **Thinker → Talker**: Per decode step (typically chunk_size=1)
|
||||
- **Talker → Code2Wav**: Accumulated to code2wav chunk_size(default=25, currently only support default, will support chunk_size soon) before sending
|
||||
- **Talker → Code2Wav**: Accumulated to `codec_chunk_frames` (default=25) before sending. Set `initial_codec_chunk_frames` to emit smaller chunks during the initial phase for reduced TTFA
|
||||
- **Code2Wav**: Streaming decode with code2wav chunk_size
|
||||
|
||||
With `async_chunk`:
|
||||
|
||||
@@ -96,6 +96,7 @@ Content-Type: application/json
|
||||
| `language` | string | "Auto" | Language (see supported languages below) |
|
||||
| `instructions` | string | "" | Voice style/emotion instructions |
|
||||
| `max_new_tokens` | integer | 2048 | Maximum tokens to generate |
|
||||
| `initial_codec_chunk_frames` | integer | null | Initial chunk size for reduced TTFA (overrides stage config) |
|
||||
|
||||
**Supported languages:** Auto, Chinese, English, Japanese, Korean, German, French, Russian, Portuguese, Spanish, Italian
|
||||
|
||||
|
||||
@@ -98,7 +98,8 @@ Add `--streaming` to stream audio chunks progressively via `AsyncOmni` (requires
|
||||
python end2end.py --query-type CustomVoice --streaming --output-dir /tmp/out_stream
|
||||
```
|
||||
|
||||
Each 25-frame Code2Wav chunk is logged as it arrives. The final WAV file is written once generation
|
||||
Each Code2Wav chunk is logged as it arrives (default 25 frames; configurable via `codec_chunk_frames`
|
||||
and `initial_codec_chunk_frames` in the stage config). The final WAV file is written once generation
|
||||
completes. This demonstrates that audio data is available progressively rather than only at the end.
|
||||
|
||||
> **Note:** Streaming uses `AsyncOmni` internally. The non-streaming path (`Omni`) is unchanged.
|
||||
|
||||
@@ -268,7 +268,7 @@ Returns binary audio data with appropriate `Content-Type` header (e.g., `audio/w
|
||||
## Streaming
|
||||
|
||||
Set `stream=true` with `response_format="pcm"` to receive raw PCM audio chunks as they are decoded
|
||||
(one chunk per 25-frame Code2Wav window):
|
||||
(one chunk per Code2Wav window, default 25 frames; configurable in the stage config):
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:8091/v1/audio/speech \
|
||||
|
||||
Reference in new issue
Block a user