[Feat][Qwen3TTS] reduce TTFA with flexible initial phase (#1583)

Signed-off-by: pablo <pablo@agigo.ai>
Co-authored-by: pablo <pablo@agigo.ai>
This commit is contained in:
Juan Pablo Zuluagaandpablo authored and GitHub committed 2026-03-05 23:14:41 +08:00
1 parent 070ea0ddc3
commit 6a45efb287
14 files changed
+202 -44

No files matched your search

@@ -95,7 +95,8 @@ Add `--streaming` to stream audio chunks progressively via `AsyncOmni` (requires
python end2end.py --query-type CustomVoice --streaming --output-dir /tmp/out_stream
```
Each 25-frame Code2Wav chunk is logged as it arrives. The final WAV file is written once generation
Each Code2Wav chunk is logged as it arrives (default 25 frames; configurable via `codec_chunk_frames`
and `initial_codec_chunk_frames` in the stage config). The final WAV file is written once generation
completes. This demonstrates that audio data is available progressively rather than only at the end.
> **Note:** Streaming uses `AsyncOmni` internally. The non-streaming path (`Omni`) is unchanged.
@@ -7,6 +7,7 @@ tasks, then runs Omni generation and saves output wav files.
import asyncio
import logging
import os
import time
from typing import Any, NamedTuple
import soundfile as sf
@@ -337,13 +338,27 @@ async def main_streaming(args):
for i, prompt in enumerate(inputs):
request_id = str(i)
t_start = time.perf_counter()
t_prev = t_start
chunk_idx = 0
async for stage_output in omni.generate(prompt, request_id=request_id):
mm = stage_output.request_output.outputs[0].multimodal_output
if not stage_output.finished:
t_now = time.perf_counter()
audio = mm.get("audio")
n = len(audio) if isinstance(audio, list) else (0 if audio is None else 1)
logger.info(f"Request {request_id}: received chunk {n}")
dt_ms = (t_now - t_prev) * 1000
ttfa_ms = (t_now - t_start) * 1000
if chunk_idx == 0:
logger.info(f"Request {request_id}: chunk {chunk_idx} samples={n} TTFA={ttfa_ms:.1f}ms")
else:
logger.info(f"Request {request_id}: chunk {chunk_idx} samples={n} inter_chunk={dt_ms:.1f}ms")
t_prev = t_now
chunk_idx += 1
else:
t_end = time.perf_counter()
total_ms = (t_end - t_start) * 1000
logger.info(f"Request {request_id}: done total={total_ms:.1f}ms chunks={chunk_idx}")
_save_wav(output_dir, request_id, mm)
+2 -1
View File
@@ -250,6 +250,7 @@ Returns binary audio data with appropriate `Content-Type` header (e.g., `audio/w
| `language` | string | "Auto" | Language (see supported languages below) |
| `instructions` | string | "" | Voice style/emotion instructions |
| `max_new_tokens` | int | 2048 | Maximum tokens to generate |
| `initial_codec_chunk_frames` | int | null | Initial chunk size for reduced TTFA (overrides stage config) |
| `stream` | bool | false | Stream raw PCM chunks as they are decoded (requires `response_format="pcm"`) |
**Supported languages:** Auto, Chinese, English, Japanese, Korean, German, French, Russian, Portuguese, Spanish, Italian
@@ -265,7 +266,7 @@ Returns binary audio data with appropriate `Content-Type` header (e.g., `audio/w
## Streaming
Set `stream=true` with `response_format="pcm"` to receive raw PCM audio chunks as they are decoded
(one chunk per 25-frame Code2Wav window):
(one chunk per Code2Wav window, default 25 frames; configurable in the stage config):
```bash
curl -X POST http://localhost:8091/v1/audio/speech \