anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

Feat: non_streaming_mode for Qwen3-TTS Base Models During Online Inference (#4198)

Browse Source 
Signed-off-by: nagisa-kun <1434936049@qq.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Sy03 <1370724210@qq.com>
This commit is contained in:
hurukawaYueqian LinSy03
 authored and GitHub committed 2026-06-11 22:15:05 +00:00
1 parent 03cc888485
commit 152343d5fb
5 files changed
+149 -4
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/serving/speech_api.md
+5 -1
View File
@@ -114,6 +114,7 @@ Content-Type: application/json
| `instructions` | string | "" | Voice style/emotion instructions |
| `max_new_tokens` | integer | 2048 | Maximum tokens to generate |
| `initial_codec_chunk_frames` | integer | null | Per-request initial chunk size override for TTFA tuning. When null, IC is computed dynamically based on server load. |
| `non_streaming_mode` | bool | null | Qwen3-TTS prompt construction mode override. Does not affect HTTP response streaming or async-chunk pipelining. When null, Qwen3-TTS uses model defaults: Base=false, CustomVoice/VoiceDesign=true. |
| `stream` | bool | false | Stream raw PCM chunks as they are decoded (requires `response_format="pcm"`) |
**Supported languages:** Only applicable to Qwen3-TTS. Derived from the model configuration (`talker_config.codec_language_id` in the checkpoint's `config.json`), plus `Auto`, which is always accepted. Official Qwen3-TTS checkpoints support: Auto, Chinese, English, Japanese, Korean, German, French, Russian, Portuguese, Spanish, Italian.
@@ -332,7 +333,8 @@ curl -X POST http://localhost:8091/v1/audio/speech \
"input": "Hello, this is a cloned voice",
"task_type": "Base",
"ref_audio": "https://example.com/reference.wav",
"ref_text": "Original transcript of the reference audio"
"ref_text": "Original transcript of the reference audio",
"non_streaming_mode": true
}' --output cloned.wav
```
@@ -454,6 +456,7 @@ Content-Type: application/json
| `ref_audio` | string | null | Default reference audio (Base task) |
| `ref_text` | string | null | Default reference transcript (Base task) |
| `max_new_tokens` | integer | null | Default max tokens |
| `non_streaming_mode` | bool | null | Default Qwen3-TTS prompt construction mode override. Does not affect HTTP response streaming or async-chunk pipelining. When null, Qwen3-TTS uses model defaults: Base=false, CustomVoice/VoiceDesign=true. |
Each item in the `items` array requires only `input` (the text). All other fields are optional and override the batch-level defaults when set:
@@ -469,6 +472,7 @@ Each item in the `items` array requires only `input` (the text). All other field
| `ref_audio` | string | Override reference audio |
| `ref_text` | string | Override reference transcript |
| `max_new_tokens` | integer | Override max tokens |
| `non_streaming_mode` | bool | Override Qwen3-TTS prompt construction mode. Does not affect HTTP response streaming or async-chunk pipelining. When null, inherits the batch-level value (then the model default). |
### Response Format