vllm-omni/recipes/Qwen/Qwen3-TTS.md

Qwen3-TTS

Text-to-speech serving (CustomVoice / VoiceDesign / Base)

Summary

• Vendor: Qwen
• Model: Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice (and VoiceDesign / Base variants)
• Task: Text-to-speech with predefined voices, voice design, or voice cloning
• Mode: Online serving with the OpenAI-compatible /v1/audio/speech API
• Maintainer: Community

When to use this recipe

Use this recipe when you want a known-good starting point for serving Qwen3-TTS models with vLLM-Omni and validate the deployment with the existing TTS client examples in this repository.

Qwen3-TTS supports three task types, each backed by a dedicated model checkpoint:

Task Type	Model	Description
CustomVoice	Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice	Predefined speaker voices with optional style/emotion control
VoiceDesign	Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign	Generate speech from a natural language voice description
Base	Qwen/Qwen3-TTS-12Hz-1.7B-Base	Voice cloning from reference audio + transcript

Smaller 0.6B variants are also available for CustomVoice and Base.

References

• Upstream or canonical docs: docs/user_guide/examples/online_serving/qwen3_tts.md
• Related examples under examples/: examples/online_serving/qwen3_tts/, examples/offline_inference/qwen3_tts/
• Related issue or discussion: RFC: add recipes folder

Environment

• OS: Linux
• Python: 3.10+
• vLLM / vLLM-Omni: use versions from your current checkout, >=0.20.0

Command

Start the server from the repository root. Pick the model that matches your task type:

# CustomVoice (predefined speakers with optional style control)
vllm serve Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice \
    --deploy-config vllm_omni/deploy/qwen3_tts.yaml \
    --omni --port 8091

# VoiceDesign (natural language voice description)
vllm serve Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign \
    --deploy-config vllm_omni/deploy/qwen3_tts.yaml \
    --omni --port 8091

# Base (voice cloning from reference audio)
vllm serve Qwen/Qwen3-TTS-12Hz-1.7B-Base \
    --deploy-config vllm_omni/deploy/qwen3_tts.yaml \
    --omni --port 8091

Alternatively, use the convenience script:

./examples/online_serving/qwen3_tts/run_server.sh                  # Default: CustomVoice
./examples/online_serving/qwen3_tts/run_server.sh VoiceDesign      # VoiceDesign
./examples/online_serving/qwen3_tts/run_server.sh Base             # Base (voice clone)

The bundled deploy config (vllm_omni/deploy/qwen3_tts.yaml) enables async chunking for low first-audio latency. For advanced deployment tuning, pass a custom deploy config:

vllm serve Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice \
    --deploy-config /path/to/your_qwen3_tts_overrides.yaml \
    --omni --port 8091 --trust-remote-code

Verification

Quick smoke test with curl (CustomVoice):

curl -X POST http://localhost:8091/v1/audio/speech \
    -H "Content-Type: application/json" \
    -d '{
        "input": "Hello, how are you?",
        "voice": "vivian",
        "language": "English"
    }' --output output.wav

CustomVoice with emotion instruction:

curl -X POST http://localhost:8091/v1/audio/speech \
    -H "Content-Type: application/json" \
    -d '{
        "input": "I am so excited!",
        "voice": "vivian",
        "instructions": "Speak with great enthusiasm"
    }' --output excited.wav

List available voices:

curl http://localhost:8091/v1/audio/voices

Using the Python client:

cd examples/online_serving/qwen3_tts

# CustomVoice
python openai_speech_client.py \
    --text "Hello, how are you?" \
    --speaker vivian --language English

# VoiceDesign (requires VoiceDesign model)
python openai_speech_client.py \
    --model Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign \
    --task-type VoiceDesign \
    --text "Hello world" \
    --instructions "A warm, friendly female voice"

# Base / voice clone (requires Base model)
python openai_speech_client.py \
    --model Qwen/Qwen3-TTS-12Hz-1.7B-Base \
    --task-type Base \
    --text "Hello, this is a cloned voice" \
    --ref-audio /path/to/reference.wav \
    --ref-text "Transcript of the reference audio"

Streaming audio (low latency):

curl -X POST http://localhost:8091/v1/audio/speech \
    -H "Content-Type: application/json" \
    -d '{
        "input": "Hello, how are you?",
        "voice": "vivian",
        "language": "English",
        "stream": true,
        "response_format": "pcm"
    }' --no-buffer | play -t raw -r 24000 -e signed -b 16 -c 1 -

Offline inference (no server needed):

python examples/offline_inference/qwen3_tts/end2end.py --query-type CustomVoice
python examples/offline_inference/qwen3_tts/end2end.py --query-type CustomVoice --streaming

Notes

• Memory usage: The deploy config allocates gpu_memory_utilization: 0.3 per stage (talker + code2wav share a single GPU). For the 0.6B variants or constrained GPUs, adjust via --gpu-memory-utilization.
• Key flags: --omni is required. --deploy-config points to the bundled two-stage pipeline config.
• Async chunking: Enabled by default in qwen3_tts.yaml for streaming-friendly first-audio latency. Streaming requires stream=true with response_format="pcm".
• Task/model matching: Each task type requires its matching model checkpoint. Using a CustomVoice model for a Base (voice clone) request will fail.
• Known limitations: The server serves one model variant at a time. To switch task types (e.g., CustomVoice to Base), restart the server with the corresponding model.