[Model] Add Voxtral TTS model (#1803)

Signed-off-by: Chen-Yo Sun <chenyo.sun@mistral.ai>
This commit is contained in:
Chen-Yo Sun authored and GitHub committed 2026-03-19 16:02:11 -04:00
1 parent b766c4702c
commit e9fe78e28c
29 files changed
+5135 -51

No files matched your search

+15
View File
@@ -4,6 +4,7 @@ vLLM-Omni provides an OpenAI-compatible API for text-to-speech (TTS) generation.
- **Qwen3-TTS** (`Qwen/Qwen3-TTS-12Hz-*`) -- Qwen3-based TTS with CustomVoice, VoiceDesign, and Base (voice cloning) task types. Output: 24 kHz.
- **Fish Speech S2 Pro** (`fishaudio/s2-pro`) -- Dual-AR TTS with DAC codec. Supports text-to-speech and voice cloning via reference audio. Output: 44.1 kHz.
- **Voxtral TTS** (`mistralai/tts-model`) -- AR + FlowMatching TTS with preset voices. Output: 24 kHz.
Each server instance runs a single model (specified at startup via `vllm serve <model> --omni`).
@@ -28,6 +29,14 @@ vllm-omni serve fishaudio/s2-pro \
--trust-remote-code \
--enforce-eager \
--gpu-memory-utilization 0.9
# Voxtral TTS
vllm serve mistralai/tts-model \
--stage-configs-path vllm_omni/model_executor/stage_configs/voxtral_tts.yaml \
--omni \
--port 8091 \
--trust-remote-code \
--enforce-eager
```
### Generate Speech
@@ -347,6 +356,12 @@ curl -X POST http://localhost:8091/v1/audio/speech \
Fish Speech uses `ref_audio` and `ref_text` for voice cloning (no `task_type` needed). The `voice` field should be set to `"default"`. See the [Fish Speech online serving example](../user_guide/examples/online_serving/fish_speech.md) for details.
### Voxtral TTS
| Model | Description |
|-------|-------------|
| `mistralai/tts-model` | 3B AR + FlowMatching TTS. Supports text-to-speech with preset voices. |
## Error Responses
### 400 Bad Request