[Doc] Add Voxtral TTS RTX 5090 recipe (#4774)

Signed-off-by: Ayaka Mikazuki <ayaka@mail.shn.hk>
Signed-off-by: Ayaka <ayaka@mail.shn.hk>
Co-authored-by: Sy03 <1370724210@qq.com>
This commit is contained in:
Ayaka MikazukiandSy03 authored and GitHub committed 2026-07-01 04:56:23 +02:00
1 parent a560ed184d
commit c21e9a35f8
1 file changed
+73
+73
View File
@@ -27,6 +27,79 @@ examples in this repository.
## GPU
### 1x NVIDIA GeForce RTX 5090 32GB
#### Environment
- OS: Ubuntu 24.04.4 LTS
- Python: 3.12.3
- PyTorch: 2.11.0+cu130
- Driver / runtime: NVIDIA driver 580.159.03, CUDA 13.0
- GPU: NVIDIA GeForce RTX 5090, 32 GB (`32607 MiB`)
- vLLM version: 0.23.0
- vLLM-Omni version or commit: 0.23.0rc2.dev140+gbe60d7c7c (`be60d7c7`)
- Tested with: `mistral-common==1.11.5`
#### Command
```bash
vllm serve mistralai/Voxtral-4B-TTS-2603 \
--omni \
--port 8091 \
--trust-remote-code
```
The deploy config is auto-loaded from `vllm_omni/deploy/voxtral_tts.yaml`.
#### Verification
Health check and voices:
```bash
curl -s -o /tmp/voxtral_health.txt -w '%{http_code}' http://127.0.0.1:8091/health
curl -s http://127.0.0.1:8091/v1/audio/voices
```
Non-streaming WAV:
```bash
curl -X POST http://127.0.0.1:8091/v1/audio/speech \
-H "Content-Type: application/json" \
-d '{
"model": "mistralai/Voxtral-4B-TTS-2603",
"input": "Hello, this is Voxtral TTS running on an NVIDIA GeForce RTX 5090 with vLLM-Omni.",
"voice": "casual_female",
"language": "English",
"response_format": "wav"
}' \
--output /tmp/voxtral_5090.wav
```
Streaming PCM:
```bash
curl -X POST http://127.0.0.1:8091/v1/audio/speech \
-H "Content-Type: application/json" \
-d '{
"model": "mistralai/Voxtral-4B-TTS-2603",
"input": "Hello, this is Voxtral TTS streaming PCM on an NVIDIA GeForce RTX 5090.",
"voice": "casual_female",
"language": "English",
"stream": true,
"response_format": "pcm"
}' \
--output /tmp/voxtral_5090_stream.pcm
ffmpeg -f s16le -ar 24000 -ac 1 -i /tmp/voxtral_5090_stream.pcm /tmp/voxtral_5090_stream.wav -y
ffprobe -hide_banner /tmp/voxtral_5090_stream.wav
```
#### Notes
- Output: 24 kHz mono WAV/PCM; both non-streaming and streaming requests passed.
- Memory usage: ~27.3 GiB after warmup (Stage 0 ~25.5 GiB, Stage 1 ~1.7 GiB).
- Key flags: `--omni` is required; `--trust-remote-code` is recommended.
### 1x RTX 4090 24GB
#### Environment