anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

[BugFix][Doc]Update voxtral_tts end2end.py & README.md (#2026)

Browse Source 
Signed-off-by: Chen-Yo Sun <chenyo.sun@mistral.ai>
This commit is contained in:
Chen-Yo Sun authored and GitHub committed 2026-03-20 05:05:39 +00:00
1 parent c2a5420d97
commit 6813432f90
2 files changed
+30 -23
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

examples/offline_inference/voxtral_tts/README.md
+29 -22
View File
@@ -6,37 +6,44 @@ When `mistral_common` has `SpeechRequest` support, prompt token IDs are built vi
## Usage Examples
```bash
# Basic single-prompt with reference audio
python3 examples/offline_inference/voxtral_tts/end2end.py \
--write-audio \
--model mistralai/tts-model \
--text "That eerie silence after the first storm was just the calm before another round of chaos, wasn't it?" \
--audio-path path/to/reference_audio.wav
# 32 replicate prompts with reference audio
```bash
# Basic single-prompt with cheerful_female voice preset
python3 examples/offline_inference/voxtral_tts/end2end.py \
--num-prompts 32 --write-audio \
--stage-configs-path vllm_omni/model_executor/stage_configs/voxtral_tts.yaml \
--write-audio --voice cheerful_female \
--model mistralai/tts-model \
--text "That eerie silence after the first storm was just the calm before another round of chaos, wasn't it?" \
--audio-path path/to/reference_audio.wav
--text "That eerie silence after the first storm was just the calm before another round of chaos, wasn't it?"
# 32 replicate prompts with cheerful_female voice preset
python3 examples/offline_inference/voxtral_tts/end2end.py \
--stage-configs-path vllm_omni/model_executor/stage_configs/voxtral_tts.yaml \
--num-prompts 32 --write-audio --voice cheerful_female \
--model mistralai/tts-model \
--text "That eerie silence after the first storm was just the calm before another round of chaos, wasn't it?"
# Streaming with neutral_female voice preset
python3 examples/offline_inference/voxtral_tts/end2end.py \
--stage-configs-path vllm_omni/model_executor/stage_configs/voxtral_tts.yaml \
--streaming --write-audio --voice neutral_female \
--model mistralai/tts-model \
--text "That eerie silence after the first storm was just the calm before another round of chaos, wasn't it?"
# 32 prompts, 8 concurrent requests per wave, streaming with neutral_female voice
python3 examples/offline_inference/voxtral_tts/end2end.py \
--stage-configs-path vllm_omni/model_executor/stage_configs/voxtral_tts.yaml \
--num-prompts 32 --concurrency 8 --streaming --write-audio --voice neutral_female \
--model mistralai/tts-model \
--text "That eerie silence after the first storm was just the calm before another round of chaos, wasn't it?"
# Short debug prompt with reference audio
# Note: Reference audio capability is not yet released.
python3 examples/offline_inference/voxtral_tts/end2end.py \
--stage-configs-path vllm_omni/model_executor/stage_configs/voxtral_tts.yaml \
--write-audio \
--model mistralai/tts-model \
--text "This is a test message." \
--audio-path path/to/reference_audio.wav
# Streaming with neutral_female voice preset
python3 examples/offline_inference/voxtral_tts/end2end.py \
--streaming --write-audio --voice neutral_female \
--model mistralai/tts-model
# 32 prompts, 8 concurrent requests per wave, streaming with casual_male voice
python3 examples/offline_inference/voxtral_tts/end2end.py \
--num-prompts 32 --concurrency 8 --streaming --write-audio --voice casual_male \
--model mistralai/tts-model
```
## Arguments
@@ -47,7 +54,7 @@ python3 examples/offline_inference/voxtral_tts/end2end.py \
| `--text TEXT` | Text to synthesize (default: `"This is a test message."`) |
| `--audio-path PATH` | Path to reference audio file for voice cloning |
| `--output-dir DIR` | Directory to write output WAV files (default: `output_audio`) |
| `--stage-configs-path PATH` | Path to stage configs YAML (auto-resolved from model if not set) |
| `--stage-configs-path PATH` | Path to stage configs YAML (currently it must be set for VoxtralTTS) |
| `--num-prompts N` | Number of replicate prompts to run for measuring performance (default: 1) |
| `--streaming` | Use streaming generation via `AsyncOmni` (default: blocking `Omni`) |
| `--concurrency N` | Max concurrent requests per wave (must be used with `--streaming`, must evenly divide `--num-prompts`) |
examples/offline_inference/voxtral_tts/end2end.py
+1 -1
View File
@@ -208,7 +208,7 @@ def run_non_streaming(inputs, sampling_params_list, model_name, args, output_dir
output_audio_dur = 0.0
for batch_idx, o in enumerate(outputs):
audio_tensor = torch.cat(o.request_output[0].outputs[0].multimodal_output["audio"])
audio_tensor = torch.cat(o.multimodal_output["audio"])
audio_array = audio_tensor.tolist()
output_audio_dur += float(len(audio_array)) / 24000
if args.write_audio: