vllm-omni/benchmarks/voxcpm

VoxCPM Benchmark

This directory contains both:

• online serving benchmark through the OpenAI-compatible /v1/audio/speech API
• offline benchmark for Omni / AsyncOmni
• full offline smoke-matrix orchestration

Both benchmark paths report:

• TTFP: time to first PCM packet
• E2E latency
• RTF: real-time factor (e2e / audio_duration)

Offline Benchmark

Single offline benchmark run:

python benchmarks/voxcpm/vllm_omni/bench_tts_offline.py \
    --model /path/to/voxcpm-model \
    --stage-configs-path vllm_omni/model_executor/stage_configs/voxcpm.yaml \
    --text "This is a split-stage VoxCPM synthesis example running on vLLM Omni." \
    --warmup-runs 1 \
    --output-dir benchmarks/voxcpm/results/offline_single

Streaming offline benchmark:

python benchmarks/voxcpm/vllm_omni/bench_tts_offline.py \
    --model /path/to/voxcpm-model \
    --stage-configs-path vllm_omni/model_executor/stage_configs/voxcpm_async_chunk.yaml \
    --text "This is a split-stage VoxCPM streaming example running on vLLM Omni." \
    --warmup-runs 1 \
    --output-dir benchmarks/voxcpm/results/offline_streaming

Full fixed offline matrix, equivalent to the old examples/offline_inference/voxcpm/test.py:

python benchmarks/voxcpm/vllm_omni/run_offline_matrix.py \
    --model /path/to/voxcpm-model \
    --ref-audio /path/to/reference.wav \
    --ref-text "The exact transcript spoken in reference.wav." \
    --output-root benchmarks/voxcpm/results/offline_matrix

The full matrix covers both routes:

• streaming: voxcpm_async_chunk.yaml
• sync: voxcpm.yaml

And these six scenarios under each route:

• warmup + single TTS
• warmup + single voice cloning
• warmup + batch TTS
• warmup + batch voice cloning
• cold single TTS
• cold single voice cloning

bench_tts_offline.py itself no longer writes summary.json / results.json; it prints TTFP / RTF inline and saves generated WAV files only. The matrix runner keeps only per-case run.log.

Start the Server

Async-chunk:

vllm serve /path/to/voxcpm-model \
    --stage-configs-path vllm_omni/model_executor/stage_configs/voxcpm_async_chunk.yaml \
    --trust-remote-code \
    --enforce-eager \
    --omni \
    --port 8091

Non-streaming:

vllm serve /path/to/voxcpm-model \
    --stage-configs-path vllm_omni/model_executor/stage_configs/voxcpm.yaml \
    --trust-remote-code \
    --enforce-eager \
    --omni \
    --port 8091

Run the Benchmark

python benchmarks/voxcpm/vllm_omni/bench_tts_serve.py \
    --host 127.0.0.1 \
    --port 8091 \
    --num-prompts 20 \
    --max-concurrency 1 \
    --result-dir /tmp/voxcpm_bench

Voice cloning benchmark:

python benchmarks/voxcpm/vllm_omni/bench_tts_serve.py \
    --host 127.0.0.1 \
    --port 8091 \
    --num-prompts 10 \
    --max-concurrency 1 \
    --ref-audio https://example.com/reference.wav \
    --ref-text "The exact transcript spoken in the reference audio." \
    --result-dir /tmp/voxcpm_clone_bench

Notes

• The benchmark uses stream=true and response_format=pcm so TTFP is measured from the first audio packet.
• RTF < 1.0 means the server generates audio faster than real time.
• For voxcpm_async_chunk.yaml, keep concurrency at 1. This matches native VoxCPM streaming more closely.
• Do not benchmark concurrent online streaming on voxcpm_async_chunk.yaml; use voxcpm.yaml for multi-request throughput runs.
• For the offline matrix mode, --ref-audio and --ref-text are required because clone cases are part of the fixed coverage set.