vllm-omni/recipes/OpenBMB/VoxCPM2.md

VoxCPM2

Online + offline TTS with native AR pipeline (48 kHz, 30+ languages)

Summary

• Vendor: OpenBMB
• Model: openbmb/VoxCPM2
• Task: Text-to-speech (zero-shot synthesis and reference-audio voice cloning)
• Mode: Online serving via the OpenAI-compatible /v1/audio/speech API, plus an offline end-to-end example
• Maintainer: Community

When to use this recipe

Use this recipe as a known-good starting point for serving openbmb/VoxCPM2 on vLLM-Omni on consumer-class GPUs. VoxCPM2 is a 2B-parameter native AR TTS model that emits 48 kHz mono audio and supports 30+ languages out of the box. It runs as a single-stage pipeline (MiniCPM4 base LM → FSQ → MiniCPM4 residual LM → LocDiT CFM solver → AudioVAE) and supports both zero-shot synthesis and reference-audio voice cloning, with optional gapless streaming via the bundled Gradio AudioWorklet demo.

Because the model is only 2B params, a single 24 GB consumer GPU is sufficient — no tensor parallelism is required.

References

• Example guide (online): examples/online_serving/text_to_speech/README.md
• Online client / Gradio demo: examples/online_serving/text_to_speech/voxcpm2/
• Offline end-to-end script: examples/offline_inference/text_to_speech/voxcpm2/end2end.py
• Default deploy config: vllm_omni/deploy/voxcpm2.yaml
• Talker / pipeline source: vllm_omni/model_executor/models/voxcpm2/
• Speech API reference: docs/serving/speech_api.md
• Related issue or discussion: RFC: add recipes folder

Hardware Support

This recipe documents tested single-card configurations for a 24 GB NVIDIA GPU and an Ascend Atlas A2.

Ascend NPU

1 x Ascend 910B3 64GB (Atlas A2)

VoxCPM2 runs on one Atlas A2 card with the standard deploy config. The model's LocDiT path uses eager PyTorch on NPU; keep enforce_eager: true. The default config enables fused QKV, fused gate/up projection, and the zero-delta-time cache for this path.

Environment

• Hardware: 1 x Ascend 910B3 64GB (Atlas A2)
• Container: quay.nju.edu.cn/ascend/vllm-omni:v0.28.0
• vLLM: 0.28.0
• vLLM-Omni: current main
• Model: openbmb/VoxCPM2, bfloat16

Command

Expose one physical NPU to the container, then start the server normally. The visible card is addressed as logical device 0 by voxcpm2.yaml.

export ASCEND_RT_VISIBLE_DEVICES=5
export VLLM_WORKER_MULTIPROC_METHOD=spawn

vllm serve /path/to/VoxCPM2 --omni \
    --host 0.0.0.0 --port 8000

Performance validation

The component timer was measured after model warmup with a single Chinese TTS request. Each decode step executes nine LocDiT estimator calls, so per-step measurements compare the same amount of model work even when generated audio duration differs.

path	CFM estimator / call	decode / step	forward / step
Before (optimization flags skipped on NPU)	18.84 ms	193.86 ms	265.35 ms
Fused QKV + fused MLP + zero-dt cache	17.50 ms	182.22 ms	254.07 ms
Above + experimental reduced-precision RoPE	14.93 ms	157.36 ms	227.33 ms

The default fused path reduces CFM estimator latency by about 7.1% and complete decode-step latency by 6.0%. Its generated WAV was byte-identical to the pre-optimization baseline. Setting enable_loc_dit_fast_rope: true can increase the reductions to about 20.8% and 18.8%, respectively, but changes the recurrent acoustic state and is therefore an opt-in quality/performance tradeoff. Its test audio was valid 48 kHz mono PCM with finite samples and no clipping; evaluate voice quality on a representative corpus before enabling it in production.

Do not set enforce_eager: false for VoxCPM2 on Ascend yet. ACL Graph captures the language-model runner, but the request-owned VoxCPM2 state has dynamic prefill/decode shapes; replay can therefore feed an empty captured row into a non-empty residual state and fail at aclnnCat.

GPU

1 x RTX 4090 24GB (Single GPU, Minimum Recommended)

VoxCPM2 (2B params in bfloat16) fits comfortably on a single 24 GB GPU. The bundled default config at vllm_omni/deploy/voxcpm2.yaml (gpu_memory_utilization: 0.9, max_num_seqs: 4, enforce_eager: true) loads cleanly on a 4090 with ~4.9 GiB for model weights, ~15.2 GiB for KV cache, and ~2 GiB for the talker's CUDA-Graph / CFM / VAE buffers, for a total resident footprint of about 22 GiB / 24 GiB in steady state.

Environment

• OS: Linux (Ubuntu 22.04)
• Python: 3.12
• Driver / runtime: NVIDIA driver 575.57.08, CUDA 12.9 runtime
• torch: 2.11.0+cu128
• vLLM: 0.21.0
• vLLM-Omni: 0.20.1.dev (current main)
• Native VoxCPM2 source: voxcpm >= 2.0

Command

Start the server from the repository root. On a clean 4090 with no other tenant on the GPU, the defaults from voxcpm2.yaml work as-is:

vllm serve openbmb/VoxCPM2 --omni \
    --host 0.0.0.0 --port 8000

The deploy config at vllm_omni/deploy/voxcpm2.yaml is loaded automatically by the model registry (HF model_type=voxcpm2). Pass --deploy-config <path> to override.

Verification

Server cold-start: ~60 s from vllm serve to Application startup complete (subprocess fork, vLLM 0.21 init, model load, flashinfer JIT, torch.compile of LocDiT / feat_encoder / AudioVAE, and CUDA-Graph warmup). The first request after startup pays a small additional cost; steady-state requests are much faster.

T1 — zero-shot synthesis (curl, WAV):

curl -X POST http://127.0.0.1:8000/v1/audio/speech \
    -H "Content-Type: application/json" \
    -d '{
        "model": "voxcpm2",
        "input": "Hello, this is VoxCPM2 served by vLLM-Omni on a single RTX 4090.",
        "voice": "default",
        "response_format": "wav"
    }' --output output.wav

Observed: output.wav 491 KB, 5.12 s @ 48 kHz mono, peak amplitude 0.996. Server-to-client wall time: 2.6 s (server-side RTF ~0.5 after warmup).

T2 — bundled Python client (zero-shot):

python examples/online_serving/text_to_speech/voxcpm2/openai_speech_client.py \
    --text "This audio was generated by the bundled openai_speech_client.py script against a vLLM-Omni server." \
    --api-base http://127.0.0.1:8000 \
    --output output.wav

Observed: output.wav 968 KB, 10.08 s @ 48 kHz mono, peak amplitude 0.949.

T3 — voice cloning from a reference WAV:

python examples/online_serving/text_to_speech/voxcpm2/openai_speech_client.py \
    --text "This should sound like the reference speaker, but saying a different sentence on a single RTX 4090." \
    --ref-audio /path/to/reference.wav \
    --api-base http://127.0.0.1:8000 \
    --output cloned.wav

Observed (using the T2 output output.wav as --ref-audio): cloned.wav 568 KB, 5.92 s @ 48 kHz mono, peak amplitude 0.990; end-to-end wall time 3.16 s.

--ref-audio accepts local file paths (auto-base64), HTTP/HTTPS URLs, or data:audio/wav;base64,... data URIs. The voice field is required by the OpenAI schema but ignored by VoxCPM2 unless it refers to an uploaded voice — voice cloning is driven entirely by ref_audio.

T4 — streaming PCM (raw, 48 kHz s16le):

curl -X POST http://127.0.0.1:8000/v1/audio/speech \
    -H "Content-Type: application/json" \
    -d '{
        "model": "voxcpm2",
        "input": "Streaming PCM test on a single RTX 4090 with VoxCPM2.",
        "voice": "default",
        "stream": true,
        "stream_format": "audio",
        "response_format": "pcm"
    }' --no-buffer | play -t raw -r 48000 -e signed -b 16 -c 1 -

Observed: 230400 int16 samples = 4.80 s of PCM at 48 kHz, peak 32525 / 32767 (~99 % full scale). The player sample rate is 48 kHz, not 24 kHz.

T5 — gapless streaming (Gradio AudioWorklet demo):

python examples/online_serving/text_to_speech/voxcpm2/gradio_demo.py \
    --api-base http://127.0.0.1:8000

Observed: the demo binds to http://0.0.0.0:7860 in ~4 s after launch. The UI ("VoxCPM2 Streaming Demo") exposes three modes — Voice Design (control instruction only), Controllable Cloning (ref audio + optional style control), and Ultimate Cloning (ref audio + transcript for audio continuation) — plus a Stream (gapless) checkbox (default on) that drives the AudioWorklet streaming path. The right pane includes runnable example prompts and a Ready status indicator. End-to-end browser playback through Generate Speech was verified working on the same vLLM-Omni server backing T1–T4 above.

T6 — offline inference (no server, single process per request):

python examples/offline_inference/text_to_speech/voxcpm2/end2end.py \
    --text "Hello, this is VoxCPM2 native AR synthesis running on vLLM-Omni and a single RTX 4090." \
    --output-dir output_audio

end2end.py is a one-shot script (init → one generate() → exit), so its reported RTF includes per-process compile + CUDA-Graph capture + first-request runtime warmup. Note that the default voxcpm2.yaml sets enforce_eager: true, so vLLM's engine-level CUDA graphs are off by design; the "CUDA-Graph capture" referenced throughout this recipe is the talker's own _CapturedGraph over the CFM solver / AudioVAE, not the LM decode loop. Flipping enforce_eager=false does not remove this cost. To separate the warmup cost from the talker's true per-step throughput, the table below reports 5 consecutive engine.generate([prompt]) calls in the same process after a one-off ~28 s engine init:

call	duration	inference	RTF	notes
#1	6.72 s	11.97 s	1.782	cold: torch.compile + CUDA-Graph capture
#2	6.24 s	11.43 s	1.831	still runtime warmup
#3	6.88 s	0.82 s	0.120	steady-state
#4	6.56 s	0.78 s	0.119	steady-state
#5	5.76 s	0.70 s	0.121	steady-state

Steady-state RTF (mean of calls #3–#5): 0.120, i.e. ~8× real-time on a single 4090. The high call-#1 / call-#2 numbers are one-time warmup costs that amortise to zero after the second request — they are not the model's per-step throughput.

A literal one-shot python end2end.py invocation in a fresh process only sees call #1 (~RTF 1.8). For repeated workloads, either reuse the same engine across calls (as in the table above) or use the online serving path (T1–T5); a long-lived server amortises the warmup across all subsequent requests (T1 wall-time RTF was 0.51 with a warm server).

Voice cloning + continuation mode (ref_audio + transcript):

python examples/offline_inference/text_to_speech/voxcpm2/end2end.py \
    --text "Hello, this is the offline voice cloning test on a single RTX 4090." \
    --ref-audio /path/to/reference.wav \
    --ref-text "Original transcript of the reference audio."

Same 5-call methodology as the zero-shot table, this time with a 10.08 s reference WAV supplied via --ref-audio (and matching --ref-text):

call	duration	inference	RTF	notes
#1	5.44 s	12.38 s	2.276	cold: compile + CUDA-Graph capture + ref encode
#2	5.12 s	2.47 s	0.482	most warmup done
#3	5.44 s	0.76 s	0.139	steady-state
#4	4.96 s	0.68 s	0.137	steady-state
#5	5.28 s	0.71 s	0.134	steady-state

Steady-state cloning RTF (mean of calls #3–#5): 0.137, i.e. only +0.017 RTF above the zero-shot steady-state of 0.120. That delta is the per-request AudioVAE.encode over the reference clip (~85 ms for a 10 s ref WAV) plus the ref-continuation prompt build; the AR decode steps themselves run at the same per-step rate as zero-shot.

Notes

• Prerequisites: pip install voxcpm soundfile httpx ninja. ninja must be on PATH because flashinfer JIT-compiles a top-k sampling kernel during the first profile pass (without it, startup dies with FileNotFoundError: 'ninja').
• Memory usage: ~4.9 GiB model weights + ~15.2 GiB KV cache ≈ 22 GiB / 24 GiB resident with the default voxcpm2.yaml. On a shared GPU, pass --gpu-memory-utilization 0.75 (or lower) if startup fails the free-memory check.
• Key flags: --omni is required. --trust-remote-code is not needed — the HF config registers under model_type=voxcpm2.
• Output: 48 kHz mono. When streaming PCM to a player, use -r 48000.
• Cold start: ~60 s server boot + ~25 s first-inference overhead (torch.compile + flashinfer JIT + the talker's own _CapturedGraph capture over CFM / AudioVAE — not vLLM engine CUDA graphs, which enforce_eager: true disables). Steady-state inference RTF is ~0.12 (~8× real-time on a single 4090; see T6 table for the per-call breakdown); online server requests run at ~RTF 0.5 wall-time (includes HTTP round-trip). Keep the server warm for interactive use.