From ac4106900b761d3b3526cd82a0bc7b783b7c536f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E7=8E=8B=E9=87=91=E6=97=AD?= <105263726+wjinxu@users.noreply.github.com> Date: Mon, 1 Jun 2026 14:39:54 +0800 Subject: [PATCH] [skip ci][Recipe] OpenBMB/VoxCPM2 (#3850) Signed-off-by: wjinxu <1299461899@qq.com> Co-authored-by: Cursor --- recipes/OpenBMB/VoxCPM2.md | 249 +++++++++++++++++++++++++++++++++++++ recipes/README.md | 1 + 2 files changed, 250 insertions(+) create mode 100644 recipes/OpenBMB/VoxCPM2.md diff --git a/recipes/OpenBMB/VoxCPM2.md b/recipes/OpenBMB/VoxCPM2.md new file mode 100644 index 000000000..9414d8a48 --- /dev/null +++ b/recipes/OpenBMB/VoxCPM2.md @@ -0,0 +1,249 @@ +# VoxCPM2 + +> Online + offline TTS with native AR pipeline (48 kHz, 30+ languages) + +## Summary + +- Vendor: OpenBMB +- Model: `openbmb/VoxCPM2` +- Task: Text-to-speech (zero-shot synthesis and reference-audio voice cloning) +- Mode: Online serving via the OpenAI-compatible `/v1/audio/speech` API, plus + an offline end-to-end example +- Maintainer: Community + +## When to use this recipe + +Use this recipe as a known-good starting point for serving `openbmb/VoxCPM2` +on vLLM-Omni on consumer-class GPUs. VoxCPM2 is a 2B-parameter native AR +TTS model that emits 48 kHz mono audio and supports 30+ languages out of +the box. It runs as a single-stage pipeline (`MiniCPM4 base LM → FSQ → +MiniCPM4 residual LM → LocDiT CFM solver → AudioVAE`) and supports both +zero-shot synthesis and reference-audio voice cloning, with optional +gapless streaming via the bundled Gradio AudioWorklet demo. + +Because the model is only 2B params, a single 24 GB consumer GPU is +sufficient — no tensor parallelism is required. + +## References + +- Example guide (online): + [`examples/online_serving/text_to_speech/README.md`](../../examples/online_serving/text_to_speech/README.md#voxcpm2) +- Online client / Gradio demo: + [`examples/online_serving/text_to_speech/voxcpm2/`](../../examples/online_serving/text_to_speech/voxcpm2/) +- Offline end-to-end script: + [`examples/offline_inference/text_to_speech/voxcpm2/end2end.py`](../../examples/offline_inference/text_to_speech/voxcpm2/end2end.py) +- Default deploy config: + [`vllm_omni/deploy/voxcpm2.yaml`](../../vllm_omni/deploy/voxcpm2.yaml) +- Talker / pipeline source: + [`vllm_omni/model_executor/models/voxcpm2/`](../../vllm_omni/model_executor/models/voxcpm2/) +- Speech API reference: + [`docs/serving/speech_api.md`](../../docs/serving/speech_api.md#voxcpm2) +- Related issue or discussion: + [RFC: add recipes folder](https://github.com/vllm-project/vllm-omni/issues/2645) + +## Hardware Support + +This recipe documents one tested 24 GB consumer-GPU configuration. +Larger-VRAM (H20 / H100 / A100) and other vendor sections (ROCm, NPU) are +welcome as community validation lands. + +## GPU + +### 1 x RTX 4090 24GB (Single GPU, Minimum Recommended) + +VoxCPM2 (2B params in bfloat16) fits comfortably on a single 24 GB GPU. +The bundled default config at +[`vllm_omni/deploy/voxcpm2.yaml`](../../vllm_omni/deploy/voxcpm2.yaml) +(`gpu_memory_utilization: 0.9`, `max_num_seqs: 4`, `enforce_eager: true`) +loads cleanly on a 4090 with ~4.9 GiB for model weights, ~15.2 GiB for +KV cache, and ~2 GiB for the talker's CUDA-Graph / CFM / VAE buffers, +for a total resident footprint of about **22 GiB / 24 GiB** in steady +state. + +#### Environment + +- OS: Linux (Ubuntu 22.04) +- Python: 3.12 +- Driver / runtime: NVIDIA driver 575.57.08, CUDA 12.9 runtime +- torch: 2.11.0+cu128 +- vLLM: 0.21.0 +- vLLM-Omni: 0.20.1.dev (current `main`) +- Native VoxCPM2 source: `voxcpm` >= 2.0 + +#### Command + +Start the server from the repository root. On a clean 4090 with no other +tenant on the GPU, the defaults from `voxcpm2.yaml` work as-is: + +```bash +vllm serve openbmb/VoxCPM2 --omni \ + --host 0.0.0.0 --port 8000 +``` + +The deploy config at +[`vllm_omni/deploy/voxcpm2.yaml`](../../vllm_omni/deploy/voxcpm2.yaml) +is loaded automatically by the model registry (HF `model_type=voxcpm2`). +Pass `--deploy-config ` to override. + +#### Verification + +**Server cold-start**: ~60 s from `vllm serve` to `Application startup +complete` (subprocess fork + vLLM 0.21 init + model load + flashinfer JIT ++ torch.compile of LocDiT / feat_encoder / AudioVAE + CUDA-Graph warmup). +The first request after startup pays a small additional cost; steady-state +requests are much faster. + +**T1 — zero-shot synthesis (curl, WAV)**: + +```bash +curl -X POST http://127.0.0.1:8000/v1/audio/speech \ + -H "Content-Type: application/json" \ + -d '{ + "model": "voxcpm2", + "input": "Hello, this is VoxCPM2 served by vLLM-Omni on a single RTX 4090.", + "voice": "default", + "response_format": "wav" + }' --output output.wav +``` + +Observed: `output.wav` 491 KB, 5.12 s @ 48 kHz mono, peak amplitude 0.996. +Server-to-client wall time: 2.6 s (server-side RTF ~0.5 after warmup). + +**T2 — bundled Python client (zero-shot)**: + +```bash +python examples/online_serving/text_to_speech/voxcpm2/openai_speech_client.py \ + --text "This audio was generated by the bundled openai_speech_client.py script against a vLLM-Omni server." \ + --api-base http://127.0.0.1:8000 \ + --output output.wav +``` + +Observed: `output.wav` 968 KB, 10.08 s @ 48 kHz mono, peak amplitude 0.949. + +**T3 — voice cloning from a reference WAV**: + +```bash +python examples/online_serving/text_to_speech/voxcpm2/openai_speech_client.py \ + --text "This should sound like the reference speaker, but saying a different sentence on a single RTX 4090." \ + --ref-audio /path/to/reference.wav \ + --api-base http://127.0.0.1:8000 \ + --output cloned.wav +``` + +Observed (using the T2 output `output.wav` as `--ref-audio`): +`cloned.wav` 568 KB, 5.92 s @ 48 kHz mono, peak amplitude 0.990; +end-to-end wall time 3.16 s. + +`--ref-audio` accepts local file paths (auto-base64), HTTP/HTTPS URLs, or +`data:audio/wav;base64,...` data URIs. The `voice` field is required by +the OpenAI schema but ignored by VoxCPM2 unless it refers to an +uploaded voice — voice cloning is driven entirely by `ref_audio`. + +**T4 — streaming PCM (raw, 48 kHz s16le)**: + +```bash +curl -X POST http://127.0.0.1:8000/v1/audio/speech \ + -H "Content-Type: application/json" \ + -d '{ + "model": "voxcpm2", + "input": "Streaming PCM test on a single RTX 4090 with VoxCPM2.", + "voice": "default", + "stream": true, + "response_format": "pcm" + }' --no-buffer | play -t raw -r 48000 -e signed -b 16 -c 1 - +``` + +Observed: 230400 int16 samples = 4.80 s of PCM at 48 kHz, peak 32525 / +32767 (~99 % full scale). The player sample rate is **48 kHz**, not +24 kHz. + +**T5 — gapless streaming (Gradio AudioWorklet demo)**: + +```bash +python examples/online_serving/text_to_speech/voxcpm2/gradio_demo.py \ + --api-base http://127.0.0.1:8000 +``` + +Observed: the demo binds to `http://0.0.0.0:7860` in ~4 s after launch. +The UI ("VoxCPM2 Streaming Demo") exposes three modes — **Voice Design** +(control instruction only), **Controllable Cloning** (ref audio + +optional style control), and **Ultimate Cloning** (ref audio + +transcript for audio continuation) — plus a `Stream (gapless)` +checkbox (default on) that drives the AudioWorklet streaming path. The +right pane includes runnable example prompts and a `Ready` status +indicator. End-to-end browser playback through `Generate Speech` was +verified working on the same vLLM-Omni server backing T1–T4 above. + +**T6 — offline inference (no server, single process per request)**: + +```bash +python examples/offline_inference/text_to_speech/voxcpm2/end2end.py \ + --text "Hello, this is VoxCPM2 native AR synthesis running on vLLM-Omni and a single RTX 4090." \ + --output-dir output_audio +``` + +`end2end.py` is a one-shot script (init → one `generate()` → exit), so +its reported RTF includes per-process compile + CUDA-Graph capture + +first-request runtime warmup. Note that the default `voxcpm2.yaml` sets +`enforce_eager: true`, so vLLM's engine-level CUDA graphs are off by +design; the "CUDA-Graph capture" referenced throughout this recipe is the +talker's own `_CapturedGraph` over the CFM solver / AudioVAE, not the LM +decode loop. Flipping `enforce_eager=false` does **not** remove this cost. +To separate the warmup cost from the +talker's true per-step throughput, the table below reports **5 +consecutive `engine.generate([prompt])` calls in the same process** +after a one-off ~28 s engine init: + +| call | duration | inference | RTF | notes | +| ---- | -------- | --------- | ----- | ------------------------------------------------ | +| #1 | 6.72 s | 11.97 s | 1.782 | cold: torch.compile + CUDA-Graph capture | +| #2 | 6.24 s | 11.43 s | 1.831 | still runtime warmup | +| #3 | 6.88 s | 0.82 s | 0.120 | ⚡ steady-state | +| #4 | 6.56 s | 0.78 s | 0.119 | steady-state | +| #5 | 5.76 s | 0.70 s | 0.121 | steady-state | + +**Steady-state RTF (mean of calls #3–#5): 0.120**, i.e. ~8× real-time on +a single 4090. The high call-#1 / call-#2 numbers are one-time warmup +costs that **amortise to zero after the second request** — they are not +the model's per-step throughput. + +A literal one-shot `python end2end.py` invocation in a fresh process +only sees call #1 (~RTF 1.8). For repeated workloads, either reuse the +same engine across calls (as in the table above) or use the online +serving path (T1–T5); a long-lived server amortises the warmup across +all subsequent requests (T1 wall-time RTF was 0.51 with a warm server). + +Voice cloning + continuation mode (`ref_audio` + transcript): + +```bash +python examples/offline_inference/text_to_speech/voxcpm2/end2end.py \ + --text "Hello, this is the offline voice cloning test on a single RTX 4090." \ + --ref-audio /path/to/reference.wav \ + --ref-text "Original transcript of the reference audio." +``` + +Same 5-call methodology as the zero-shot table, this time with a +10.08 s reference WAV supplied via `--ref-audio` (and matching +`--ref-text`): + +| call | duration | inference | RTF | notes | +| ---- | -------- | --------- | ----- | ------------------------------------------------ | +| #1 | 5.44 s | 12.38 s | 2.276 | cold: compile + CUDA-Graph capture + ref encode | +| #2 | 5.12 s | 2.47 s | 0.482 | most warmup done | +| #3 | 5.44 s | 0.76 s | 0.139 | ⚡ steady-state | +| #4 | 4.96 s | 0.68 s | 0.137 | steady-state | +| #5 | 5.28 s | 0.71 s | 0.134 | steady-state | + +**Steady-state cloning RTF (mean of calls #3–#5): 0.137**, i.e. only +**+0.017 RTF** above the zero-shot steady-state of 0.120. That delta is +the per-request `AudioVAE.encode` over the reference clip (~85 ms for a +10 s ref WAV) plus the ref-continuation prompt build; the AR decode +steps themselves run at the same per-step rate as zero-shot. + +#### Notes + +- Prerequisites: `pip install voxcpm soundfile httpx ninja`. `ninja` must be on `PATH` because `flashinfer` JIT-compiles a top-k sampling kernel during the first profile pass (without it, startup dies with `FileNotFoundError: 'ninja'`). +- Memory usage: ~4.9 GiB model weights + ~15.2 GiB KV cache ≈ 22 GiB / 24 GiB resident with the default `voxcpm2.yaml`. On a shared GPU, pass `--gpu-memory-utilization 0.75` (or lower) if startup fails the free-memory check. +- Key flags: `--omni` is required. `--trust-remote-code` is not needed — the HF config registers under `model_type=voxcpm2`. +- Output: 48 kHz mono. When streaming PCM to a player, use `-r 48000`. +- Cold start: ~60 s server boot + ~25 s first-inference overhead (torch.compile + flashinfer JIT + the talker's own `_CapturedGraph` capture over CFM / AudioVAE — not vLLM engine CUDA graphs, which `enforce_eager: true` disables). Steady-state inference RTF is **~0.12** (~8× real-time on a single 4090; see T6 table for the per-call breakdown); online server requests run at ~RTF 0.5 wall-time (includes HTTP round-trip). Keep the server warm for interactive use. diff --git a/recipes/README.md b/recipes/README.md index ccf0efd53..005bd37ec 100644 --- a/recipes/README.md +++ b/recipes/README.md @@ -35,6 +35,7 @@ recipes/ | [`LTX/LTX-2.md`](./LTX/LTX-2.md) | Text-to-video and image-to-video serving | 1x H200 141GB | | [`LTX/LTX-2.3.md`](./LTX/LTX-2.3.md) | Text-to-video with audio generation (22B) | 1x GPU (96GB VRAM) | | [`mistralai/Voxtral-TTS.md`](./mistralai/Voxtral-TTS.md) | Online serving for TTS | 1x RTX 4090 24GB | +| [`OpenBMB/VoxCPM2.md`](./OpenBMB/VoxCPM2.md) | Online + offline TTS with native AR pipeline (48 kHz, 30+ languages) | 1x RTX 4090 24GB | | [`Qwen/Qwen-Image.md`](./Qwen/Qwen-Image.md) | Text-to-image serving with step-wise continuous batching replay and ModelOpt mixed FP8/NVFP4 | 1x A100 80GB / 2x B200 | | [`Qwen/Qwen-Image.md`](./Qwen/Qwen-Image.md) | Text-to-image serving with step-wise continuous batching replay | 1x A100 80GB | | [`Qwen/Qwen-Image-Edit.md`](./Qwen/Qwen-Image-Edit.md) | Text-guided single-image editing | 1x or 2x H200 141GB |