anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

[Doc][TTS] CosyVoice3 online docs + residual TTS yaml cleanup + remove VoxCPM v1 (#3748)

Browse Source 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
This commit is contained in:
Yueqian Lin authored and GitHub committed 2026-05-20 15:24:06 +08:00
1 parent 1c530d984d
commit fc8486cf20
35 files changed
+248 -4147
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

examples/offline_inference/text_to_speech/README.md
+2 -53
View File
@@ -14,13 +14,12 @@ list of supported architectures across all modalities, see
| Model | HuggingFace repo | Stages | Voice cloning | Streaming | Special modes | Sample rate |
|---|---|---|---|---|---|---|
| CosyVoice3 | `FunAudioLLM/Fun-CosyVoice3-0.5B-2512` | 2 (talker + code2wav) | ✓ | ✓ | — | 22.05 kHz |
| CosyVoice3 | `FunAudioLLM/Fun-CosyVoice3-0.5B-2512` | 2 (talker + code2wav) | ✓ | ✓ | — | 24 kHz |
| Fish Speech S2 Pro | `fishaudio/s2-pro` | dual-AR | ✓ | ✓ | — | 44.1 kHz |
| Ming-flash-omni-TTS | `Jonathan1909/Ming-flash-omni-2.0` | single (talker only) | — (caption-controlled) | — | style / IP / basic captions | 44.1 kHz |
| MOSS-TTS-Nano | `OpenMOSS-Team/MOSS-TTS-Nano` | single (AR + codec) | ✓ (required) | ✓ | voice_clone, continuation | 48 kHz |
| OmniVoice | `k2-fsa/OmniVoice` | 2 (gen + dec) | ✓ | — | voice design, language hint | 24 kHz |
| Qwen3-TTS | `Qwen/Qwen3-TTS-12Hz-1.7B-{CustomVoice,VoiceDesign,Base}` | 2 (talker + code2wav) | ✓ (Base) | ✓ | 3 task variants | 24 kHz |
| VoxCPM | `openbmb/VoxCPM-0.5B` | split | ✓ | ✓ | — | 24 kHz |
| VoxCPM2 | `openbmb/VoxCPM2` | single (native AR) | ✓ | ✓ (online) | continuation | 48 kHz |
| Voxtral TTS | `mistralai/Voxtral-4B-TTS-2603` | varies | ✓ | ✓ | voice presets | 24 kHz |
@@ -41,7 +40,7 @@ python examples/offline_inference/text_to_speech/<model>/end2end.py \
## CosyVoice3
2-stage TTS pipeline (`talker` + `code2wav`) at 22.05 kHz.
2-stage TTS pipeline (`talker` + `code2wav`) at 24 kHz.
### Prerequisites
```bash
@@ -308,56 +307,6 @@ python examples/offline_inference/text_to_speech/qwen3_tts/end2end.py \
---
## VoxCPM
Split-stage TTS. The hub example covers single TTS, single voice cloning, and streaming. Use `benchmarks/voxcpm/` for warmup, batch JSONL prompts, profiler injection, and offline TTFP / RTF measurement.
### Prerequisites
```bash
pip install voxcpm soundfile
# or use a local source tree:
export VLLM_OMNI_VOXCPM_CODE_PATH=/path/to/VoxCPM/src
```
### Quick start
```bash
python examples/offline_inference/text_to_speech/voxcpm/end2end.py \
--model openbmb/VoxCPM-0.5B \
--text "This is a split-stage VoxCPM synthesis example running on vLLM Omni."
```
### Voice cloning
```bash
python examples/offline_inference/text_to_speech/voxcpm/end2end.py \
--model openbmb/VoxCPM-0.5B \
--text "This sentence is synthesized with a cloned voice." \
--ref-audio /path/to/reference.wav \
--ref-text "The exact transcript spoken in reference.wav."
```
### Streaming
Pass `--streaming` together with the legacy async-chunk stage config:
```bash
python examples/offline_inference/text_to_speech/voxcpm/end2end.py \
--model openbmb/VoxCPM-0.5B \
--streaming \
--stage-configs-path vllm_omni/model_executor/stage_configs/voxcpm_async_chunk.yaml \
--text "This is a split-stage VoxCPM streaming example running on vLLM Omni."
```
### Persistent HF config (optional)
The engine auto-patches a missing `model_type` in the checkpoint's `config.json` in-memory at startup (see `vllm_omni/engine/arg_utils.py`), so no manual setup is required.
If you'd rather maintain a persistent patched config dir (e.g. to share across runs or to add fields the auto-patcher doesn't set), export `VLLM_OMNI_VOXCPM_HF_CONFIG_PATH` to its location — both the bundled deploy and async-chunk yamls read it via `${oc.env:VLLM_OMNI_VOXCPM_HF_CONFIG_PATH,}`.
### Notes
- Non-streaming auto-loads `vllm_omni/deploy/voxcpm.yaml` from the model directory name / HF `model_type`. Streaming uses the legacy `voxcpm_async_chunk.yaml` passed via `--stage-configs-path`.
- Streaming is currently single-request oriented.
- `--ref-text` must be the real transcript of `--ref-audio`; mismatched text degrades quality.
- For online serving, see the [VoxCPM section in the online hub](../../online_serving/text_to_speech/README.md#voxcpm). For benchmark reporting, see [`benchmarks/voxcpm`](../../../benchmarks/voxcpm/README.md).
---
## VoxCPM2
Single-stage native AR TTS at 48 kHz. Pipeline: `feat_encoder → MiniCPM4 → FSQ → residual_lm → LocDiT → AudioVAE`.
examples/offline_inference/text_to_speech/voxcpm/end2end.py
-220
View File
@@ -1,220 +0,0 @@
"""Minimal offline VoxCPM example for vLLM Omni."""
from __future__ import annotations
import asyncio
import time
from pathlib import Path
from typing import Any
import soundfile as sf
import torch
from vllm.utils.argparse_utils import FlexibleArgumentParser
from vllm_omni import AsyncOmni, Omni
from vllm_omni.engine.arg_utils import nullify_stage_engine_defaults
def _build_prompt(args) -> dict[str, Any]:
additional_information: dict[str, list[Any]] = {
"text": [args.text],
"cfg_value": [args.cfg_value],
"inference_timesteps": [args.inference_timesteps],
"min_len": [args.min_len],
"max_new_tokens": [args.max_new_tokens],
}
if args.streaming_prefix_len is not None:
additional_information["streaming_prefix_len"] = [args.streaming_prefix_len]
if args.ref_audio is not None:
additional_information["ref_audio"] = [args.ref_audio]
if args.ref_text is not None:
additional_information["ref_text"] = [args.ref_text]
return {
"prompt_token_ids": [1],
"additional_information": additional_information,
}
def _extract_audio_tensor(mm: dict[str, Any]) -> torch.Tensor:
audio = mm.get("audio", mm.get("model_outputs"))
if audio is None:
raise ValueError("No audio output found in multimodal output.")
if isinstance(audio, list):
parts = [torch.as_tensor(item).float().cpu().reshape(-1) for item in audio]
audio = torch.cat(parts, dim=-1) if parts else torch.zeros(0)
if not isinstance(audio, torch.Tensor):
audio = torch.as_tensor(audio)
return audio.float().cpu().reshape(-1)
def _extract_sample_rate(mm: dict[str, Any]) -> int:
sr_raw = mm.get("sr", 24000)
if isinstance(sr_raw, list) and sr_raw:
sr_raw = sr_raw[-1]
if hasattr(sr_raw, "item"):
return int(sr_raw.item())
return int(sr_raw)
def _save_audio(audio: torch.Tensor, sample_rate: int, output_dir: Path, request_id: str) -> Path:
output_dir.mkdir(parents=True, exist_ok=True)
output_path = output_dir / f"output_{request_id}.wav"
sf.write(
output_path,
audio.float().cpu().clamp(-1.0, 1.0).numpy(),
sample_rate,
format="WAV",
subtype="PCM_16",
)
return output_path
async def _run_streaming(args) -> Path:
prompt = _build_prompt(args)
output_dir = Path(args.output_dir) if args.output_dir is not None else Path("output_audio_streaming")
request_id = "streaming_example"
sample_rate = 24000
buffered_samples = 0
chunks: list[torch.Tensor] = []
started = time.perf_counter()
omni = AsyncOmni(
model=args.model,
stage_configs_path=args.stage_configs_path,
deploy_config=args.deploy_config,
log_stats=args.log_stats,
stage_init_timeout=args.stage_init_timeout,
)
try:
async for stage_output in omni.generate(prompt, request_id=request_id):
mm = getattr(stage_output, "multimodal_output", None)
if not isinstance(mm, dict):
request_output = getattr(stage_output, "request_output", None)
if request_output is None:
continue
mm = getattr(request_output, "multimodal_output", None)
if not isinstance(mm, dict) and getattr(request_output, "outputs", None):
mm = getattr(request_output.outputs[0], "multimodal_output", None)
if not isinstance(mm, dict):
continue
audio = _extract_audio_tensor(mm)
if audio.numel() == 0:
continue
sample_rate = _extract_sample_rate(mm)
if audio.numel() > buffered_samples:
delta = audio[buffered_samples:]
buffered_samples = int(audio.numel())
else:
delta = audio
buffered_samples += int(delta.numel())
if delta.numel() > 0:
chunks.append(delta)
if not chunks:
raise RuntimeError("No streaming audio chunks received from VoxCPM.")
output_audio = torch.cat(chunks, dim=0)
output_path = _save_audio(output_audio, sample_rate, output_dir, request_id)
print(f"Saved streaming audio to: {output_path} ({time.perf_counter() - started:.2f}s)")
return output_path
finally:
omni.shutdown()
def _run_sync(args) -> Path:
prompt = _build_prompt(args)
output_dir = Path(args.output_dir) if args.output_dir is not None else Path("output_audio")
request_id = "sync_example"
started = time.perf_counter()
last_mm: dict[str, Any] | None = None
omni = Omni(
model=args.model,
stage_configs_path=args.stage_configs_path,
deploy_config=args.deploy_config,
log_stats=args.log_stats,
stage_init_timeout=args.stage_init_timeout,
)
for stage_outputs in omni.generate(prompt):
request_output = getattr(stage_outputs, "request_output", None)
if request_output is None:
continue
outputs = getattr(request_output, "outputs", None)
if outputs:
for output in outputs:
mm = getattr(output, "multimodal_output", None)
if isinstance(mm, dict):
last_mm = mm
mm = getattr(request_output, "multimodal_output", None)
if isinstance(mm, dict):
last_mm = mm
if last_mm is None:
raise RuntimeError("No audio output received from VoxCPM.")
output_path = _save_audio(
_extract_audio_tensor(last_mm),
_extract_sample_rate(last_mm),
output_dir,
request_id,
)
print(f"Saved audio to: {output_path} ({time.perf_counter() - started:.2f}s)")
return output_path
def parse_args():
parser = FlexibleArgumentParser(description="Minimal offline VoxCPM example for vLLM Omni.")
parser.add_argument("--model", type=str, required=True, help="Local VoxCPM model directory.")
parser.add_argument(
"--deploy-config",
type=str,
default=None,
help=("Override the deploy config path."),
)
parser.add_argument(
"--stage-configs-path",
type=str,
default=None,
help=(
"Legacy stage_args yaml path. Required for streaming "
"(vllm_omni/model_executor/stage_configs/voxcpm_async_chunk.yaml); "
"leave unset for non-streaming to use the auto-resolved deploy config."
),
)
parser.add_argument(
"--streaming",
action="store_true",
default=False,
help="Stream audio chunks as they arrive via AsyncOmni.",
)
parser.add_argument("--text", type=str, required=True, help="Input text for synthesis.")
parser.add_argument("--ref-audio", type=str, default=None, help="Reference audio path for voice cloning.")
parser.add_argument("--ref-text", type=str, default=None, help="Transcript of the reference audio.")
parser.add_argument("--output-dir", type=str, default=None, help="Output directory for generated wav files.")
parser.add_argument("--cfg-value", type=float, default=2.0, help="Guidance value passed to VoxCPM.")
parser.add_argument("--inference-timesteps", type=int, default=10, help="Number of diffusion timesteps.")
parser.add_argument("--min-len", type=int, default=2, help="Minimum latent length.")
parser.add_argument("--max-new-tokens", type=int, default=4096, help="Maximum latent length.")
parser.add_argument(
"--streaming-prefix-len",
type=int,
default=3,
help="Streaming prefix length used by voxcpm_async_chunk.yaml.",
)
parser.add_argument("--stage-init-timeout", type=int, default=600, help="Stage initialization timeout in seconds.")
parser.add_argument("--log-stats", action="store_true", help="Enable vLLM Omni stats logging.")
nullify_stage_engine_defaults(parser)
args = parser.parse_args()
if (args.ref_audio is None) != (args.ref_text is None):
raise ValueError("Voice cloning requires --ref-audio and --ref-text together.")
return args
def main(args) -> None:
route = "streaming" if args.streaming else "sync"
print(f"Model: {args.model}")
print(f"Deploy config: {args.deploy_config or '<auto from HF model_type>'}")
print(f"Stage configs path: {args.stage_configs_path or '<unused>'}")
print(f"Route: {route}")
if route == "streaming":
asyncio.run(_run_streaming(args))
else:
_run_sync(args)
if __name__ == "__main__":
main(parse_args())
examples/online_serving/text_to_speech/README.md
+4 -67
View File
@@ -19,7 +19,6 @@ For the full list of supported architectures across all modalities, see
| MOSS-TTS-Nano | `OpenMOSS-Team/MOSS-TTS-Nano` | ✓ (`ref_audio` required) | ✓ (PCM stream) | — | ✓ |
| OmniVoice | `k2-fsa/OmniVoice` | (offline only) | — | — | — |
| Qwen3-TTS | `Qwen/Qwen3-TTS-12Hz-1.7B-{CustomVoice,VoiceDesign,Base}` | ✓ (Base) | ✓ (PCM + WebSocket) | ✓ (presets + `/v1/audio/voices` upload) | ✓ (standard + FastRTC) |
| VoxCPM | `openbmb/VoxCPM-0.5B` | ✓ | ✓ (PCM stream) | — | — |
| VoxCPM2 | `openbmb/VoxCPM2` | ✓ | ✓ (AudioWorklet via gradio) | — | ✓ |
| Voxtral TTS | `mistralai/Voxtral-4B-TTS-2603` | ✓ (gated upstream) | ✓ | ✓ (presets) | ✓ |
@@ -294,10 +293,10 @@ vllm serve Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice --omni --port 8091
./qwen3_tts/run_server.sh Base
```
### Choosing an executor backend (uniproc vs mp)
Stage configs ship with the chunked-streaming default. To use the uniproc executor (lower IPC overhead for the Base cloning task), pass `--stage-configs-path vllm_omni/model_executor/stage_configs/qwen3_tts_uniproc.yaml`. See [#2603](https://github.com/vllm-project/vllm-omni/issues/2603) and [#2604](https://github.com/vllm-project/vllm-omni/pull/2604) for the full investigation.
### Executor backend
Single-GPU serves now default to the uniproc executor (lower IPC overhead, the Base cloning use case from [#2603](https://github.com/vllm-project/vllm-omni/issues/2603) / [#2604](https://github.com/vllm-project/vllm-omni/pull/2604)). `vllm_omni/deploy/qwen3_tts.yaml` is the only Qwen3-TTS deploy config; pass `--deploy-config <path>` to override.
To opt out of chunked streaming, pass `--no-async-chunk` instead — the pipeline auto-dispatches to the end-to-end codec processor.
To opt out of chunked streaming, pass `--no-async-chunk` — the pipeline auto-dispatches to the end-to-end codec processor.
### Sending requests
```bash
@@ -375,73 +374,11 @@ python qwen3_tts/streaming_speech_client.py --text "..." --simulate-stt --stt-de
`qwen3_tts/batch_speech_client.py` issues many concurrent requests for throughput measurement.
### Notes
- Base voice cloning has uniproc-vs-mp tradeoffs depending on per-request reference audio cost; see the executor-backend section above.
- Base voice cloning has per-request reference-audio cost; the uniproc default keeps IPC overhead off the critical path. See the executor-backend section above for background.
- `vllm_omni/deploy/qwen3_tts.yaml` is the default deploy config (loaded by HF `model_type`); per-stage runtime overrides are available via `--stage-N-<field> <value>`.
---
## VoxCPM
Split-stage TTS at 24 kHz.
### Prerequisites
```bash
pip install voxcpm
# or use a local source tree:
export VLLM_OMNI_VOXCPM_CODE_PATH=/path/to/VoxCPM/src
```
If the native VoxCPM `config.json` lacks HF `model_type`, set up an HF-compatible config dir:
```bash
export VOXCPM_MODEL=/path/to/voxcpm-model
export VLLM_OMNI_VOXCPM_HF_CONFIG_PATH=/tmp/voxcpm_hf_config
mkdir -p "$VLLM_OMNI_VOXCPM_HF_CONFIG_PATH"
cp "$VOXCPM_MODEL/config.json" "$VLLM_OMNI_VOXCPM_HF_CONFIG_PATH/config.json"
cp "$VOXCPM_MODEL/generation_config.json" "$VLLM_OMNI_VOXCPM_HF_CONFIG_PATH/generation_config.json" 2>/dev/null || true
python3 -c 'import json, os; p=os.path.join(os.environ["VLLM_OMNI_VOXCPM_HF_CONFIG_PATH"], "config.json"); cfg=json.load(open(p, "r", encoding="utf-8")); cfg["model_type"]="voxcpm"; cfg.setdefault("architectures", ["VoxCPMForConditionalGeneration"]); json.dump(cfg, open(p, "w", encoding="utf-8"), indent=2, ensure_ascii=False)'
```
### Launch
```bash
export VOXCPM_MODEL=/path/to/voxcpm-model
./voxcpm/run_server.sh # async-chunk streaming (default)
./voxcpm/run_server.sh sync # non-streaming
```
Or directly:
```bash
vllm serve "$VOXCPM_MODEL" \
--stage-configs-path vllm_omni/model_executor/stage_configs/voxcpm_async_chunk.yaml \
--trust-remote-code --enforce-eager --omni --port 8091
```
### Sending requests
```bash
# Basic TTS
python voxcpm/openai_speech_client.py \
--model "$VOXCPM_MODEL" \
--text "This is a VoxCPM online text-to-speech example."
# Voice cloning
python voxcpm/openai_speech_client.py \
--model "$VOXCPM_MODEL" \
--text "This sentence is synthesized with a cloned voice." \
--ref-audio /path/to/reference.wav \
--ref-text "The exact transcript spoken in reference.wav."
# Streaming PCM
python voxcpm/openai_speech_client.py \
--model "$VOXCPM_MODEL" \
--text "This is a streaming VoxCPM request." \
--stream --output voxcpm_stream.pcm
```
### Notes
- `voxcpm.yaml` for one-shot decode; `voxcpm_async_chunk.yaml` for single-request streaming. Do not use the async-chunk config for concurrent requests or `/v1/audio/speech/batch`.
- Generic TTS fields not supported by VoxCPM: `voice`, `instructions`, `language`, `speaker_embedding`, `x_vector_only_mode`.
- For benchmark measurement, see [`benchmarks/voxcpm`](../../../benchmarks/voxcpm/README.md).
---
## VoxCPM2
Single-stage native AR TTS at 48 kHz.
examples/online_serving/text_to_speech/cosyvoice3/run_server.sh
Executable
+28
View File
@@ -0,0 +1,28 @@
#!/bin/bash
# Launch vLLM-Omni server for CosyVoice3 TTS
#
# Usage:
# ./run_server.sh
# CUDA_VISIBLE_DEVICES=0 ./run_server.sh
#
# Streaming (async-chunk) is on by default via vllm_omni/deploy/cosyvoice3.yaml.
# Set NO_ASYNC_CHUNK=1 to use the legacy synchronous path.
set -e
MODEL="${MODEL:-FunAudioLLM/Fun-CosyVoice3-0.5B-2512}"
PORT="${PORT:-8091}"
EXTRA_ARGS=()
if [[ -n "${NO_ASYNC_CHUNK:-}" ]]; then
EXTRA_ARGS+=(--no-async-chunk)
fi
echo "Starting CosyVoice3 server with model: $MODEL"
vllm serve "$MODEL" \
--host 0.0.0.0 \
--port "$PORT" \
--trust-remote-code \
--omni \
"${EXTRA_ARGS[@]}"
examples/online_serving/text_to_speech/cosyvoice3/speech_client.py
+142
View File
@@ -0,0 +1,142 @@
"""Client for CosyVoice3 TTS via /v1/audio/speech endpoint.
CosyVoice3 has no built-in voice presets: every request is voice cloning
driven by ``ref_audio`` + ``ref_text``. The defaults below point at the
official upstream zero-shot prompt so the script runs out of the box.
Examples:
# Voice cloning with the default upstream prompt
python speech_client.py --text "收到好友从远方寄来的生日礼物。"
# Custom reference clip + transcript
python speech_client.py --text "Hello, this is a cloned voice." \
--ref-audio /path/to/reference.wav \
--ref-text "Transcript of the reference audio."
# Streaming PCM output
python speech_client.py --text "Hello world" --stream --output output.pcm
"""
import argparse
import base64
import os
import httpx
DEFAULT_API_BASE = "http://localhost:8091"
DEFAULT_API_KEY = "EMPTY"
DEFAULT_MODEL = "FunAudioLLM/Fun-CosyVoice3-0.5B-2512"
# Official CosyVoice zero-shot prompt and its transcript.
DEFAULT_REF_AUDIO = "https://raw.githubusercontent.com/FunAudioLLM/CosyVoice/main/asset/zero_shot_prompt.wav"
DEFAULT_REF_TEXT = "希望你以后能够做的比我还好呦。"
def encode_audio_to_base64(audio_path: str) -> str:
"""Encode a local audio file to a base64 data URL."""
if not os.path.exists(audio_path):
raise FileNotFoundError(f"Audio file not found: {audio_path}")
ext = audio_path.lower().rsplit(".", 1)[-1]
mime_map = {"wav": "audio/wav", "mp3": "audio/mpeg", "flac": "audio/flac", "ogg": "audio/ogg"}
mime_type = mime_map.get(ext, "audio/wav")
with open(audio_path, "rb") as f:
audio_b64 = base64.b64encode(f.read()).decode("utf-8")
return f"data:{mime_type};base64,{audio_b64}"
def run_tts(args) -> None:
"""Generate speech via the /v1/audio/speech API."""
payload = {
"model": args.model,
"input": args.text,
"response_format": args.response_format,
}
if args.ref_audio.startswith(("http://", "https://")):
payload["ref_audio"] = args.ref_audio
else:
payload["ref_audio"] = encode_audio_to_base64(args.ref_audio)
payload["ref_text"] = args.ref_text
if args.stream:
payload["stream"] = True
payload["response_format"] = "pcm"
print(f"Model: {args.model}")
print(f"Text: {args.text}")
print(f"Voice cloning: ref_audio={args.ref_audio}, ref_text={args.ref_text}")
print("Generating audio...")
api_url = f"{args.api_base}/v1/audio/speech"
headers = {
"Content-Type": "application/json",
"Authorization": f"Bearer {args.api_key}",
}
if args.stream:
output_path = args.output or "output.pcm"
with httpx.Client(timeout=300.0) as client:
with client.stream("POST", api_url, json=payload, headers=headers) as resp:
if resp.status_code != 200:
print(f"Error: {resp.status_code}")
print(resp.read().decode())
return
total_bytes = 0
with open(output_path, "wb") as f:
for chunk in resp.iter_bytes():
f.write(chunk)
total_bytes += len(chunk)
print(f"Streamed {total_bytes} bytes to: {output_path}")
else:
with httpx.Client(timeout=300.0) as client:
response = client.post(api_url, json=payload, headers=headers)
if response.status_code != 200:
print(f"Error: {response.status_code}")
print(response.text)
return
try:
text = response.content.decode("utf-8")
if text.startswith('{"error"'):
print(f"Error: {text}")
return
except UnicodeDecodeError:
pass
output_path = args.output or "output.wav"
with open(output_path, "wb") as f:
f.write(response.content)
print(f"Audio saved to: {output_path}")
def main():
parser = argparse.ArgumentParser(description="CosyVoice3 TTS client")
parser.add_argument("--api-base", default=DEFAULT_API_BASE, help="API base URL")
parser.add_argument("--api-key", default=DEFAULT_API_KEY, help="API key")
parser.add_argument("--model", "-m", default=DEFAULT_MODEL, help="Model name")
parser.add_argument("--text", required=True, help="Text to synthesize")
parser.add_argument(
"--ref-audio",
default=DEFAULT_REF_AUDIO,
help="Reference audio for voice cloning (path or URL)",
)
parser.add_argument(
"--ref-text",
default=DEFAULT_REF_TEXT,
help="Transcript of the reference audio",
)
parser.add_argument("--stream", action="store_true", help="Enable streaming (PCM output)")
parser.add_argument(
"--response-format",
default="wav",
choices=["wav", "mp3", "flac", "pcm", "aac", "opus"],
help="Audio format (default: wav)",
)
parser.add_argument("--output", "-o", default=None, help="Output file path")
args = parser.parse_args()
run_tts(args)
if __name__ == "__main__":
main()
examples/online_serving/text_to_speech/voxcpm/openai_speech_client.py
-155
View File
@@ -1,155 +0,0 @@
"""OpenAI-compatible client for VoxCPM via /v1/audio/speech.
Examples:
# Basic text-to-speech
python openai_speech_client.py --text "Hello from VoxCPM"
# Voice cloning
python openai_speech_client.py \
--text "This sentence uses the cloned voice." \
--ref-audio /path/to/reference.wav \
--ref-text "The exact transcript spoken in the reference audio."
# Streaming PCM output
python openai_speech_client.py \
--text "This is a streaming VoxCPM request." \
--stream \
--output output.pcm
"""
import argparse
import base64
import os
import httpx
DEFAULT_API_BASE = "http://localhost:8091"
DEFAULT_API_KEY = "EMPTY"
DEFAULT_MODEL = "OpenBMB/VoxCPM1.5"
def encode_audio_to_base64(audio_path: str) -> str:
"""Encode a local audio file to base64 data URL."""
if not os.path.exists(audio_path):
raise FileNotFoundError(f"Audio file not found: {audio_path}")
ext = audio_path.lower().rsplit(".", 1)[-1]
mime_map = {
"wav": "audio/wav",
"mp3": "audio/mpeg",
"flac": "audio/flac",
"ogg": "audio/ogg",
}
mime_type = mime_map.get(ext, "audio/wav")
with open(audio_path, "rb") as f:
audio_b64 = base64.b64encode(f.read()).decode("utf-8")
return f"data:{mime_type};base64,{audio_b64}"
def build_payload(args) -> dict[str, object]:
payload: dict[str, object] = {
"model": args.model,
"input": args.text,
"response_format": "pcm" if args.stream else args.response_format,
}
if args.ref_audio:
if args.ref_audio.startswith(("http://", "https://", "data:")):
payload["ref_audio"] = args.ref_audio
else:
payload["ref_audio"] = encode_audio_to_base64(args.ref_audio)
if args.ref_text:
payload["ref_text"] = args.ref_text
if args.max_new_tokens is not None:
payload["max_new_tokens"] = args.max_new_tokens
if args.stream:
payload["stream"] = True
return payload
def run_tts(args) -> None:
payload = build_payload(args)
api_url = f"{args.api_base}/v1/audio/speech"
headers = {
"Content-Type": "application/json",
"Authorization": f"Bearer {args.api_key}",
}
print(f"Model: {args.model}")
print(f"Text: {args.text}")
if args.ref_audio:
print("Mode: voice cloning")
print(f"Reference audio: {args.ref_audio}")
else:
print("Mode: text-to-speech")
if args.stream:
output_path = args.output or "voxcpm_output.pcm"
with httpx.Client(timeout=300.0) as client:
with client.stream("POST", api_url, json=payload, headers=headers) as response:
if response.status_code != 200:
print(f"Error: {response.status_code}")
print(response.read().decode("utf-8", errors="ignore"))
return
total_bytes = 0
with open(output_path, "wb") as f:
for chunk in response.iter_bytes():
if not chunk:
continue
f.write(chunk)
total_bytes += len(chunk)
print(f"Streamed {total_bytes} bytes to: {output_path}")
return
with httpx.Client(timeout=300.0) as client:
response = client.post(api_url, json=payload, headers=headers)
if response.status_code != 200:
print(f"Error: {response.status_code}")
print(response.text)
return
try:
text = response.content.decode("utf-8")
if text.startswith('{"error"'):
print(f"Error: {text}")
return
except UnicodeDecodeError:
pass
output_path = args.output or "voxcpm_output.wav"
with open(output_path, "wb") as f:
f.write(response.content)
print(f"Audio saved to: {output_path}")
def main():
parser = argparse.ArgumentParser(description="VoxCPM OpenAI-compatible speech client")
parser.add_argument("--api-base", default=DEFAULT_API_BASE, help="API base URL")
parser.add_argument("--api-key", default=DEFAULT_API_KEY, help="API key")
parser.add_argument("--model", "-m", default=DEFAULT_MODEL, help="Model name or path")
parser.add_argument("--text", required=True, help="Text to synthesize")
parser.add_argument("--ref-audio", default=None, help="Reference audio path, URL, or data URL")
parser.add_argument(
"--ref-text",
default=None,
help="The exact transcript spoken in the reference audio",
)
parser.add_argument("--stream", action="store_true", help="Enable streaming PCM output")
parser.add_argument(
"--response-format",
default="wav",
choices=["wav", "pcm", "flac", "mp3", "aac", "opus"],
help="Audio format for non-streaming mode (default: wav)",
)
parser.add_argument("--max-new-tokens", type=int, default=None, help="Maximum tokens to generate")
parser.add_argument("--output", "-o", default=None, help="Output file path")
args = parser.parse_args()
run_tts(args)
if __name__ == "__main__":
main()
examples/online_serving/text_to_speech/voxcpm/run_server.sh
-38
View File
@@ -1,38 +0,0 @@
#!/bin/bash
# Launch vLLM-Omni server for VoxCPM online speech serving.
#
# Usage:
# ./run_server.sh # default: async_chunk stage config
# ./run_server.sh async # async_chunk stage config
# ./run_server.sh sync # no-async-chunk stage config
# VOXCPM_MODEL=/path/to/model ./run_server.sh
set -e
MODE="${1:-async}"
MODEL="${VOXCPM_MODEL:-OpenBMB/VoxCPM1.5}"
case "$MODE" in
async)
STAGE_CONFIG="vllm_omni/model_executor/stage_configs/voxcpm_async_chunk.yaml"
;;
sync)
STAGE_CONFIG="vllm_omni/deploy/voxcpm.yaml"
;;
*)
echo "Unknown mode: $MODE"
echo "Supported: async, sync"
exit 1
;;
esac
echo "Starting VoxCPM server with model: $MODEL"
echo "Stage config: $STAGE_CONFIG"
vllm serve "$MODEL" \
--stage-configs-path "$STAGE_CONFIG" \
--host 0.0.0.0 \
--port 8091 \
--trust-remote-code \
--enforce-eager \
--omni