Add voxcpm model support. (#2467)

Signed-off-by: Celeste-jq <591998922@qq.com>
Signed-off-by: lyj-jjj <liuyingjun5@huawei.com>
Signed-off-by: IsleOfDawnlight <stellamou@qq.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Celeste-jq <591998922@qq.com>
Co-authored-by: lyj-jjj <liuyingjun5@huawei.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>

examples/offline_inference/voxcpm/README.md

@@ -0,0 +1,123 @@
+# VoxCPM Offline Example
+
+This directory contains the minimal offline VoxCPM example for vLLM Omni.
+
+`end2end.py` is intentionally small and only covers:
+
+- single text-to-speech
+- single voice cloning with `ref_audio` + `ref_text`
+- non-streaming with `vllm_omni/model_executor/stage_configs/voxcpm.yaml`
+- streaming with `vllm_omni/model_executor/stage_configs/voxcpm_async_chunk.yaml`
+
+Advanced workflows were moved out of the getting-started example:
+
+- `benchmarks/voxcpm/vllm_omni/bench_tts_offline.py`: warmup, batch prompts, profiler, offline TTFP / RTF
+- `benchmarks/voxcpm/vllm_omni/run_offline_matrix.py`: fixed offline smoke matrix
+- `benchmarks/voxcpm/`: benchmark scripts and benchmark docs
+
+## Prerequisites
+
+Install VoxCPM in one of these ways:
+
+```bash
+pip install voxcpm
+```
+
+or point vLLM Omni to the local VoxCPM source tree:
+
+```bash
+export VLLM_OMNI_VOXCPM_CODE_PATH=/path/to/VoxCPM/src
+```
+
+The example writes WAV files with `soundfile`:
+
+```bash
+pip install soundfile
+```
+
+## Model Path
+
+Pass the native VoxCPM model directory directly:
+
+```bash
+export VOXCPM_MODEL=/path/to/voxcpm-model
+```
+
+If the native VoxCPM `config.json` does not contain HuggingFace metadata such as
+`model_type`, prepare a persistent HF-compatible config directory and point the
+stage configs to it with `VLLM_OMNI_VOXCPM_HF_CONFIG_PATH`:
+
+```bash
+export VLLM_OMNI_VOXCPM_HF_CONFIG_PATH=/tmp/voxcpm_hf_config
+mkdir -p "$VLLM_OMNI_VOXCPM_HF_CONFIG_PATH"
+cp "$VOXCPM_MODEL/config.json" "$VLLM_OMNI_VOXCPM_HF_CONFIG_PATH/config.json"
+cp "$VOXCPM_MODEL/generation_config.json" "$VLLM_OMNI_VOXCPM_HF_CONFIG_PATH/generation_config.json" 2>/dev/null || true
+python3 -c 'import json, os; p=os.path.join(os.environ["VLLM_OMNI_VOXCPM_HF_CONFIG_PATH"], "config.json"); cfg=json.load(open(p, "r", encoding="utf-8")); cfg["model_type"]="voxcpm"; cfg.setdefault("architectures", ["VoxCPMForConditionalGeneration"]); json.dump(cfg, open(p, "w", encoding="utf-8"), indent=2, ensure_ascii=False)'
+```
+
+If the model directory itself already has `model_type`, this extra directory is
+not required.
+
+## Quick Start
+
+Single text-to-speech, non-streaming:
+
+```bash
+python examples/offline_inference/voxcpm/end2end.py \
+  --model "$VOXCPM_MODEL" \
+  --text "This is a split-stage VoxCPM synthesis example running on vLLM Omni."
+```
+
+Single voice cloning, non-streaming:
+
+```bash
+python examples/offline_inference/voxcpm/end2end.py \
+  --model "$VOXCPM_MODEL" \
+  --text "This sentence is synthesized with a cloned voice." \
+  --ref-audio /path/to/reference.wav \
+  --ref-text "The exact transcript spoken in reference.wav."
+```
+
+Streaming:
+
+```bash
+python examples/offline_inference/voxcpm/end2end.py \
+  --model "$VOXCPM_MODEL" \
+  --stage-configs-path vllm_omni/model_executor/stage_configs/voxcpm_async_chunk.yaml \
+  --text "This is a split-stage VoxCPM streaming example running on vLLM Omni."
+```
+
+By default, `end2end.py` writes to `output_audio/` for non-streaming and
+`output_audio_streaming/` for streaming.
+
+## Advanced Workflows
+
+Use `benchmarks/voxcpm/vllm_omni/bench_tts_offline.py` when you need:
+
+- warmup runs
+- prompt files
+- batch JSONL inputs
+- profiler injection
+- offline TTFP / RTF emission
+
+Use `benchmarks/voxcpm/vllm_omni/run_offline_matrix.py` when you need the fixed offline smoke matrix that previously lived in `test.py`.
+
+Full matrix benchmark example:
+
+```bash
+python benchmarks/voxcpm/vllm_omni/run_offline_matrix.py \
+  --model "$VOXCPM_MODEL" \
+  --ref-audio /path/to/reference.wav \
+  --ref-text "The exact transcript spoken in reference.wav."
+```
+
+For online serving examples, see [examples/online_serving/voxcpm](../../online_serving/voxcpm/README.md).
+
+For benchmark reporting, see [benchmarks/voxcpm](../../../benchmarks/voxcpm/README.md).
+
+## Notes
+
+- `voxcpm.yaml` is the default non-streaming stage config.
+- `voxcpm_async_chunk.yaml` is the streaming stage config.
+- Streaming is currently single-request oriented; the fixed smoke matrix now lives in `benchmarks/voxcpm/vllm_omni/run_offline_matrix.py`.
+- `ref_text` must be the real transcript of the reference audio. Mismatched text usually causes obvious quality degradation.

examples/offline_inference/voxcpm/end2end.py

@@ -0,0 +1,206 @@
+"""Minimal offline VoxCPM example for vLLM Omni."""
+
+from __future__ import annotations
+
+import asyncio
+import time
+from pathlib import Path
+from typing import Any
+
+import soundfile as sf
+import torch
+from vllm.utils.argparse_utils import FlexibleArgumentParser
+
+from vllm_omni import AsyncOmni, Omni
+
+REPO_ROOT = Path(__file__).resolve().parents[3]
+DEFAULT_SYNC_STAGE_CONFIG = REPO_ROOT / "vllm_omni" / "model_executor" / "stage_configs" / "voxcpm.yaml"
+
+
+def _build_prompt(args) -> dict[str, Any]:
+    additional_information: dict[str, list[Any]] = {
+        "text": [args.text],
+        "cfg_value": [args.cfg_value],
+        "inference_timesteps": [args.inference_timesteps],
+        "min_len": [args.min_len],
+        "max_new_tokens": [args.max_new_tokens],
+    }
+    if args.streaming_prefix_len is not None:
+        additional_information["streaming_prefix_len"] = [args.streaming_prefix_len]
+    if args.ref_audio is not None:
+        additional_information["ref_audio"] = [args.ref_audio]
+    if args.ref_text is not None:
+        additional_information["ref_text"] = [args.ref_text]
+    return {
+        "prompt_token_ids": [1],
+        "additional_information": additional_information,
+    }
+
+
+def _extract_audio_tensor(mm: dict[str, Any]) -> torch.Tensor:
+    audio = mm.get("audio", mm.get("model_outputs"))
+    if audio is None:
+        raise ValueError("No audio output found in multimodal output.")
+    if isinstance(audio, list):
+        parts = [torch.as_tensor(item).float().cpu().reshape(-1) for item in audio]
+        audio = torch.cat(parts, dim=-1) if parts else torch.zeros(0)
+    if not isinstance(audio, torch.Tensor):
+        audio = torch.as_tensor(audio)
+    return audio.float().cpu().reshape(-1)
+
+
+def _extract_sample_rate(mm: dict[str, Any]) -> int:
+    sr_raw = mm.get("sr", 24000)
+    if isinstance(sr_raw, list) and sr_raw:
+        sr_raw = sr_raw[-1]
+    if hasattr(sr_raw, "item"):
+        return int(sr_raw.item())
+    return int(sr_raw)
+
+
+def _is_streaming_stage_config(stage_config_path: str) -> bool:
+    return "async_chunk" in Path(stage_config_path).stem
+
+
+def _save_audio(audio: torch.Tensor, sample_rate: int, output_dir: Path, request_id: str) -> Path:
+    output_dir.mkdir(parents=True, exist_ok=True)
+    output_path = output_dir / f"output_{request_id}.wav"
+    sf.write(
+        output_path,
+        audio.float().cpu().clamp(-1.0, 1.0).numpy(),
+        sample_rate,
+        format="WAV",
+        subtype="PCM_16",
+    )
+    return output_path
+
+
+async def _run_streaming(args) -> Path:
+    prompt = _build_prompt(args)
+    output_dir = Path(args.output_dir) if args.output_dir is not None else Path("output_audio_streaming")
+    request_id = "streaming_example"
+    sample_rate = 24000
+    buffered_samples = 0
+    chunks: list[torch.Tensor] = []
+    started = time.perf_counter()
+    omni = AsyncOmni(
+        model=args.model,
+        stage_configs_path=args.stage_configs_path,
+        log_stats=args.log_stats,
+        stage_init_timeout=args.stage_init_timeout,
+    )
+    try:
+        async for stage_output in omni.generate(prompt, request_id=request_id):
+            mm = getattr(stage_output, "multimodal_output", None)
+            if not isinstance(mm, dict):
+                request_output = getattr(stage_output, "request_output", None)
+                if request_output is None:
+                    continue
+                mm = getattr(request_output, "multimodal_output", None)
+                if not isinstance(mm, dict) and getattr(request_output, "outputs", None):
+                    mm = getattr(request_output.outputs[0], "multimodal_output", None)
+            if not isinstance(mm, dict):
+                continue
+            audio = _extract_audio_tensor(mm)
+            if audio.numel() == 0:
+                continue
+            sample_rate = _extract_sample_rate(mm)
+            if audio.numel() > buffered_samples:
+                delta = audio[buffered_samples:]
+                buffered_samples = int(audio.numel())
+            else:
+                delta = audio
+                buffered_samples += int(delta.numel())
+            if delta.numel() > 0:
+                chunks.append(delta)
+        if not chunks:
+            raise RuntimeError("No streaming audio chunks received from VoxCPM.")
+        output_audio = torch.cat(chunks, dim=0)
+        output_path = _save_audio(output_audio, sample_rate, output_dir, request_id)
+        print(f"Saved streaming audio to: {output_path} ({time.perf_counter() - started:.2f}s)")
+        return output_path
+    finally:
+        omni.shutdown()
+
+
+def _run_sync(args) -> Path:
+    prompt = _build_prompt(args)
+    output_dir = Path(args.output_dir) if args.output_dir is not None else Path("output_audio")
+    request_id = "sync_example"
+    started = time.perf_counter()
+    last_mm: dict[str, Any] | None = None
+    omni = Omni(
+        model=args.model,
+        stage_configs_path=args.stage_configs_path,
+        log_stats=args.log_stats,
+        stage_init_timeout=args.stage_init_timeout,
+    )
+    for stage_outputs in omni.generate(prompt):
+        request_output = getattr(stage_outputs, "request_output", None)
+        if request_output is None:
+            continue
+        outputs = getattr(request_output, "outputs", None)
+        if outputs:
+            for output in outputs:
+                mm = getattr(output, "multimodal_output", None)
+                if isinstance(mm, dict):
+                    last_mm = mm
+        mm = getattr(request_output, "multimodal_output", None)
+        if isinstance(mm, dict):
+            last_mm = mm
+    if last_mm is None:
+        raise RuntimeError("No audio output received from VoxCPM.")
+    output_path = _save_audio(
+        _extract_audio_tensor(last_mm),
+        _extract_sample_rate(last_mm),
+        output_dir,
+        request_id,
+    )
+    print(f"Saved audio to: {output_path} ({time.perf_counter() - started:.2f}s)")
+    return output_path
+
+
+def parse_args():
+    parser = FlexibleArgumentParser(description="Minimal offline VoxCPM example for vLLM Omni.")
+    parser.add_argument("--model", type=str, required=True, help="Local VoxCPM model directory.")
+    parser.add_argument(
+        "--stage-configs-path",
+        type=str,
+        default=str(DEFAULT_SYNC_STAGE_CONFIG),
+        help=("Stage config path. Use voxcpm.yaml for non-streaming or voxcpm_async_chunk.yaml for streaming."),
+    )
+    parser.add_argument("--text", type=str, required=True, help="Input text for synthesis.")
+    parser.add_argument("--ref-audio", type=str, default=None, help="Reference audio path for voice cloning.")
+    parser.add_argument("--ref-text", type=str, default=None, help="Transcript of the reference audio.")
+    parser.add_argument("--output-dir", type=str, default=None, help="Output directory for generated wav files.")
+    parser.add_argument("--cfg-value", type=float, default=2.0, help="Guidance value passed to VoxCPM.")
+    parser.add_argument("--inference-timesteps", type=int, default=10, help="Number of diffusion timesteps.")
+    parser.add_argument("--min-len", type=int, default=2, help="Minimum latent length.")
+    parser.add_argument("--max-new-tokens", type=int, default=4096, help="Maximum latent length.")
+    parser.add_argument(
+        "--streaming-prefix-len",
+        type=int,
+        default=3,
+        help="Streaming prefix length used by voxcpm_async_chunk.yaml.",
+    )
+    parser.add_argument("--stage-init-timeout", type=int, default=600, help="Stage initialization timeout in seconds.")
+    parser.add_argument("--log-stats", action="store_true", help="Enable vLLM Omni stats logging.")
+    args = parser.parse_args()
+    if (args.ref_audio is None) != (args.ref_text is None):
+        raise ValueError("Voice cloning requires --ref-audio and --ref-text together.")
+    return args
+
+
+def main(args) -> None:
+    route = "streaming" if _is_streaming_stage_config(args.stage_configs_path) else "sync"
+    print(f"Model: {args.model}")
+    print(f"Stage config: {args.stage_configs_path}")
+    print(f"Route: {route}")
+    if route == "streaming":
+        asyncio.run(_run_streaming(args))
+    else:
+        _run_sync(args)
+
+
+if __name__ == "__main__":
+    main(parse_args())

examples/online_serving/voxcpm/README.md

@@ -0,0 +1,166 @@
+# VoxCPM
+
+## Prerequisites
+
+Install VoxCPM in one of these ways:
+
+```bash
+pip install voxcpm
+```
+
+or point vLLM-Omni to a local VoxCPM source tree:
+
+```bash
+export VLLM_OMNI_VOXCPM_CODE_PATH=/path/to/VoxCPM/src
+```
+
+If the native VoxCPM `config.json` lacks HF metadata such as `model_type`,
+prepare a persistent HF-compatible config directory and export:
+
+```bash
+export VLLM_OMNI_VOXCPM_HF_CONFIG_PATH=/tmp/voxcpm_hf_config
+mkdir -p "$VLLM_OMNI_VOXCPM_HF_CONFIG_PATH"
+cp "$VOXCPM_MODEL/config.json" "$VLLM_OMNI_VOXCPM_HF_CONFIG_PATH/config.json"
+cp "$VOXCPM_MODEL/generation_config.json" "$VLLM_OMNI_VOXCPM_HF_CONFIG_PATH/generation_config.json" 2>/dev/null || true
+python3 -c 'import json, os; p=os.path.join(os.environ["VLLM_OMNI_VOXCPM_HF_CONFIG_PATH"], "config.json"); cfg=json.load(open(p, "r", encoding="utf-8")); cfg["model_type"]="voxcpm"; cfg.setdefault("architectures", ["VoxCPMForConditionalGeneration"]); json.dump(cfg, open(p, "w", encoding="utf-8"), indent=2, ensure_ascii=False)'
+```
+
+The VoxCPM stage configs read `VLLM_OMNI_VOXCPM_HF_CONFIG_PATH` directly. The `python3 -c` form above avoids heredoc/indentation issues in interactive shells.
+
+## Launch the Server
+
+Use the async-chunk stage config by default:
+
+```bash
+export VOXCPM_MODEL=/path/to/voxcpm-model
+cd examples/online_serving/voxcpm
+./run_server.sh
+```
+
+Use the non-streaming stage config:
+
+```bash
+./run_server.sh sync
+```
+
+You can also launch the server directly:
+
+```bash
+vllm serve "$VOXCPM_MODEL" \
+    --stage-configs-path vllm_omni/model_executor/stage_configs/voxcpm_async_chunk.yaml \
+    --trust-remote-code \
+    --enforce-eager \
+    --omni \
+    --port 8091
+```
+
+## Send Requests
+
+### Basic text-to-speech
+
+```bash
+python openai_speech_client.py \
+    --model "$VOXCPM_MODEL" \
+    --text "This is a VoxCPM online text-to-speech example."
+```
+
+### Voice cloning
+
+```bash
+python openai_speech_client.py \
+    --model "$VOXCPM_MODEL" \
+    --text "This sentence is synthesized with a cloned voice." \
+    --ref-audio /path/to/reference.wav \
+    --ref-text "The exact transcript spoken in reference.wav."
+```
+
+`ref_text` must be the real transcript of the reference audio. Placeholder text or mismatched text will usually degrade quality badly.
+
+### Streaming PCM output
+
+```bash
+python openai_speech_client.py \
+    --model "$VOXCPM_MODEL" \
+    --text "This is a streaming VoxCPM request." \
+    --stream \
+    --output voxcpm_stream.pcm
+```
+
+### Using curl
+
+```bash
+curl -X POST http://localhost:8091/v1/audio/speech \
+    -H "Content-Type: application/json" \
+    -d '{
+        "model": "OpenBMB/VoxCPM1.5",
+        "input": "Hello from VoxCPM online serving.",
+        "response_format": "wav"
+    }' --output output.wav
+```
+
+Voice cloning:
+
+```bash
+curl -X POST http://localhost:8091/v1/audio/speech \
+    -H "Content-Type: application/json" \
+    -d '{
+        "model": "OpenBMB/VoxCPM1.5",
+        "input": "This sentence uses a cloned voice.",
+        "ref_audio": "https://example.com/reference.wav",
+        "ref_text": "The exact transcript spoken in the reference audio.",
+        "response_format": "wav"
+    }' --output cloned.wav
+```
+
+Streaming PCM:
+
+```bash
+curl -X POST http://localhost:8091/v1/audio/speech \
+    -H "Content-Type: application/json" \
+    -d '{
+        "model": "OpenBMB/VoxCPM1.5",
+        "input": "This is a streaming VoxCPM request.",
+        "stream": true,
+        "response_format": "pcm"
+    }' --output output.pcm
+```
+
+## Supported Request Shape
+
+VoxCPM online serving currently supports:
+
+- plain text-to-speech
+- voice cloning with `ref_audio` + `ref_text`
+- `stream=true` with `response_format=pcm` or `wav`
+
+VoxCPM online serving does not use these generic TTS fields:
+
+- `voice`
+- `instructions`
+- `language`
+- `speaker_embedding`
+- `x_vector_only_mode`
+
+## Streaming vs Non-Streaming
+
+- `voxcpm_async_chunk.yaml` enables async-chunk streaming and is best for single-request streaming latency.
+- `voxcpm.yaml` performs one-shot latent generation then VAE decode.
+
+Like native VoxCPM, the async streaming path should be treated as single-request. If you need stable throughput benchmarking, prefer `voxcpm.yaml`.
+
+Do not use `voxcpm_async_chunk.yaml` for concurrent online streaming or `/v1/audio/speech/batch`. For multiple requests, prefer `voxcpm.yaml`.
+
+## Benchmark
+
+The serving benchmark reports TTFP and RTF:
+
+```bash
+python benchmarks/voxcpm/vllm_omni/bench_tts_serve.py \
+    --host 127.0.0.1 \
+    --port 8091 \
+    --num-prompts 10 \
+    --max-concurrency 1 \
+    --result-dir /tmp/voxcpm_bench
+```
+
+For the async-chunk server, keep `--max-concurrency 1`.

examples/online_serving/voxcpm/openai_speech_client.py

@@ -0,0 +1,155 @@
+"""OpenAI-compatible client for VoxCPM via /v1/audio/speech.
+
+Examples:
+    # Basic text-to-speech
+    python openai_speech_client.py --text "Hello from VoxCPM"
+
+    # Voice cloning
+    python openai_speech_client.py \
+        --text "This sentence uses the cloned voice." \
+        --ref-audio /path/to/reference.wav \
+        --ref-text "The exact transcript spoken in the reference audio."
+
+    # Streaming PCM output
+    python openai_speech_client.py \
+        --text "This is a streaming VoxCPM request." \
+        --stream \
+        --output output.pcm
+"""
+
+import argparse
+import base64
+import os
+
+import httpx
+
+DEFAULT_API_BASE = "http://localhost:8091"
+DEFAULT_API_KEY = "EMPTY"
+DEFAULT_MODEL = "OpenBMB/VoxCPM1.5"
+
+
+def encode_audio_to_base64(audio_path: str) -> str:
+    """Encode a local audio file to base64 data URL."""
+    if not os.path.exists(audio_path):
+        raise FileNotFoundError(f"Audio file not found: {audio_path}")
+
+    ext = audio_path.lower().rsplit(".", 1)[-1]
+    mime_map = {
+        "wav": "audio/wav",
+        "mp3": "audio/mpeg",
+        "flac": "audio/flac",
+        "ogg": "audio/ogg",
+    }
+    mime_type = mime_map.get(ext, "audio/wav")
+
+    with open(audio_path, "rb") as f:
+        audio_b64 = base64.b64encode(f.read()).decode("utf-8")
+    return f"data:{mime_type};base64,{audio_b64}"
+
+
+def build_payload(args) -> dict[str, object]:
+    payload: dict[str, object] = {
+        "model": args.model,
+        "input": args.text,
+        "response_format": "pcm" if args.stream else args.response_format,
+    }
+
+    if args.ref_audio:
+        if args.ref_audio.startswith(("http://", "https://", "data:")):
+            payload["ref_audio"] = args.ref_audio
+        else:
+            payload["ref_audio"] = encode_audio_to_base64(args.ref_audio)
+    if args.ref_text:
+        payload["ref_text"] = args.ref_text
+    if args.max_new_tokens is not None:
+        payload["max_new_tokens"] = args.max_new_tokens
+    if args.stream:
+        payload["stream"] = True
+
+    return payload
+
+
+def run_tts(args) -> None:
+    payload = build_payload(args)
+    api_url = f"{args.api_base}/v1/audio/speech"
+    headers = {
+        "Content-Type": "application/json",
+        "Authorization": f"Bearer {args.api_key}",
+    }
+
+    print(f"Model: {args.model}")
+    print(f"Text: {args.text}")
+    if args.ref_audio:
+        print("Mode: voice cloning")
+        print(f"Reference audio: {args.ref_audio}")
+    else:
+        print("Mode: text-to-speech")
+
+    if args.stream:
+        output_path = args.output or "voxcpm_output.pcm"
+        with httpx.Client(timeout=300.0) as client:
+            with client.stream("POST", api_url, json=payload, headers=headers) as response:
+                if response.status_code != 200:
+                    print(f"Error: {response.status_code}")
+                    print(response.read().decode("utf-8", errors="ignore"))
+                    return
+
+                total_bytes = 0
+                with open(output_path, "wb") as f:
+                    for chunk in response.iter_bytes():
+                        if not chunk:
+                            continue
+                        f.write(chunk)
+                        total_bytes += len(chunk)
+        print(f"Streamed {total_bytes} bytes to: {output_path}")
+        return
+
+    with httpx.Client(timeout=300.0) as client:
+        response = client.post(api_url, json=payload, headers=headers)
+
+    if response.status_code != 200:
+        print(f"Error: {response.status_code}")
+        print(response.text)
+        return
+
+    try:
+        text = response.content.decode("utf-8")
+        if text.startswith('{"error"'):
+            print(f"Error: {text}")
+            return
+    except UnicodeDecodeError:
+        pass
+
+    output_path = args.output or "voxcpm_output.wav"
+    with open(output_path, "wb") as f:
+        f.write(response.content)
+    print(f"Audio saved to: {output_path}")
+
+
+def main():
+    parser = argparse.ArgumentParser(description="VoxCPM OpenAI-compatible speech client")
+    parser.add_argument("--api-base", default=DEFAULT_API_BASE, help="API base URL")
+    parser.add_argument("--api-key", default=DEFAULT_API_KEY, help="API key")
+    parser.add_argument("--model", "-m", default=DEFAULT_MODEL, help="Model name or path")
+    parser.add_argument("--text", required=True, help="Text to synthesize")
+    parser.add_argument("--ref-audio", default=None, help="Reference audio path, URL, or data URL")
+    parser.add_argument(
+        "--ref-text",
+        default=None,
+        help="The exact transcript spoken in the reference audio",
+    )
+    parser.add_argument("--stream", action="store_true", help="Enable streaming PCM output")
+    parser.add_argument(
+        "--response-format",
+        default="wav",
+        choices=["wav", "pcm", "flac", "mp3", "aac", "opus"],
+        help="Audio format for non-streaming mode (default: wav)",
+    )
+    parser.add_argument("--max-new-tokens", type=int, default=None, help="Maximum tokens to generate")
+    parser.add_argument("--output", "-o", default=None, help="Output file path")
+    args = parser.parse_args()
+    run_tts(args)
+
+
+if __name__ == "__main__":
+    main()

examples/online_serving/voxcpm/run_server.sh

@@ -0,0 +1,38 @@
+#!/bin/bash
+# Launch vLLM-Omni server for VoxCPM online speech serving.
+#
+# Usage:
+#   ./run_server.sh                 # default: async_chunk stage config
+#   ./run_server.sh async           # async_chunk stage config
+#   ./run_server.sh sync            # no-async-chunk stage config
+#   VOXCPM_MODEL=/path/to/model ./run_server.sh
+
+set -e
+
+MODE="${1:-async}"
+MODEL="${VOXCPM_MODEL:-OpenBMB/VoxCPM1.5}"
+
+case "$MODE" in
+    async)
+        STAGE_CONFIG="vllm_omni/model_executor/stage_configs/voxcpm_async_chunk.yaml"
+        ;;
+    sync)
+        STAGE_CONFIG="vllm_omni/model_executor/stage_configs/voxcpm.yaml"
+        ;;
+    *)
+        echo "Unknown mode: $MODE"
+        echo "Supported: async, sync"
+        exit 1
+        ;;
+esac
+
+echo "Starting VoxCPM server with model: $MODEL"
+echo "Stage config: $STAGE_CONFIG"
+
+vllm serve "$MODEL" \
+    --stage-configs-path "$STAGE_CONFIG" \
+    --host 0.0.0.0 \
+    --port 8091 \
+    --trust-remote-code \
+    --enforce-eager \
+    --omni