diff --git a/docs/user_guide/examples/offline_inference/text_to_speech.md b/docs/user_guide/examples/offline_inference/text_to_speech.md index 95ffc5543..62e6b96d2 100644 --- a/docs/user_guide/examples/offline_inference/text_to_speech.md +++ b/docs/user_guide/examples/offline_inference/text_to_speech.md @@ -2,7 +2,6 @@ Source . - vLLM-Omni supports several autoregressive TTS models. They share a common CLI shape (`--text`, `--ref-audio`, `--ref-text`, `--output-dir`) and live together in this hub. Each model has its own subdirectory containing a @@ -15,7 +14,7 @@ list of supported architectures across all modalities, see ## Supported Models | Model | HuggingFace repo | Stages | Voice cloning | Streaming | Special modes | Sample rate | -|---|---|---|---|---|---|---| +| --- | --- | --- | --- | --- | --- | --- | | VoxCPM2 | `openbmb/VoxCPM2` | single (native AR) | ✓ | — | continuation (`--ref-audio` + `--ref-text`) | 48 kHz | | CosyVoice3 | `FunAudioLLM/Fun-CosyVoice3-0.5B-2512` | 2 (talker + code2wav) | ✓ | ✓ (`async_chunk: true` default) | — | 22.05 kHz | | Fish Speech S2 Pro | `fishaudio/s2-pro` | dual-AR | ✓ | ✓ (`--streaming`) | — | 44.1 kHz | @@ -45,13 +44,15 @@ python examples/offline_inference/text_to_speech//end2end.py \ Single-stage native AR TTS at 48 kHz. Pipeline: `feat_encoder → MiniCPM4 → FSQ → residual_lm → LocDiT → AudioVAE`. ### Prerequisites + ```bash pip install voxcpm # or, for a local source checkout: -export VLLM_OMNI_VOXCPM_CODE_PATH=/path/to/voxcpm +pip install -e /path/to/voxcpm ``` ### Quick start + ```bash python examples/offline_inference/text_to_speech/voxcpm2/end2end.py \ --model openbmb/VoxCPM2 \ @@ -59,7 +60,9 @@ python examples/offline_inference/text_to_speech/voxcpm2/end2end.py \ ``` ### Voice cloning + Pass a reference audio for isolated cloning, or both `--ref-audio` + `--ref-text` for prompt continuation: + ```bash python examples/offline_inference/text_to_speech/voxcpm2/end2end.py \ --text "Hello, this is a voice clone demo." \ @@ -68,6 +71,7 @@ python examples/offline_inference/text_to_speech/voxcpm2/end2end.py \ ``` ### Notes + - Output: 48 kHz mono WAV. - Deploy config: `vllm_omni/deploy/voxcpm2.yaml` (default). @@ -78,12 +82,14 @@ python examples/offline_inference/text_to_speech/voxcpm2/end2end.py \ 2-stage TTS pipeline (`talker` + `code2wav`) at 22.05 kHz. ### Prerequisites + ```bash uv pip install -e . # Includes soundfile, onnxruntime, x-transformers, einops via requirements. ``` Download the model snapshot: + ```python from huggingface_hub import snapshot_download snapshot_download('FunAudioLLM/Fun-CosyVoice3-0.5B-2512', @@ -91,15 +97,18 @@ snapshot_download('FunAudioLLM/Fun-CosyVoice3-0.5B-2512', ``` If your downloaded checkpoint lacks `config.json`, add it: + ```json { "model_type": "cosyvoice3", "architectures": ["CosyVoice3Model"] } ``` + This is required because `AutoConfig.register("cosyvoice3", CosyVoice3Config)` only registers the class mapping; the loader still reads `model_type` from `config.json` to select the class. ### Quick start + ```bash python examples/offline_inference/text_to_speech/cosyvoice3/end2end.py \ --model pretrained_models/Fun-CosyVoice3-0.5B \ @@ -107,7 +116,9 @@ python examples/offline_inference/text_to_speech/cosyvoice3/end2end.py \ ``` ### Voice cloning + Pass a reference audio. Note that CosyVoice3's `--prompt-text` is a system-style prompt for the GPT stage, not a reference transcript: + ```bash python examples/offline_inference/text_to_speech/cosyvoice3/end2end.py \ --model pretrained_models/Fun-CosyVoice3-0.5B \ @@ -117,6 +128,7 @@ python examples/offline_inference/text_to_speech/cosyvoice3/end2end.py \ ``` ### Notes + - Stage 0 (`talker`) emits speech tokens; stage 1 (`code2wav`) runs flow matching + HiFiGAN to synthesize waveform. - Deploy config auto-loads from `vllm_omni/deploy/cosyvoice3.yaml` based on HF `model_type`. Pass `--deploy-config ` to override. - `async_chunk: true` is the default; pass `--no-async-chunk` to switch to the legacy synchronous path. @@ -129,12 +141,14 @@ python examples/offline_inference/text_to_speech/cosyvoice3/end2end.py \ No extra packages are required; the DAC codec is vendored in vLLM-Omni. ### Quick start + ```bash python examples/offline_inference/text_to_speech/fish_speech/end2end.py \ --text "Hello, this is a test of the Fish Speech text to speech system." ``` ### Voice cloning + ```bash python examples/offline_inference/text_to_speech/fish_speech/end2end.py \ --text "Hello, this is a cloned voice." \ @@ -143,14 +157,17 @@ python examples/offline_inference/text_to_speech/fish_speech/end2end.py \ ``` ### Streaming + ```bash python examples/offline_inference/text_to_speech/fish_speech/end2end.py \ --text "Hello, this is a streaming test." \ --streaming ``` + Streaming requires `async_chunk: true` in the deploy config. ### Notes + - Output: 44.1 kHz mono WAV. - DAC codec weights (`codec.pth`) are loaded lazily from the model directory. @@ -161,6 +178,7 @@ Streaming requires `async_chunk: true` in the deploy config. 2-stage TTS pipeline (AR + DiT flow-matching) at 24 kHz. Every request requires reference audio and its transcript for zero-shot voice cloning. ### Quick start + ```bash python examples/offline_inference/text_to_speech/glm_tts/end2end.py \ --model zai-org/GLM-TTS \ @@ -171,12 +189,14 @@ python examples/offline_inference/text_to_speech/glm_tts/end2end.py \ ``` ### Architecture -``` + +```text Text → [Stage 0: AR] → Speech Tokens → [Stage 1: DiT + HiFT] → Audio (24 kHz) (Llama-based) (32k vocab) (Flow Matching) ``` ### Notes + - `--ref-audio` and `--ref-text` are **required** together; GLM-TTS does not support text-only synthesis. - Reference audio should be 3-10 seconds. - First run may be slow due to lazy loading of WhisperVQ tokenizer and CampPlus ONNX speaker embedder. @@ -190,12 +210,15 @@ Text → [Stage 0: AR] → Speech Tokens → [Stage 1: DiT + HiFT] → Audio (24 Zero-shot multilingual TTS supporting 600+ languages, with three modes (auto / clone / design). ### Prerequisites + ```bash huggingface-cli download k2-fsa/OmniVoice ``` + Voice cloning requires `transformers>=5.3.0`. Auto and design modes work with `transformers>=4.57.0`. ### Quick start (auto voice) + ```bash python examples/offline_inference/text_to_speech/omnivoice/end2end.py \ --model k2-fsa/OmniVoice \ @@ -203,6 +226,7 @@ python examples/offline_inference/text_to_speech/omnivoice/end2end.py \ ``` ### Voice cloning + ```bash python examples/offline_inference/text_to_speech/omnivoice/end2end.py \ --model k2-fsa/OmniVoice \ @@ -212,6 +236,7 @@ python examples/offline_inference/text_to_speech/omnivoice/end2end.py \ ``` ### Voice design + ```bash python examples/offline_inference/text_to_speech/omnivoice/end2end.py \ --model k2-fsa/OmniVoice \ @@ -220,6 +245,7 @@ python examples/offline_inference/text_to_speech/omnivoice/end2end.py \ ``` ### Language hint + ```bash python examples/offline_inference/text_to_speech/omnivoice/end2end.py \ --model k2-fsa/OmniVoice \ @@ -228,6 +254,7 @@ python examples/offline_inference/text_to_speech/omnivoice/end2end.py \ ``` ### Notes + - Stage 0 (Generator): Qwen3-0.6B with 32-step iterative unmasking. - Stage 1 (Decoder): HiggsAudioV2 RVQ + DAC at 24 kHz. @@ -238,13 +265,16 @@ python examples/offline_inference/text_to_speech/omnivoice/end2end.py \ 3-task-variant TTS with 24 kHz output. Has its own argparse surface (this script does not follow the common `--text` / `--ref-audio` shape). ### Prerequisites + For ROCm builds, replace `onnxruntime` with `onnxruntime-rocm`: + ```bash pip uninstall onnxruntime pip install onnxruntime-rocm ``` ### Task variants + - `CustomVoice`: predefined speaker (speaker ID) with optional style instruction. - `VoiceDesign`: text + descriptive instruction designs a new voice. - `Base`: voice cloning from reference audio + transcript. @@ -264,16 +294,20 @@ python examples/offline_inference/text_to_speech/qwen3_tts/end2end.py --query-ty ``` ### Streaming + ```bash python examples/offline_inference/text_to_speech/qwen3_tts/end2end.py \ --query-type CustomVoice \ --streaming \ --output-dir /tmp/out_stream ``` + Streaming requires `async_chunk: true` in the deploy config. ### Batched decoding + The Code2Wav stage supports batched decoding through the SpeechTokenizer. Configure both stages with `max_num_seqs > 1` via `--stage-overrides` and pass multiple prompts via `--txt-prompts`: + ```bash python examples/offline_inference/text_to_speech/qwen3_tts/end2end.py \ --query-type CustomVoice \ @@ -281,9 +315,11 @@ python examples/offline_inference/text_to_speech/qwen3_tts/end2end.py \ --batch-size 4 \ --stage-overrides '{"0":{"max_num_seqs":4,"gpu_memory_utilization":0.2},"1":{"max_num_seqs":4,"gpu_memory_utilization":0.2}}' ``` + `--batch-size` must match a CUDA-graph capture size (1, 2, 4, 8, 16…). ### Notes + - Run `--help` for the full argument surface. - See `qwen3_tts/end2end.py` for the prompt-length-estimation logic the Talker uses. @@ -294,12 +330,15 @@ python examples/offline_inference/text_to_speech/qwen3_tts/end2end.py \ Voxtral-4B-TTS (Mistral). Has its own argparse surface; uses voice presets and the `mistral_common` `SpeechRequest` protocol. ### Prerequisites + Latest `mistral_common` with `SpeechRequest` support: + ```bash pip install -e /path/to/mistral-common # or upgrade from PyPI when available ``` ### Quick start (voice preset) + ```bash python examples/offline_inference/text_to_speech/voxtral_tts/end2end.py \ --write-audio --voice cheerful_female \ @@ -308,6 +347,7 @@ python examples/offline_inference/text_to_speech/voxtral_tts/end2end.py \ ``` ### Voice cloning (capability gated upstream) + ```bash python examples/offline_inference/text_to_speech/voxtral_tts/end2end.py \ --write-audio \ @@ -317,15 +357,18 @@ python examples/offline_inference/text_to_speech/voxtral_tts/end2end.py \ ``` ### Streaming + concurrency + ```bash python examples/offline_inference/text_to_speech/voxtral_tts/end2end.py \ --num-prompts 32 --concurrency 8 --streaming --write-audio --voice neutral_female \ --model mistralai/Voxtral-4B-TTS-2603 \ --text "..." ``` + Available voice presets are listed on the HF model card (`mistralai/Voxtral-4B-TTS-2603`). ### Notes + - `--num-prompts N` replicates the prompt for performance measurement. - `--concurrency M` requires `--streaming` and must evenly divide `--num-prompts`. - Run `--help` for the full argument surface. diff --git a/examples/offline_inference/text_to_speech/README.md b/examples/offline_inference/text_to_speech/README.md index 36d1b44ee..f031ee2e7 100644 --- a/examples/offline_inference/text_to_speech/README.md +++ b/examples/offline_inference/text_to_speech/README.md @@ -563,7 +563,7 @@ Single-stage native AR TTS at 48 kHz. Pipeline: `feat_encoder → MiniCPM4 → F ```bash pip install voxcpm # or, for a local source checkout: -export VLLM_OMNI_VOXCPM_CODE_PATH=/path/to/voxcpm +pip install -e /path/to/voxcpm ``` ### Quick start diff --git a/examples/offline_inference/text_to_speech/voxcpm2/end2end.py b/examples/offline_inference/text_to_speech/voxcpm2/end2end.py index feba91b7e..53941fb24 100644 --- a/examples/offline_inference/text_to_speech/voxcpm2/end2end.py +++ b/examples/offline_inference/text_to_speech/voxcpm2/end2end.py @@ -1,7 +1,10 @@ +# SPDX-License-Identifier: Apache-2.0 +# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project + """Offline VoxCPM2 inference example (native AR pipeline). Uses the single-stage native AR config (voxcpm2.yaml). -Requires the `voxcpm` package or VLLM_OMNI_VOXCPM_CODE_PATH env var. +Requires the `voxcpm` package (pip install voxcpm). """ from __future__ import annotations diff --git a/tests/model_executor/models/voxcpm2/test_voxcpm2_import_utils.py b/tests/model_executor/models/voxcpm2/test_voxcpm2_import_utils.py new file mode 100644 index 000000000..f1f0e74c9 --- /dev/null +++ b/tests/model_executor/models/voxcpm2/test_voxcpm2_import_utils.py @@ -0,0 +1,37 @@ +# SPDX-License-Identifier: Apache-2.0 +# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project +"""VLLM_OMNI_VOXCPM_CODE_PATH still works but warns that it is deprecated (#6232).""" + +from pathlib import Path + +import pytest + +from vllm_omni.model_executor.models.voxcpm2 import voxcpm2_import_utils + +pytestmark = [pytest.mark.core_model, pytest.mark.cpu] + + +def _capture_warnings(monkeypatch) -> list[str]: + messages: list[str] = [] + monkeypatch.setattr(voxcpm2_import_utils.logger, "warning_once", lambda msg, *args: messages.append(msg)) + return messages + + +def test_code_path_env_warns_and_is_still_used(monkeypatch, tmp_path: Path): + messages = _capture_warnings(monkeypatch) + monkeypatch.setenv("VLLM_OMNI_VOXCPM_CODE_PATH", str(tmp_path)) + + candidates = voxcpm2_import_utils._iter_voxcpm2_src_candidates() + + assert candidates[0] == tmp_path + assert len(messages) == 1 + assert "VLLM_OMNI_VOXCPM_CODE_PATH is deprecated" in messages[0] + + +def test_no_warning_without_code_path_env(monkeypatch): + messages = _capture_warnings(monkeypatch) + monkeypatch.delenv("VLLM_OMNI_VOXCPM_CODE_PATH", raising=False) + + voxcpm2_import_utils._iter_voxcpm2_src_candidates() + + assert messages == [] diff --git a/vllm_omni/model_executor/models/voxcpm2/voxcpm2_import_utils.py b/vllm_omni/model_executor/models/voxcpm2/voxcpm2_import_utils.py index 231a51bbc..0a1b75a4a 100644 --- a/vllm_omni/model_executor/models/voxcpm2/voxcpm2_import_utils.py +++ b/vllm_omni/model_executor/models/voxcpm2/voxcpm2_import_utils.py @@ -1,9 +1,9 @@ # SPDX-License-Identifier: Apache-2.0 -# SPDX-FileCopyrightText: Copyright contributors to the vLLM project +# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project """Dynamic import utilities for the native VoxCPM2 package. Supports three discovery modes (first match wins): -1. ``VLLM_OMNI_VOXCPM_CODE_PATH`` env var (explicit source tree) +1. ``VLLM_OMNI_VOXCPM_CODE_PATH`` env var (explicit source tree; deprecated) 2. Sibling ``../VoxCPM/src`` relative to the vllm-omni repo root 3. pip-installed ``voxcpm`` package (>= 2.0) """ @@ -26,6 +26,10 @@ def _iter_voxcpm2_src_candidates() -> list[Path]: candidates: list[Path] = [] env_path = os.environ.get("VLLM_OMNI_VOXCPM_CODE_PATH") if env_path: + logger.warning_once( + "VLLM_OMNI_VOXCPM_CODE_PATH is deprecated and will be removed in a future release. " + "Install voxcpm>=2.0 instead (pip install voxcpm, or pip install -e for a source checkout)." + ) candidates.append(Path(env_path).expanduser()) repo_root = Path(__file__).resolve().parents[4] @@ -71,7 +75,6 @@ def _import_voxcpm2_attrs(module_name: str, *attr_names: str) -> tuple[Any, ...] raise ImportError( f"Could not import {attr_names} from {module_name}. " f"Install voxcpm>=2.0: pip install voxcpm. " - f"Or set VLLM_OMNI_VOXCPM_CODE_PATH to the VoxCPM source tree. " f"Last error: {last_exc}" )