[Docs] Consolidate moss_tts_nano + ming_flash_omni_tts into TTS hub (#3358)
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
This commit is contained in:
Yueqian Lin authored and GitHub committed
1 parent
44cde33eb5
commit
5cf3f7947b
20 files changed
+210
-494
No files matched your search
@@ -9,7 +9,7 @@ vLLM-Omni supports two deployment modes:
|
||||
| Thinker + Talker (omni-speech, default) | `vllm_omni/deploy/ming_flash_omni.yaml` | Text + Audio |
|
||||
| Thinker only (multimodal understanding) | `vllm_omni/deploy/ming_flash_omni_thinker_only.yaml` | Text |
|
||||
|
||||
For standalone TTS (talker only), see [`examples/offline_inference/ming_flash_omni_tts/`](../ming_flash_omni_tts/).
|
||||
For standalone TTS (talker only), see the [Ming-flash-omni-TTS section in the Text-To-Speech hub](../text_to_speech/README.md#ming-flash-omni-tts).
|
||||
|
||||
## Setup
|
||||
|
||||
|
||||
@@ -1,47 +0,0 @@
|
||||
# Ming-flash-omni Standalone TTS (Offline)
|
||||
|
||||
This example runs **Ming-flash-omni-2.0 talker-only** offline inference with:
|
||||
|
||||
- `model`: `Jonathan1909/Ming-flash-omni-2.0`
|
||||
- `deploy config`: `vllm_omni/deploy/ming_flash_omni_tts.yaml`
|
||||
|
||||
It follows the Ming cookbook parameter style:
|
||||
|
||||
- `prompt`: `"Please generate speech based on the following description.\n"`
|
||||
- `max_decode_steps`: `200`
|
||||
- `cfg`: `2.0`
|
||||
- `sigma`: `0.25`
|
||||
- `temperature`: `0.0`
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
python examples/offline_inference/ming_flash_omni_tts/end2end.py --case style
|
||||
```
|
||||
|
||||
## Cases
|
||||
|
||||
```bash
|
||||
# Style
|
||||
python examples/offline_inference/ming_flash_omni_tts/end2end.py --case style
|
||||
|
||||
# IP
|
||||
python examples/offline_inference/ming_flash_omni_tts/end2end.py --case ip
|
||||
|
||||
# Basic (speed/pitch/volume control)
|
||||
python examples/offline_inference/ming_flash_omni_tts/end2end.py --case basic
|
||||
```
|
||||
|
||||
## Useful Arguments
|
||||
|
||||
- `--text`: override default text in the selected case
|
||||
- `--output`: custom output wav path
|
||||
- `--model`: local model path or HF repo id
|
||||
- `--deploy-config`: custom talker deploy YAML path
|
||||
- `--log-stats`: enable runtime stats logs
|
||||
|
||||
## Notes
|
||||
|
||||
- This directory is for **standalone talker deployment (TTS)**.
|
||||
- For Ming thinker multimodal understanding examples, see:
|
||||
`examples/offline_inference/ming_flash_omni/`.
|
||||
@@ -1,97 +0,0 @@
|
||||
# MOSS-TTS-Nano Offline Inference
|
||||
|
||||
## Overview
|
||||
|
||||
Single-stage offline TTS pipeline using the 0.1B MOSS-TTS-Nano AR LM and MOSS-Audio-Tokenizer-Nano codec. Outputs 48 kHz mono WAV (the upstream tokenizer is stereo at 48 kHz; the wrapper mixes down to mono so it lines up with the rest of the engine's single-channel audio path).
|
||||
|
||||
> **No built-in speaker presets.** Every request needs `--prompt-audio`
|
||||
> (a reference clip). The default `--mode voice_clone` is upstream's
|
||||
> recommended workflow and is the only mode the OpenAI server exposes;
|
||||
> the offline CLI also exposes `--mode continuation` for completeness,
|
||||
> but note that upstream's continuation-with-prompt path emits very
|
||||
> short / near-silent output, so it is rarely useful in practice. See
|
||||
> upstream's `infer.py` for the full surface.
|
||||
>
|
||||
> Sample reference clips ship in the upstream repo under
|
||||
> [`assets/audio/`](https://github.com/OpenMOSS/MOSS-TTS-Nano/tree/main/assets/audio)
|
||||
> (e.g. `zh_1.wav`, `en_2.wav`, `jp_2.wav`).
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
# Fetch a sample reference clip from upstream (one-off, user-scoped cache).
|
||||
REF_DIR="${XDG_CACHE_HOME:-$HOME/.cache}/moss-tts-nano"
|
||||
mkdir -p "$REF_DIR"
|
||||
[ -s "$REF_DIR/zh_1.wav" ] || \
|
||||
curl -L -o "$REF_DIR/zh_1.wav" https://raw.githubusercontent.com/OpenMOSS/MOSS-TTS-Nano/main/assets/audio/zh_1.wav
|
||||
|
||||
python end2end.py \
|
||||
--text "你好，这是MOSS-TTS-Nano的语音合成演示。" \
|
||||
--prompt-audio "$REF_DIR/zh_1.wav"
|
||||
```
|
||||
|
||||
The first run downloads `OpenMOSS-Team/MOSS-TTS-Nano` and `OpenMOSS-Team/MOSS-Audio-Tokenizer-Nano` from Hugging Face.
|
||||
|
||||
## Usage
|
||||
|
||||
```
|
||||
python end2end.py [OPTIONS]
|
||||
|
||||
Required:
|
||||
--prompt-audio PATH Reference WAV/MP3 for voice cloning / continuation
|
||||
|
||||
Options:
|
||||
--text TEXT Text to synthesize (default: "Hello, this is MOSS-TTS-Nano speaking.")
|
||||
--prompt-text TEXT Optional. Required only with --mode continuation;
|
||||
rejected by upstream in --mode voice_clone.
|
||||
--mode MODE voice_clone (default) or continuation
|
||||
--max-new-frames N Max AR frames, default 375 (~14 s audio)
|
||||
--seed INT Random seed for reproducibility
|
||||
--audio-temperature F Audio sampling temperature (default: 0.8)
|
||||
--audio-top-k N Audio top-k sampling (default: 25)
|
||||
--audio-top-p F Audio top-p sampling (default: 0.95)
|
||||
--text-temperature F Text layer temperature (default: 1.0)
|
||||
--output-dir DIR Directory for WAV outputs (default: $XDG_CACHE_HOME/moss_tts_nano_output, falls back to ~/.cache/...)
|
||||
--deploy-config PATH Override deploy YAML (defaults to vllm_omni/deploy/moss_tts_nano.yaml)
|
||||
--stage-init-timeout INT Timeout in seconds for stage init (default: 120)
|
||||
```
|
||||
|
||||
## Examples
|
||||
|
||||
```bash
|
||||
REF_DIR="${XDG_CACHE_HOME:-$HOME/.cache}/moss-tts-nano"
|
||||
|
||||
# Chinese reference clip → Chinese synthesis (voice_clone, default)
|
||||
python end2end.py \
|
||||
--text "你好，这是 MOSS-TTS-Nano 的语音合成测试。" \
|
||||
--prompt-audio "$REF_DIR/zh_1.wav"
|
||||
|
||||
# Reproducible output
|
||||
python end2end.py \
|
||||
--text "Deterministic test." \
|
||||
--prompt-audio "$REF_DIR/en_2.wav" \
|
||||
--seed 42
|
||||
```
|
||||
|
||||
## Deploy Config
|
||||
|
||||
Runtime knobs live in `vllm_omni/deploy/moss_tts_nano.yaml` (auto-loaded;
|
||||
override with `--deploy-config PATH`). Key stage-level settings:
|
||||
|
||||
```yaml
|
||||
stages:
|
||||
- stage_id: 0
|
||||
gpu_memory_utilization: 0.3 # ~2 GB VRAM; increase for faster init
|
||||
max_num_seqs: 4 # concurrent requests
|
||||
max_model_len: 4096
|
||||
```
|
||||
|
||||
## Output Format
|
||||
|
||||
WAV files, 48 kHz, mono. The MOSS audio tokenizer is internally stereo (2-channel) at 48 kHz; the wrapper averages the two channels into mono before reaching the engine, so playback duration / pitch are correct against the WAV header's 48 kHz rate.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- **`libnvrtc.so.13: cannot open shared object file`**: torchaudio 2.10+ torchcodec backend requires NVRTC. The model patches `torchaudio.load/save` automatically at load time to fall back to soundfile.
|
||||
- **`flash_attn not installed`**: The model falls back to `sdpa` attention automatically.
|
||||
- **Empty audio**: Check that `--text` is non-empty and the model loaded successfully (look for "MOSS-TTS-Nano LM loaded" in logs).
|
||||
@@ -16,9 +16,11 @@ list of supported architectures across all modalities, see
|
||||
|---|---|---|---|---|---|---|
|
||||
| CosyVoice3 | `FunAudioLLM/Fun-CosyVoice3-0.5B-2512` | 2 (talker + code2wav) | ✓ | ✓ | — | 22.05 kHz |
|
||||
| Fish Speech S2 Pro | `fishaudio/s2-pro` | dual-AR | ✓ | ✓ | — | 44.1 kHz |
|
||||
| Ming-flash-omni-TTS | `Jonathan1909/Ming-flash-omni-2.0` | single (talker only) | — (caption-controlled) | — | style / IP / basic captions | 44.1 kHz |
|
||||
| MOSS-TTS-Nano | `OpenMOSS-Team/MOSS-TTS-Nano` | single (AR + codec) | ✓ (required) | ✓ | voice_clone, continuation | 48 kHz |
|
||||
| OmniVoice | `k2-fsa/OmniVoice` | 2 (gen + dec) | ✓ | — | voice design, language hint | 24 kHz |
|
||||
| Qwen3-TTS | `Qwen/Qwen3-TTS-12Hz-1.7B-{CustomVoice,VoiceDesign,Base}` | 2 (talker + code2wav) | ✓ (Base) | ✓ | 3 task variants | 24 kHz |
|
||||
| VoxCPM | local model dir | split | ✓ | ✓ | — | 24 kHz |
|
||||
| VoxCPM | `openbmb/VoxCPM-0.5B` | split | ✓ | ✓ | — | 24 kHz |
|
||||
| VoxCPM2 | `openbmb/VoxCPM2` | single (native AR) | ✓ | ✓ (online) | continuation | 48 kHz |
|
||||
| Voxtral TTS | `mistralai/Voxtral-4B-TTS-2603` | varies | ✓ | ✓ | voice presets | 24 kHz |
|
||||
|
||||
@@ -126,6 +128,76 @@ Streaming requires `async_chunk: true` in the stage config.
|
||||
|
||||
---
|
||||
|
||||
## Ming-flash-omni-TTS
|
||||
|
||||
Standalone talker-only deployment of Ming-flash-omni-2.0 at 44.1 kHz. Voice is controlled through caption fields (`风格` / `IP` / `语速`/`基频`/`音量`) rather than reference audio.
|
||||
|
||||
### Prerequisites
|
||||
The example calls into `vllm_omni.model_executor.models.ming_flash_omni.prompt_utils` for the default prompt and instruction builder; no extra pip install on top of the base vLLM-Omni install.
|
||||
|
||||
### Quick start
|
||||
```bash
|
||||
python examples/offline_inference/text_to_speech/ming_flash_omni_tts/end2end.py --case style
|
||||
```
|
||||
|
||||
### Cases
|
||||
```bash
|
||||
# ASMR-style whisper (caption-driven)
|
||||
python examples/offline_inference/text_to_speech/ming_flash_omni_tts/end2end.py --case style
|
||||
|
||||
# IP voice (preset character voice via caption)
|
||||
python examples/offline_inference/text_to_speech/ming_flash_omni_tts/end2end.py --case ip
|
||||
|
||||
# Basic speed/pitch/volume control
|
||||
python examples/offline_inference/text_to_speech/ming_flash_omni_tts/end2end.py --case basic
|
||||
```
|
||||
|
||||
Override the default text per case with `--text`, write to a custom path with `--output`.
|
||||
|
||||
### Notes
|
||||
- Talker-only deployment — for the multimodal Ming-flash-omni example, see [`examples/offline_inference/ming_flash_omni/`](../../ming_flash_omni/).
|
||||
- Deploy config: `vllm_omni/deploy/ming_flash_omni_tts.yaml` (single GPU, `enforce_eager`, `max_num_seqs: 1`).
|
||||
- Decode defaults from the Ming cookbook: `max_decode_steps=200`, `cfg=2.0`, `sigma=0.25`, `temperature=0.0`, `use_zero_spk_emb=True`.
|
||||
|
||||
---
|
||||
|
||||
## MOSS-TTS-Nano
|
||||
|
||||
Single-stage 0.1B AR LM + MOSS-Audio-Tokenizer-Nano codec at 48 kHz mono (mixed down from upstream stereo). ZH / EN / JA. Every request requires a reference clip via `--ref-audio`.
|
||||
|
||||
> **No built-in speaker presets.** `--ref-audio` is required on every call. Default `--mode voice_clone` matches upstream's recommended workflow; `--mode continuation` is exposed for completeness but upstream's continuation-with-prompt path emits very short / near-silent output, so it is rarely useful in practice. Sample reference clips ship in the upstream repo under [`assets/audio/`](https://github.com/OpenMOSS/MOSS-TTS-Nano/tree/main/assets/audio) (e.g. `zh_1.wav`, `en_2.wav`, `jp_2.wav`).
|
||||
|
||||
### Quick start
|
||||
```bash
|
||||
# Fetch a sample reference clip (one-off, user-scoped cache).
|
||||
REF_DIR="${XDG_CACHE_HOME:-$HOME/.cache}/moss-tts-nano"
|
||||
mkdir -p "$REF_DIR"
|
||||
[ -s "$REF_DIR/zh_1.wav" ] || \
|
||||
curl -L -o "$REF_DIR/zh_1.wav" https://raw.githubusercontent.com/OpenMOSS/MOSS-TTS-Nano/main/assets/audio/zh_1.wav
|
||||
|
||||
python examples/offline_inference/text_to_speech/moss_tts_nano/end2end.py \
|
||||
--text "你好，这是MOSS-TTS-Nano的语音合成演示。" \
|
||||
--ref-audio "$REF_DIR/zh_1.wav"
|
||||
```
|
||||
The first run downloads `OpenMOSS-Team/MOSS-TTS-Nano` and `OpenMOSS-Team/MOSS-Audio-Tokenizer-Nano` from Hugging Face.
|
||||
|
||||
### Reproducible runs
|
||||
```bash
|
||||
python examples/offline_inference/text_to_speech/moss_tts_nano/end2end.py \
|
||||
--text "Deterministic test." \
|
||||
--ref-audio "$REF_DIR/en_2.wav" \
|
||||
--seed 42
|
||||
```
|
||||
|
||||
### Notes
|
||||
- Output: 48 kHz mono WAV (the tokenizer is internally stereo at 48 kHz; the wrapper averages to mono before reaching the engine).
|
||||
- Deploy config: `vllm_omni/deploy/moss_tts_nano.yaml` (auto-loaded; override with `--deploy-config`).
|
||||
- Default `--max-new-frames 375` ≈ 14 s of audio; raise for longer outputs.
|
||||
- `--ref-text` is rejected in `voice_clone` mode and required only with `--mode continuation`.
|
||||
- Run `--help` for the full sampling-knob surface (`--audio-temperature`, `--audio-top-k`, `--audio-top-p`, `--text-temperature`).
|
||||
|
||||
---
|
||||
|
||||
## OmniVoice
|
||||
|
||||
Zero-shot multilingual TTS supporting 600+ languages, with three modes (auto / clone / design).
|
||||
|
||||
File renamed without changes.
+11
-11
@@ -9,20 +9,20 @@ the existing single-channel audio writer in vLLM-Omni stays correct).
|
||||
|
||||
MOSS-TTS-Nano upstream supports two modes (matching ``infer.py``):
|
||||
|
||||
* ``voice_clone`` (recommended): only ``--prompt-audio`` is required.
|
||||
* ``continuation``: ``--prompt-audio`` + ``--prompt-text`` together.
|
||||
* ``voice_clone`` (recommended): only ``--ref-audio`` is required.
|
||||
* ``continuation``: ``--ref-audio`` + ``--ref-text`` together.
|
||||
|
||||
Usage:
|
||||
# Voice clone (recommended): ref audio only, no transcript needed.
|
||||
python end2end.py \\
|
||||
--text "Hello!" \\
|
||||
--prompt-audio /path/to/ref.wav
|
||||
--ref-audio /path/to/ref.wav
|
||||
|
||||
# Continuation: ref audio + its transcript.
|
||||
python end2end.py \\
|
||||
--text "Hello!" \\
|
||||
--prompt-audio /path/to/ref.wav \\
|
||||
--prompt-text "Transcript of the reference clip." \\
|
||||
--ref-audio /path/to/ref.wav \\
|
||||
--ref-text "Transcript of the reference clip." \\
|
||||
--mode continuation
|
||||
|
||||
# Sample reference clips ship in the upstream repo:
|
||||
@@ -120,11 +120,11 @@ def main(args) -> None:
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
print(f"Synthesizing: {args.text!r}")
|
||||
print(f" ref_audio: {args.prompt_audio}")
|
||||
print(f" ref_audio: {args.ref_audio}")
|
||||
inputs = build_request(
|
||||
text=args.text,
|
||||
prompt_audio_path=args.prompt_audio,
|
||||
prompt_text=args.prompt_text,
|
||||
prompt_audio_path=args.ref_audio,
|
||||
prompt_text=args.ref_text,
|
||||
mode=args.mode,
|
||||
max_new_frames=args.max_new_frames,
|
||||
seed=args.seed,
|
||||
@@ -158,15 +158,15 @@ def parse_args():
|
||||
parser = FlexibleArgumentParser(description="MOSS-TTS-Nano offline inference")
|
||||
parser.add_argument("--text", default="Hello, this is MOSS-TTS-Nano speaking.", help="Text to synthesize.")
|
||||
parser.add_argument(
|
||||
"--prompt-audio",
|
||||
"--ref-audio",
|
||||
required=True,
|
||||
help="Path to reference audio for voice cloning / continuation (required).",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--prompt-text",
|
||||
"--ref-text",
|
||||
default=None,
|
||||
help=(
|
||||
"Optional transcript of --prompt-audio. Required (and only meaningful) "
|
||||
"Optional transcript of --ref-audio. Required (and only meaningful) "
|
||||
"in --mode continuation; rejected by upstream in --mode voice_clone."
|
||||
),
|
||||
)
|
||||
@@ -1,58 +0,0 @@
|
||||
# Voxtral TTS Offline Inference
|
||||
|
||||
`end2end.py` runs Voxtral TTS end-to-end offline inference using vLLM. It supports both blocking (`Omni`) and streaming (`AsyncOmni`) generation, batched prompts with configurable concurrency, and voice selection via preset name or reference audio file.
|
||||
|
||||
When `mistral_common` has `SpeechRequest` support, prompt token IDs are built via `encode_speech_request`. Otherwise, the script falls back to manual token construction.
|
||||
|
||||
## Usage Examples
|
||||
|
||||
|
||||
```bash
|
||||
# Basic single-prompt with cheerful_female voice preset
|
||||
python3 examples/offline_inference/voxtral_tts/end2end.py \
|
||||
--write-audio --voice cheerful_female \
|
||||
--model mistralai/Voxtral-4B-TTS-2603 \
|
||||
--text "That eerie silence after the first storm was just the calm before another round of chaos, wasn't it?"
|
||||
|
||||
# 32 replicate prompts with cheerful_female voice preset
|
||||
python3 examples/offline_inference/voxtral_tts/end2end.py \
|
||||
--num-prompts 32 --write-audio --voice cheerful_female \
|
||||
--model mistralai/Voxtral-4B-TTS-2603 \
|
||||
--text "That eerie silence after the first storm was just the calm before another round of chaos, wasn't it?"
|
||||
|
||||
# Streaming with neutral_female voice preset
|
||||
python3 examples/offline_inference/voxtral_tts/end2end.py \
|
||||
--streaming --write-audio --voice neutral_female \
|
||||
--model mistralai/Voxtral-4B-TTS-2603 \
|
||||
--text "That eerie silence after the first storm was just the calm before another round of chaos, wasn't it?"
|
||||
|
||||
# 32 prompts, 8 concurrent requests per wave, streaming with neutral_female voice
|
||||
python3 examples/offline_inference/voxtral_tts/end2end.py \
|
||||
--num-prompts 32 --concurrency 8 --streaming --write-audio --voice neutral_female \
|
||||
--model mistralai/Voxtral-4B-TTS-2603 \
|
||||
--text "That eerie silence after the first storm was just the calm before another round of chaos, wasn't it?"
|
||||
|
||||
# Short debug prompt with reference audio
|
||||
python3 examples/offline_inference/voxtral_tts/end2end.py \
|
||||
--write-audio \
|
||||
--model mistralai/Voxtral-4B-TTS-2603 \
|
||||
--text "This is a test message." \
|
||||
--audio-path path/to/reference_audio.wav
|
||||
```
|
||||
|
||||
## Arguments
|
||||
|
||||
| Argument | Description |
|
||||
|---|---|
|
||||
| `--model PATH` | HuggingFace repo ID or local directory path (default: `mistralai/Voxtral-4B-TTS-2603`) |
|
||||
| `--text TEXT` | Text to synthesize (default: `"This is a test message."`) |
|
||||
| `--audio-path PATH` | Path to reference audio file for voice cloning |
|
||||
| `--output-dir DIR` | Directory to write output WAV files (default: `output_audio`) |
|
||||
| `--deploy-config PATH` | Override the deploy config path. If unset, auto-loads `vllm_omni/deploy/voxtral_tts.yaml` from the HF `model_type`. |
|
||||
| `--num-prompts N` | Number of replicate prompts to run for measuring performance (default: 1) |
|
||||
| `--streaming` | Use streaming generation via `AsyncOmni` (default: blocking `Omni`) |
|
||||
| `--concurrency N` | Max concurrent requests per wave (must be used with `--streaming`, must evenly divide `--num-prompts`) |
|
||||
| `--voice NAME` | Voice preset to use instead of reference audio file. Check Huggingface `mistralai/Voxtral-4B-TTS-2603` to get the list of available voices |
|
||||
| `--write-audio` | Write generated audio to WAV files |
|
||||
| `--profiling-mode` | Enable profiling mode (reduces max tokens to 50) |
|
||||
| `--log-stats` | Enable detailed statistics logging |
|
||||
Reference in new issue
Block a user