[Docs] Consolidate moss_tts_nano + ming_flash_omni_tts into TTS hub (#3358)

Signed-off-by: Yueqian Lin <linyueqian@outlook.com>

examples/offline_inference/text_to_speech/README.md

@@ -16,9 +16,11 @@ list of supported architectures across all modalities, see
 |---|---|---|---|---|---|---|
 | CosyVoice3 | `FunAudioLLM/Fun-CosyVoice3-0.5B-2512` | 2 (talker + code2wav) | ✓ | ✓ | — | 22.05 kHz |
 | Fish Speech S2 Pro | `fishaudio/s2-pro` | dual-AR | ✓ | ✓ | — | 44.1 kHz |
+| Ming-flash-omni-TTS | `Jonathan1909/Ming-flash-omni-2.0` | single (talker only) | — (caption-controlled) | — | style / IP / basic captions | 44.1 kHz |
+| MOSS-TTS-Nano | `OpenMOSS-Team/MOSS-TTS-Nano` | single (AR + codec) | ✓ (required) | ✓ | voice_clone, continuation | 48 kHz |
 | OmniVoice | `k2-fsa/OmniVoice` | 2 (gen + dec) | ✓ | — | voice design, language hint | 24 kHz |
 | Qwen3-TTS | `Qwen/Qwen3-TTS-12Hz-1.7B-{CustomVoice,VoiceDesign,Base}` | 2 (talker + code2wav) | ✓ (Base) | ✓ | 3 task variants | 24 kHz |
-| VoxCPM | local model dir | split | ✓ | ✓ | — | 24 kHz |
+| VoxCPM | `openbmb/VoxCPM-0.5B` | split | ✓ | ✓ | — | 24 kHz |
 | VoxCPM2 | `openbmb/VoxCPM2` | single (native AR) | ✓ | ✓ (online) | continuation | 48 kHz |
 | Voxtral TTS | `mistralai/Voxtral-4B-TTS-2603` | varies | ✓ | ✓ | voice presets | 24 kHz |
 
@@ -126,6 +128,76 @@ Streaming requires `async_chunk: true` in the stage config.
 
 ---
 
+## Ming-flash-omni-TTS
+
+Standalone talker-only deployment of Ming-flash-omni-2.0 at 44.1 kHz. Voice is controlled through caption fields (`风格` / `IP` / `语速`/`基频`/`音量`) rather than reference audio.
+
+### Prerequisites
+The example calls into `vllm_omni.model_executor.models.ming_flash_omni.prompt_utils` for the default prompt and instruction builder; no extra pip install on top of the base vLLM-Omni install.
+
+### Quick start
+```bash
+python examples/offline_inference/text_to_speech/ming_flash_omni_tts/end2end.py --case style
+```
+
+### Cases
+```bash
+# ASMR-style whisper (caption-driven)
+python examples/offline_inference/text_to_speech/ming_flash_omni_tts/end2end.py --case style
+
+# IP voice (preset character voice via caption)
+python examples/offline_inference/text_to_speech/ming_flash_omni_tts/end2end.py --case ip
+
+# Basic speed/pitch/volume control
+python examples/offline_inference/text_to_speech/ming_flash_omni_tts/end2end.py --case basic
+```
+
+Override the default text per case with `--text`, write to a custom path with `--output`.
+
+### Notes
+- Talker-only deployment — for the multimodal Ming-flash-omni example, see [`examples/offline_inference/ming_flash_omni/`](../../ming_flash_omni/).
+- Deploy config: `vllm_omni/deploy/ming_flash_omni_tts.yaml` (single GPU, `enforce_eager`, `max_num_seqs: 1`).
+- Decode defaults from the Ming cookbook: `max_decode_steps=200`, `cfg=2.0`, `sigma=0.25`, `temperature=0.0`, `use_zero_spk_emb=True`.
+
+---
+
+## MOSS-TTS-Nano
+
+Single-stage 0.1B AR LM + MOSS-Audio-Tokenizer-Nano codec at 48 kHz mono (mixed down from upstream stereo). ZH / EN / JA. Every request requires a reference clip via `--ref-audio`.
+
+> **No built-in speaker presets.** `--ref-audio` is required on every call. Default `--mode voice_clone` matches upstream's recommended workflow; `--mode continuation` is exposed for completeness but upstream's continuation-with-prompt path emits very short / near-silent output, so it is rarely useful in practice. Sample reference clips ship in the upstream repo under [`assets/audio/`](https://github.com/OpenMOSS/MOSS-TTS-Nano/tree/main/assets/audio) (e.g. `zh_1.wav`, `en_2.wav`, `jp_2.wav`).
+
+### Quick start
+```bash
+# Fetch a sample reference clip (one-off, user-scoped cache).
+REF_DIR="${XDG_CACHE_HOME:-$HOME/.cache}/moss-tts-nano"
+mkdir -p "$REF_DIR"
+[ -s "$REF_DIR/zh_1.wav" ] || \
+    curl -L -o "$REF_DIR/zh_1.wav" https://raw.githubusercontent.com/OpenMOSS/MOSS-TTS-Nano/main/assets/audio/zh_1.wav
+
+python examples/offline_inference/text_to_speech/moss_tts_nano/end2end.py \
+    --text "你好，这是MOSS-TTS-Nano的语音合成演示。" \
+    --ref-audio "$REF_DIR/zh_1.wav"
+```
+The first run downloads `OpenMOSS-Team/MOSS-TTS-Nano` and `OpenMOSS-Team/MOSS-Audio-Tokenizer-Nano` from Hugging Face.
+
+### Reproducible runs
+```bash
+python examples/offline_inference/text_to_speech/moss_tts_nano/end2end.py \
+    --text "Deterministic test." \
+    --ref-audio "$REF_DIR/en_2.wav" \
+    --seed 42
+```
+
+### Notes
+- Output: 48 kHz mono WAV (the tokenizer is internally stereo at 48 kHz; the wrapper averages to mono before reaching the engine).
+- Deploy config: `vllm_omni/deploy/moss_tts_nano.yaml` (auto-loaded; override with `--deploy-config`).
+- Default `--max-new-frames 375` ≈ 14 s of audio; raise for longer outputs.
+- `--ref-text` is rejected in `voice_clone` mode and required only with `--mode continuation`.
+- Run `--help` for the full sampling-knob surface (`--audio-temperature`, `--audio-top-k`, `--audio-top-p`, `--text-temperature`).
+
+---
+
 ## OmniVoice
 
 Zero-shot multilingual TTS supporting 600+ languages, with three modes (auto / clone / design).

examples/offline_inference/text_to_speech/ming_flash_omni_tts/end2end.py

@@ -0,0 +1,124 @@
+"""Offline e2e example for Ming-flash-omni-2.0 standalone talker (TTS)."""
+
+import os
+from typing import Any
+
+import soundfile as sf
+import torch
+from vllm.utils.argparse_utils import FlexibleArgumentParser
+
+os.environ["VLLM_WORKER_MULTIPROC_METHOD"] = "spawn"
+
+from vllm_omni.engine.arg_utils import nullify_stage_engine_defaults
+from vllm_omni.entrypoints.omni import Omni
+from vllm_omni.inputs.data import OmniTokensPrompt
+from vllm_omni.model_executor.models.ming_flash_omni.prompt_utils import (
+    DEFAULT_PROMPT,
+    create_instruction,
+)
+
+MODEL_NAME = "Jonathan1909/Ming-flash-omni-2.0"
+
+
+def get_messages(case: str, text_override: str | None) -> dict[str, Any]:
+    if case == "style":
+        text = text_override or "我会一直在这里陪着你，直到你慢慢、慢慢地沉入那个最温柔的梦里……好吗？"
+        instruction = create_instruction(
+            {
+                "风格": "这是一种ASMR耳语，属于一种旨在引发特殊感官体验的创意风格。这个女性使用轻柔的普通话进行耳语，声音气音成分重。音量极低，紧贴麦克风，语速极慢，旨在制造触发听者颅内快感的声学刺激。",
+            }
+        )
+        return {
+            "prompt": DEFAULT_PROMPT,
+            "text": text,
+            "instruction": instruction,
+            "use_zero_spk_emb": True,
+        }
+    if case == "ip":
+        text = text_override or "这款产品的名字，叫变态坑爹牛肉丸。"
+        return {
+            "prompt": DEFAULT_PROMPT,
+            "text": text,
+            "instruction": create_instruction({"IP": "灵小甄"}),
+            "use_zero_spk_emb": True,
+        }
+    if case == "basic":
+        text = text_override or "我们当迎着阳光辛勤耕作，去摘取，去制作，去品尝，去馈赠。"
+        return {
+            "prompt": DEFAULT_PROMPT,
+            "text": text,
+            "instruction": create_instruction({"语速": "快速", "基频": "中", "音量": "中"}),
+            "use_zero_spk_emb": True,
+        }
+    raise ValueError(f"Unknown case: {case}")
+
+
+def save_audio(mm: dict[str, Any], output_path: str) -> None:
+    if not mm or "audio" not in mm:
+        raise RuntimeError("No audio found in model output")
+    audio = mm["audio"]
+    sr_raw = mm.get("sr", 44100)
+    if isinstance(sr_raw, torch.Tensor):
+        sample_rate = int(sr_raw.item())
+    else:
+        sample_rate = int(sr_raw)
+    waveform = audio.squeeze().float().cpu().numpy()
+    sf.write(output_path, waveform, sample_rate)
+    print(f"Saved {output_path} ({len(waveform) / sample_rate:.2f}s, {sample_rate}Hz)")
+
+
+def parse_args():
+    parser = FlexibleArgumentParser(description="Ming-flash-omni standalone talker offline e2e example")
+    parser.add_argument("--model", type=str, default=MODEL_NAME, help="Model name or local path.")
+    parser.add_argument(
+        "--deploy-config",
+        type=str,
+        default="vllm_omni/deploy/ming_flash_omni_tts.yaml",
+        help="Path to a custom deploy YAML for the TTS deployment. ",
+    )
+    parser.add_argument(
+        "--case",
+        type=str,
+        default="style",
+        choices=["style", "ip", "basic"],
+        help="Example case.",
+    )
+    parser.add_argument("--text", type=str, default=None, help="Override default text for the selected case.")
+    parser.add_argument("--output", type=str, default=None, help="Output wav path.")
+    parser.add_argument("--log-stats", action="store_true", default=False, help="Enable stats logging.")
+    parser.add_argument("--init-timeout", type=int, default=600, help="Engine init timeout in seconds.")
+    parser.add_argument("--stage-init-timeout", type=int, default=300, help="Single stage init timeout in seconds.")
+
+    nullify_stage_engine_defaults(parser)
+    return parser.parse_args()
+
+
+def main():
+    args = parse_args()
+
+    omni = Omni.from_cli_args(args, model=args.model)
+
+    messages = get_messages(args.case, args.text)
+    decode_args = {
+        # Standalone TTS deployment
+        "ming_task": "instruct",
+        "max_decode_steps": 200,
+        "cfg": 2.0,
+        "sigma": 0.25,
+        "temperature": 0.0,
+    }
+    req = OmniTokensPrompt(
+        prompt_token_ids=[0],
+        additional_information={**messages, **decode_args},
+    )
+
+    outputs = omni.generate(req)
+    mm = outputs[0].outputs[0].multimodal_output
+
+    output_path = args.output or f"output_{args.case}.wav"
+    save_audio(mm, output_path)
+    omni.close()
+
+
+if __name__ == "__main__":
+    main()

examples/offline_inference/text_to_speech/moss_tts_nano/end2end.py

@@ -0,0 +1,198 @@
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
+"""Offline inference example for MOSS-TTS-Nano via vLLM-Omni.
+
+Single-stage pipeline: the 0.1B AR LM and MOSS-Audio-Tokenizer-Nano codec
+both run inside one generation stage. Output is 48 kHz mono WAV (the
+upstream tokenizer is stereo at 48 kHz; the wrapper mixes down to mono so
+the existing single-channel audio writer in vLLM-Omni stays correct).
+
+MOSS-TTS-Nano upstream supports two modes (matching ``infer.py``):
+
+* ``voice_clone`` (recommended): only ``--ref-audio`` is required.
+* ``continuation``: ``--ref-audio`` + ``--ref-text`` together.
+
+Usage:
+  # Voice clone (recommended): ref audio only, no transcript needed.
+  python end2end.py \\
+    --text "Hello!" \\
+    --ref-audio /path/to/ref.wav
+
+  # Continuation: ref audio + its transcript.
+  python end2end.py \\
+    --text "Hello!" \\
+    --ref-audio /path/to/ref.wav \\
+    --ref-text "Transcript of the reference clip." \\
+    --mode continuation
+
+  # Sample reference clips ship in the upstream repo:
+  #   https://github.com/OpenMOSS/MOSS-TTS-Nano/tree/main/assets/audio
+  # e.g. zh_1.wav (Chinese), en_2.wav (English), jp_2.wav (Japanese).
+"""
+
+from __future__ import annotations
+
+import os
+from pathlib import Path
+
+import soundfile as sf
+import torch
+from vllm import SamplingParams
+from vllm.utils.argparse_utils import FlexibleArgumentParser
+
+# Prevent multiprocessing from re-importing CUDA in the wrong context.
+os.environ.setdefault("VLLM_WORKER_MULTIPROC_METHOD", "spawn")
+
+from vllm_omni import Omni  # noqa: E402
+
+MODEL = "OpenMOSS-Team/MOSS-TTS-Nano"
+
+
+def build_request(
+    text: str,
+    prompt_audio_path: str,
+    prompt_text: str | None = None,
+    mode: str = "voice_clone",
+    max_new_frames: int = 375,
+    seed: int | None = None,
+    audio_temperature: float = 0.8,
+    audio_top_k: int = 25,
+    audio_top_p: float = 0.95,
+    text_temperature: float = 1.0,
+) -> dict:
+    """Build an Omni request payload for MOSS-TTS-Nano.
+
+    Upstream's ``_resolve_inference_mode`` forbids ``prompt_text`` in
+    ``voice_clone`` mode and requires it in ``continuation`` mode (with
+    ``prompt_audio_path``), so we only forward ``prompt_text`` when it is
+    actually supplied.
+    """
+    additional: dict = {
+        "text": [text],
+        "mode": [mode],
+        "prompt_audio_path": [str(prompt_audio_path)],
+        "max_new_frames": [max_new_frames],
+        "audio_temperature": [audio_temperature],
+        "audio_top_k": [audio_top_k],
+        "audio_top_p": [audio_top_p],
+        "text_temperature": [text_temperature],
+    }
+    if prompt_text is not None and prompt_text.strip():
+        additional["prompt_text"] = [prompt_text]
+    if seed is not None:
+        additional["seed"] = [seed]
+
+    return {
+        "prompt": "<|im_start|>assistant\n",  # minimal placeholder prompt
+        "additional_information": additional,
+    }
+
+
+def save_audio(waveform: torch.Tensor, path: str, sample_rate: int = 48000) -> None:
+    """Write the model's mono waveform to ``path`` at ``sample_rate``.
+
+    The model wrapper mixes the upstream tokenizer's stereo output down to
+    mono before reaching the engine, so ``waveform`` is always 1-D here —
+    no extra interleave/reshape is needed.
+    """
+    audio_np = waveform.float().numpy()
+    sf.write(path, audio_np, sample_rate)
+    print(f"  Saved {path} ({audio_np.shape}, {sample_rate} Hz)")
+
+
+def main(args) -> None:
+    omni = Omni(
+        model=MODEL,
+        deploy_config=args.deploy_config,
+        stage_init_timeout=args.stage_init_timeout,
+    )
+
+    sampling_params = SamplingParams(
+        temperature=1.0,
+        top_p=1.0,
+        top_k=50,
+        max_tokens=4096,
+        seed=args.seed if args.seed is not None else 42,
+        detokenize=False,
+    )
+
+    output_dir = Path(args.output_dir)
+    output_dir.mkdir(parents=True, exist_ok=True)
+
+    print(f"Synthesizing: {args.text!r}")
+    print(f"  ref_audio: {args.ref_audio}")
+    inputs = build_request(
+        text=args.text,
+        prompt_audio_path=args.ref_audio,
+        prompt_text=args.ref_text,
+        mode=args.mode,
+        max_new_frames=args.max_new_frames,
+        seed=args.seed,
+        audio_temperature=args.audio_temperature,
+        audio_top_k=args.audio_top_k,
+        audio_top_p=args.audio_top_p,
+        text_temperature=args.text_temperature,
+    )
+    params_list = sampling_params
+
+    for stage_outputs in omni.generate(inputs, params_list):
+        for i, req_output in enumerate(stage_outputs.request_output):
+            for j, out in enumerate(req_output.outputs):
+                mm = out.multimodal_output
+                if mm is None:
+                    print(f"  [req {i}] No audio output.")
+                    continue
+                audio = mm.get("audio")
+                sr_tensor = mm.get("sr")
+                if audio is None:
+                    print(f"  [req {i}] No waveform in multimodal_output.")
+                    continue
+                sr = int(sr_tensor.item()) if sr_tensor is not None else 48000
+                out_path = str(output_dir / f"output_{i}_{j}.wav")
+                save_audio(audio.cpu(), out_path, sr)
+
+    print("Done.")
+
+
+def parse_args():
+    parser = FlexibleArgumentParser(description="MOSS-TTS-Nano offline inference")
+    parser.add_argument("--text", default="Hello, this is MOSS-TTS-Nano speaking.", help="Text to synthesize.")
+    parser.add_argument(
+        "--ref-audio",
+        required=True,
+        help="Path to reference audio for voice cloning / continuation (required).",
+    )
+    parser.add_argument(
+        "--ref-text",
+        default=None,
+        help=(
+            "Optional transcript of --ref-audio. Required (and only meaningful) "
+            "in --mode continuation; rejected by upstream in --mode voice_clone."
+        ),
+    )
+    parser.add_argument("--mode", default="voice_clone", choices=["voice_clone", "continuation"])
+    parser.add_argument("--max-new-frames", type=int, default=375, help="Max AR frames (~14s at default).")
+    parser.add_argument("--seed", type=int, default=None, help="Random seed.")
+    parser.add_argument("--audio-temperature", type=float, default=0.8)
+    parser.add_argument("--audio-top-k", type=int, default=25)
+    parser.add_argument("--audio-top-p", type=float, default=0.95)
+    parser.add_argument("--text-temperature", type=float, default=1.0)
+    parser.add_argument(
+        "--output-dir",
+        default=os.path.join(
+            os.environ.get("XDG_CACHE_HOME", os.path.join(os.path.expanduser("~"), ".cache")),
+            "moss_tts_nano_output",
+        ),
+        help="Directory for WAV outputs (default: ~/.cache/moss_tts_nano_output).",
+    )
+    parser.add_argument(
+        "--deploy-config",
+        default=None,
+        help="Path to a deploy YAML; leave unset to auto-load vllm_omni/deploy/moss_tts_nano.yaml.",
+    )
+    parser.add_argument("--stage-init-timeout", type=int, default=120)
+    return parser.parse_args()
+
+
+if __name__ == "__main__":
+    main(parse_args())