[Model] Add Ming-omni-tts dense 0.5B pipeline (#2906)

Signed-off-by: akshatvishu <akshatnayak197@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>

examples/offline_inference/text_to_speech/README.md

@@ -17,6 +17,7 @@ list of supported architectures across all modalities, see
 | CosyVoice3 | `FunAudioLLM/Fun-CosyVoice3-0.5B-2512` | 2 (talker + code2wav) | ✓ | ✓ | — | 24 kHz |
 | Fish Speech S2 Pro | `fishaudio/s2-pro` | dual-AR | ✓ | ✓ | — | 44.1 kHz |
 | GLM-TTS | `zai-org/GLM-TTS` | 2 (AR + DiT) | ✓ (required) | ✓ | — | 24 kHz |
+| Ming-omni-tts | `inclusionAI/Ming-omni-tts-0.5B` | 2 (AR + audio VAE) | ✓ | ✓ | style / IP / dialect / TTA / podcast | 44.1 kHz |
 | Ming-flash-omni-TTS | `Jonathan1909/Ming-flash-omni-2.0` | single (talker only) | — (caption-controlled) | — | style / IP / basic captions | 44.1 kHz |
 | MOSS-TTS-Nano | `OpenMOSS-Team/MOSS-TTS-Nano` | single (AR + codec) | ✓ (required) | ✓ | voice_clone, continuation | 48 kHz |
 | OmniVoice | `k2-fsa/OmniVoice` | 2 (gen + dec) | ✓ | — | voice design, language hint | 24 kHz |
@@ -159,6 +160,46 @@ Streaming requires `async_chunk: true` in the stage config.
 
 ---
 
+## Ming-omni-tts
+
+Dense 0.5B two-stage TTS pipeline (`AR + flow` + audio VAE) at 44.1 kHz. The example covers style, IP voice, music-only generation, text-to-audio events, emotion, dialect, zero-shot cloning, podcast, speech+BGM, and speech+environment-sound cases.
+
+### Quick start
+```bash
+python examples/offline_inference/text_to_speech/ming_tts/end2end.py \
+    --case style \
+    --deploy-config vllm_omni/deploy/ming_tts.yaml \
+    --enforce-eager
+```
+
+### Voice cloning
+```bash
+python examples/offline_inference/text_to_speech/ming_tts/end2end.py \
+    --case zero_shot \
+    --ref-audio /path/to/reference.wav \
+    --ref-text "在此奉劝大家别乱打美白针。" \
+    --deploy-config vllm_omni/deploy/ming_tts.yaml \
+    --enforce-eager
+```
+
+### Streaming
+```bash
+python examples/offline_inference/text_to_speech/ming_tts/end2end.py \
+    --case basic \
+    --ref-audio /path/to/reference.wav \
+    --streaming \
+    --deploy-config vllm_omni/deploy/ming_tts.yaml \
+    --enforce-eager
+```
+
+### Notes
+- `style`, `ip`, `bgm`, and `tta` do not require reference audio.
+- Reference-audio cases use `--ref-audio`; `zero_shot` also requires `--ref-text`.
+- `podcast` uses multiple references via `--ref-audio-paths`.
+- Full case details live in [`ming_tts/README.md`](ming_tts/README.md).
+
+---
+
 ## Ming-flash-omni-TTS
 
 Standalone talker-only deployment of Ming-flash-omni-2.0 at 44.1 kHz. Voice is controlled through caption fields (`风格` / `IP` / `语速`/`基频`/`音量`) rather than reference audio.

examples/offline_inference/text_to_speech/ming_tts/README.md

@@ -0,0 +1,248 @@
+# Ming-omni-tts Offline Inference
+
+`end2end.py` runs Ming dense 0.5B end to end with vLLM-Omni. It uses the in-repo Ming prompt assembly helper directly, so the example request shape matches the real integration instead of a simplified wrapper.
+
+## Files
+
+| File | Purpose |
+|---|---|
+| `end2end.py` | Driver: CLI, case loading, prompt construction, orchestration (~150 lines) |
+| `cases.yaml` | All 11 built-in case definitions (prompt, text, instruction, ref-audio flags, flow controls) |
+| `runner.py` | Engine management and audio output (streaming + blocking paths) |
+
+## Model Overview
+
+Ming dense 0.5B is exposed here as a two-stage offline pipeline:
+
+- **Stage 0**: Qwen2-based AR generation with Ming prompt formatting and inline flow controls
+- **Stage 1**: audio VAE decode to mono 44.1 kHz waveform
+
+`config_ming_tts.py` adapts the checkpoint's HuggingFace config fields
+(LLM, DiT, aggregator, AudioVAE, token ids). `vllm_omni/deploy/ming_tts.yaml`
+selects the vLLM-Omni pipeline and stage runtime topology, including
+connectors, async chunking, memory limits, and sampling defaults.
+
+The example supports both:
+
+- **Blocking eager** via `vllm_omni/deploy/ming_tts.yaml`
+- **Async chunk eager** via `vllm_omni/deploy/ming_tts.yaml` (default `async_chunk: true`)
+
+## Setup
+
+Install vLLM-Omni with the platform requirements for your accelerator:
+
+```bash
+uv pip install -e .
+```
+
+The Ming offline example does not require a separate upstream Ming package.
+Reference-audio cases use the repo dependencies for audio loading,
+resampling, and CampPlus speaker extraction, including `soundfile`,
+`torchaudio`, and `onnxruntime-rocm` in the documented ROCm environment.
+
+The tested ROCm environment is summarized in the
+[Ming recipe](https://github.com/vllm-project/vllm-omni/blob/main/recipes/inclusionAI/Ming-omni-tts-0.5B.md).
+
+## Supported Cases
+
+These cases cover the upstream dense 0.5B cookbook surface that maps cleanly onto the current vLLM-Omni example:
+
+- `style`: zero-speaker style-conditioned speech
+- `ip`: zero-speaker IP voice generation
+- `bgm`: music-only generation
+- `tta`: text-to-audio event generation with FlowLoss controls
+- `emotion`: reference-audio speech with emotion control
+- `basic`: reference-audio speech with speed / pitch / volume control
+- `dialect`: reference-audio speech with dialect control
+- `zero_shot`: reference-audio cloning with explicit transcript
+- `podcast`: multi-reference dialogue generation with automatic speaker embedding extraction
+- `speech_bgm`: speech with background music conditioning
+- `speech_sound`: speech with environmental sound conditioning
+
+## Quick Start
+
+Run the zero-speaker style example:
+
+```bash
+python examples/offline_inference/text_to_speech/ming_tts/end2end.py \
+    --case style \
+    --deploy-config vllm_omni/deploy/ming_tts.yaml \
+    --enforce-eager
+```
+
+Run zero-shot cloning with a transcript:
+
+```bash
+python examples/offline_inference/text_to_speech/ming_tts/end2end.py \
+    --case zero_shot \
+    --ref-audio /path/to/10002287-00000094.wav \
+    --ref-text "在此奉劝大家别乱打美白针。" \
+    --deploy-config vllm_omni/deploy/ming_tts.yaml \
+    --enforce-eager
+```
+
+Run emotion-controlled speech:
+
+```bash
+python examples/offline_inference/text_to_speech/ming_tts/end2end.py \
+    --case emotion \
+    --ref-audio /path/to/emotion_prompt.wav \
+    --deploy-config vllm_omni/deploy/ming_tts.yaml \
+    --enforce-eager
+```
+
+Run podcast generation with two reference clips:
+
+```bash
+python examples/offline_inference/text_to_speech/ming_tts/end2end.py \
+    --case podcast \
+    --ref-audio-paths /path/to/CTS-CN-F2F-2019-11-11-423-012-A.wav /path/to/CTS-CN-F2F-2019-11-11-423-012-B.wav \
+    --deploy-config vllm_omni/deploy/ming_tts.yaml \
+    --enforce-eager
+```
+
+The script automatically extracts one 192-d speaker embedding per reference WAV using the Ming model's `campplus.onnx`.
+
+If you already have precomputed multi-speaker embeddings, you can override extraction with:
+
+```bash
+--speaker-embedding /path/to/podcast_speaker_embeddings.json
+```
+
+where the JSON is a list of speaker embeddings, one 192-d vector per speaker.
+
+Run text-to-audio event generation:
+
+```bash
+python examples/offline_inference/text_to_speech/ming_tts/end2end.py \
+    --case tta \
+    --deploy-config vllm_omni/deploy/ming_tts.yaml \
+    --enforce-eager
+```
+
+Use async_chunk streaming:
+
+```bash
+python examples/offline_inference/text_to_speech/ming_tts/end2end.py \
+    --case basic \
+    --ref-audio /path/to/10002287-00000095.wav \
+    --streaming \
+    --deploy-config vllm_omni/deploy/ming_tts.yaml \
+    --enforce-eager
+```
+
+`--streaming` uses `AsyncOmni` and the async_chunk stage config. It currently
+supports one prompt per process invocation; use blocking mode for
+`--num-prompts > 1`.
+
+Collect runtime stats and a manifest:
+
+```bash
+python examples/offline_inference/text_to_speech/ming_tts/end2end.py \
+    --case style \
+    --deploy-config vllm_omni/deploy/ming_tts.yaml \
+    --enforce-eager \
+    --enable-stats \
+    --stats-log-file output_audio/ming_style_pipeline.log \
+    --metadata-json output_audio/ming_style_manifest.json
+```
+
+## Reference Fixtures
+
+The upstream Ming cookbook uses these public audio fixtures from `inclusionAI/Ming-omni-tts/data/wavs`:
+
+- `10002287-00000094.wav` for zero-shot cloning
+- `10002287-00000095.wav` for `basic`
+- `emotion_prompt.wav` for `emotion`
+- `yue_prompt.wav` for `dialect`
+- `00000309-00000300.wav` for `speech_bgm` and `speech_sound`
+- `CTS-CN-F2F-2019-11-11-423-012-A.wav` and `CTS-CN-F2F-2019-11-11-423-012-B.wav` for `podcast`
+
+## Validation Matrix
+
+The repo-facing example is intended to cover the same dense TTS workflows used
+by the local Ming validation script:
+
+| Case | Blocking `deploy/ming_tts.yaml` | Async chunk `deploy/ming_tts.yaml` | Extra inputs |
+|---|---:|---:|---|
+| `style` | Yes | Optional smoke test | none |
+| `ip` | Yes | Optional smoke test | none |
+| `bgm` | Yes | Optional smoke test | none |
+| `tta` | Yes | Optional smoke test | none |
+| `emotion` | Yes | Yes | `--ref-audio emotion_prompt.wav` |
+| `basic` | Yes | Yes | `--ref-audio 10002287-00000095.wav` |
+| `dialect` | Yes | Yes | `--ref-audio yue_prompt.wav` |
+| `zero_shot` | Yes | Yes | `--ref-audio 10002287-00000094.wav --ref-text ...` |
+| `podcast` | Yes | Yes | two `--ref-audio-paths` |
+| `speech_bgm` | Yes | Yes | `--ref-audio 00000309-00000300.wav` |
+| `speech_sound` | Yes | Yes | `--ref-audio 00000309-00000300.wav` |
+
+## Validated Outputs
+
+The following measurements are retained from an earlier L4 CUDA validation;
+they are not ROCm benchmark results. Default async_chunk matched blocking
+output frame counts and Stage-1 patch counts for every case:
+
+| Case | Blocking frames / patches / sec | Async chunk frames / patches / sec |
+|---|---:|---:|
+| `style` | 409248 / 29 / 9.28 | 409248 / 29 / 9.28 |
+| `ip` | 183456 / 13 / 4.16 | 183456 / 13 / 4.16 |
+| `bgm` | 1326528 / 94 / 30.08 | 1326528 / 94 / 30.08 |
+| `tta` | 465696 / 33 / 10.56 | 465696 / 33 / 10.56 |
+| `emotion` | 324576 / 23 / 7.36 | 324576 / 23 / 7.36 |
+| `basic` | 211680 / 15 / 4.80 | 211680 / 15 / 4.80 |
+| `dialect` | 239904 / 17 / 5.44 | 239904 / 17 / 5.44 |
+| `zero_shot` | 409248 / 29 / 9.28 | 409248 / 29 / 9.28 |
+| `podcast` | 437472 / 31 / 9.92 | 437472 / 31 / 9.92 |
+| `speech_bgm` | 296352 / 21 / 6.72 | 296352 / 21 / 6.72 |
+| `speech_sound` | 352800 / 25 / 8.00 | 352800 / 25 / 8.00 |
+
+## Key Arguments
+
+| Argument | Description |
+|---|---|
+| `--model` | Hugging Face repo or local Ming checkpoint path |
+| `--deploy-config` | Deploy config YAML. Use `vllm_omni/deploy/ming_tts.yaml` |
+| `--case` | Built-in demo case |
+| `--ref-audio` | Single reference wav path for cloning-style cases |
+| `--ref-audio-paths` | Multiple reference wav paths, used by `podcast` |
+| `--ref-text` | Reference transcript. Required for `zero_shot` |
+| `--instructions` | Free-form Ming instruction string |
+| `--instruction-json` | Structured Ming instruction JSON |
+| `--speaker-embedding` | JSON file containing a 192-d speaker embedding |
+| `--extract-speaker-embeddings` | Force CampPlus speaker extraction from the provided reference audio paths |
+| `--max-decode-steps` | Override `ming_max_decode_steps` |
+| `--num-prompts` | Repeat the same case N times. Outputs are indexed when `N > 1` |
+| `--streaming` | Use `AsyncOmni` and async_chunk transport |
+| `--enforce-eager` | Recommended for Ming dense; non-eager is out of scope |
+| `--enable-stats` / `--log-stats` | Enable vLLM-Omni per-request stats logging |
+| `--stats-log-file` | Optional path for the stats log |
+| `--metadata-json` | Optional path for the run manifest JSON |
+| `--stage-init-timeout` | Per-stage initialization timeout in seconds |
+| `--init-timeout` | Total initialization timeout in seconds |
+| `--batch-timeout` | Batch timeout in seconds |
+| `--worker-backend` | `multi_process` or `ray` |
+| `--ray-address` | Ray cluster address when using `--worker-backend ray` |
+
+## Output
+
+- The script writes one mono 44.1 kHz WAV file per run
+- Default output directory: `output_audio/`
+- Default filename: `ming_<case>.wav`
+- When `--num-prompts > 1`, outputs are indexed as `ming_<case>_00000.wav`, `..._00001.wav`, etc.
+- When stats are enabled, the script can also write:
+  - a stats log file such as `ming_style_pipeline.log`
+  - a manifest JSON with per-output metadata, stage durations, peak memory info,
+    and streaming client latency metrics when `--streaming` is used
+
+## Notes
+
+- `style` and `ip` are zero-speaker paths and do not require a reference clip
+- `emotion`, `basic`, `dialect`, `speech_bgm`, and `speech_sound` require one reference clip
+- `zero_shot` requires both `--ref-audio` and `--ref-text`
+- `podcast` requires at least two reference clips via `--ref-audio-paths`
+- `podcast` automatically extracts one speaker embedding per reference clip
+- `--speaker-embedding` may contain either one 192-d vector or a list of 192-d vectors
+- `--enforce-eager` was used for the validated runs
+- The earlier L4 validation used SDPA for the Ming audio VAE instead of
+  FlashAttention2, which is the preferred default when available.

examples/offline_inference/text_to_speech/ming_tts/cases.yaml

@@ -0,0 +1,110 @@
+style:
+  prompt: "Please generate speech based on the following description.\n"
+  text: "我会一直在这里陪着你，直到你慢慢、慢慢地沉入那个最温柔的梦里……好吗？"
+  instruction:
+    风格: >-
+      这是一种ASMR耳语，属于一种旨在引发特殊感官体验的创意风格。这个女性使用轻柔的普通话进行耳语，声音气音成分重。音量极低，紧贴麦克风，语速极慢，旨在制造触发听者颅内快感的声学刺激。
+  use_zero_spk_emb: true
+  max_decode_steps: 200
+
+ip:
+  prompt: "Please generate speech based on the following description.\n"
+  text: "这款产品的名字，叫变态坑爹牛肉丸。"
+  instruction:
+    IP: "灵小甄"
+  use_zero_spk_emb: true
+  max_decode_steps: 200
+
+bgm:
+  prompt: "Please generate music based on the following description.\n"
+  text: " Genre: 电子舞曲. Mood: 自信 / 坚定. Instrument: 架子鼓. Theme: 节日. Duration: 30s."
+  instruction: null
+  use_zero_spk_emb: false
+  max_decode_steps: 400
+
+tta:
+  prompt: "Please generate audio events based on given text.\n"
+  text: "Thunder and a gentle rain"
+  instruction: null
+  use_zero_spk_emb: false
+  max_decode_steps: 200
+  cfg: 4.5
+  sigma: 0.3
+  temperature: 2.5
+
+emotion:
+  prompt: "Please generate speech based on the following description.\n"
+  text: "我竟然抢到了陈奕迅的演唱会门票！太棒了！终于可以现场听一听他的歌声了！"
+  instruction:
+    情感: "高兴"
+  requires_ref_audio: true
+  auto_extract_speaker_embeddings: true
+  max_decode_steps: 200
+
+basic:
+  prompt: "Please generate speech based on the following description.\n"
+  text: "简单地说，这相当于惠普把消费领域市场拱手相让了。"
+  instruction:
+    语速: "快速"
+    基频: "中"
+    音量: "中"
+  requires_ref_audio: true
+  auto_extract_speaker_embeddings: true
+  max_decode_steps: 200
+
+dialect:
+  prompt: "Please generate speech based on the following description.\n"
+  text: "我觉得社会企业同个人都有责任"
+  instruction:
+    方言: "广粤话"
+  requires_ref_audio: true
+  auto_extract_speaker_embeddings: true
+  max_decode_steps: 200
+
+zero_shot:
+  prompt: "Please generate speech based on the following description.\n"
+  text: "我们的愿景是构建未来服务业的数字化基础设施，为世界带来更多微小而美好的改变。"
+  instruction: null
+  requires_ref_audio: true
+  requires_ref_text: true
+  auto_extract_speaker_embeddings: true
+  max_decode_steps: 200
+
+podcast:
+  prompt: "Please generate speech based on the following description.\n"
+  text: " speaker_1:你可以说一下，就大概说一下，可能虽然我也不知道，我看过那部电影没有。\n speaker_2:就是那个叫什么，变相一节课的嘛。\n speaker_1:嗯。\n speaker_2:一部搞笑的电影。\n speaker_1:一部搞笑的。\n"
+  instruction: null
+  prompt_text: " speaker_1:并且我们还要进行每个月还要考核 笔试的话还要进行笔试，做个，当服务员还要去笔试了\n speaker_2:对啊，这真的很奇怪，就是 单纯的因，单纯自己工资不高，只是因为可能人家那个店比较出名一点，就对你苛刻要求\n"
+  requires_ref_audio_count: 2
+  auto_extract_speaker_embeddings: true
+  max_decode_steps: 200
+
+speech_bgm:
+  prompt: "Please generate speech based on the following description.\n"
+  text: "此次业绩下滑原因，可归结为企业停止服务某些品牌，而带来的负面影响。"
+  instruction:
+    BGM:
+      Genre: "当代古典音乐."
+      Mood: "温暖 / 友善."
+      Instrument: "电吉他"
+      Theme: "节日."
+      SNR: 10.0
+      ENV: null
+  requires_ref_audio: true
+  auto_extract_speaker_embeddings: true
+  max_decode_steps: 200
+
+speech_sound:
+  prompt: "Please generate speech based on the following description.\n"
+  text: "此次业绩下滑原因，可归结为企业停止服务某些品牌，而带来的负面影响。"
+  instruction:
+    BGM:
+      ENV: "Birds chirping"
+      SNR: 10.0
+      Genre: null
+      Mood: null
+      Instrument: null
+      Theme: null
+  requires_ref_audio: true
+  auto_extract_speaker_embeddings: true
+  max_decode_steps: 200

examples/offline_inference/text_to_speech/ming_tts/end2end.py

@@ -0,0 +1,200 @@
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
+"""Offline inference demo for Ming-omni-tts via vLLM Omni."""
+
+import json
+import os
+from pathlib import Path
+
+import soundfile as sf
+import torch
+import torchaudio
+import yaml
+from transformers import AutoTokenizer
+from vllm.utils.argparse_utils import FlexibleArgumentParser
+
+from vllm_omni.model_executor.models.ming_tts.config_ming_tts import (
+    KEY_CFG,
+    KEY_MAX_DECODE_STEPS,
+    KEY_SIGMA,
+    KEY_TEMPERATURE,
+    SAMPLE_RATE,
+)
+from vllm_omni.model_executor.models.ming_tts.prompt_assembly import build_ming_dense_prompt
+from vllm_omni.model_executor.models.ming_tts.speaker_extractor import MingSpeakerEmbeddingExtractor
+
+try:
+    from .runner import (
+        build_manifest,
+        build_sampling_params,
+        resolve_metadata_json,
+        resolve_stats_log_file,
+        run_generation,
+    )
+except ImportError:
+    from runner import (
+        build_manifest,
+        build_sampling_params,
+        resolve_metadata_json,
+        resolve_stats_log_file,
+        run_generation,
+    )
+
+_DEFAULT_MODEL = "inclusionAI/Ming-omni-tts-0.5B"
+_REPO_ROOT = Path(__file__).resolve().parents[4]
+_DEFAULT_DEPLOY_CONFIG = str(_REPO_ROOT / "vllm_omni/deploy/ming_tts.yaml")
+_CASES_FILE = Path(__file__).with_name("cases.yaml")
+
+CASE_DEFAULTS = yaml.safe_load(_CASES_FILE.read_text(encoding="utf-8")) or {}
+if not CASE_DEFAULTS:
+    raise RuntimeError(f"Empty or missing case definitions in {_CASES_FILE}")
+
+
+def _build_parser() -> FlexibleArgumentParser:
+    p = FlexibleArgumentParser(description="Offline Ming-omni-tts example")
+    p.add_argument("--model", default=_DEFAULT_MODEL, help="Model name or local path")
+    p.add_argument("--deploy-config", default=None, help="Deploy config YAML; auto-selected when omitted")
+    p.add_argument("--case", choices=sorted(CASE_DEFAULTS), default="style", help="Built-in demo case")
+    p.add_argument("--text", default=None, help="Override case text")
+    p.add_argument("--prompt", default=None, help="Override the system prompt prefix")
+    p.add_argument("--instructions", default=None, help="Free-form Ming instruction string")
+    p.add_argument(
+        "--instruction-json", default=None, help='Structured Ming instruction JSON, e.g. \'{"方言":"广粤话"}\''
+    )
+    p.add_argument("--ref-audio", default=None, help="Single reference audio path for cloning-style cases")
+    p.add_argument("--ref-audio-paths", nargs="+", default=None, help="Multiple reference audio paths (podcast)")
+    p.add_argument("--ref-text", default=None, help="Reference transcript; required for zero_shot")
+    p.add_argument("--speaker-embedding", default=None, help="Path to a JSON speaker embedding file")
+    p.add_argument(
+        "--extract-speaker-embeddings",
+        action="store_true",
+        help="Extract speaker embeddings from ref audio via campplus.onnx",
+    )
+    p.add_argument("--max-decode-steps", type=int, default=None, help="Override ming_max_decode_steps")
+    p.add_argument("--output-dir", default="output_audio", help="Directory for output wav files")
+    p.add_argument("--output-name", default=None, help="Output wav filename")
+    p.add_argument("--num-prompts", type=int, default=1, help="Repeat the same prompt N times")
+    p.add_argument("--streaming", action="store_true", help="Use AsyncOmni with async_chunk streaming")
+    p.add_argument("--trust-remote-code", action="store_true")
+    p.add_argument("--enforce-eager", action="store_true")
+    p.add_argument(
+        "--log-stats", "--enable-stats", dest="log_stats", action="store_true", help="Enable Omni stats logging"
+    )
+    p.add_argument("--stats-log-file", default=None, help="Path for the Omni stats log file")
+    p.add_argument("--metadata-json", default=None, help="Path for the run manifest JSON")
+    p.add_argument("--stage-init-timeout", type=int, default=300, help="Per-stage init timeout (s)")
+    p.add_argument("--init-timeout", type=int, default=600, help="Total init timeout (s)")
+    p.add_argument("--batch-timeout", type=int, default=5, help="Batch timeout (s)")
+    p.add_argument("--shm-threshold-bytes", type=int, default=65536)
+    p.add_argument("--worker-backend", default="multi_process", choices=["multi_process", "ray"])
+    p.add_argument("--ray-address", default=None, help="Ray cluster address (--worker-backend ray)")
+    return p
+
+
+def _finalize_args(args) -> None:
+    if args.instructions is not None and args.instruction_json is not None:
+        raise RuntimeError("Use either --instructions or --instruction-json, not both")
+    if args.num_prompts < 1:
+        raise RuntimeError("--num-prompts must be at least 1")
+    if args.streaming and args.num_prompts != 1:
+        raise RuntimeError("--streaming currently supports exactly one prompt")
+    if args.deploy_config is None:
+        args.deploy_config = _DEFAULT_DEPLOY_CONFIG
+
+
+def _load_waveform(path: str) -> torch.Tensor:
+    samples, sr = sf.read(path, dtype="float32")
+    wav = torch.as_tensor(samples, dtype=torch.float32)
+    if wav.ndim == 2:
+        wav = wav.mean(dim=1)
+    wav = wav.reshape(1, -1)
+    if int(sr) != SAMPLE_RATE:
+        wav = torchaudio.functional.resample(wav, int(sr), SAMPLE_RATE)
+    return wav
+
+
+def _load_speaker_embedding(path: str) -> torch.Tensor:
+    return torch.as_tensor(json.loads(Path(path).read_text(encoding="utf-8")), dtype=torch.float32)
+
+
+def _ref_audio_paths(args) -> list[str]:
+    if args.ref_audio is not None and args.ref_audio_paths is not None:
+        raise RuntimeError("Use either --ref-audio or --ref-audio-paths, not both")
+    if args.ref_audio_paths is not None:
+        return list(args.ref_audio_paths)
+    return [args.ref_audio] if args.ref_audio else []
+
+
+def _resolve_reference_inputs(args, case: dict, paths: list[str]):
+    required = int(case.get("requires_ref_audio_count", 0))
+    if required > 0 and len(paths) < required:
+        raise RuntimeError(f"Case '{args.case}' needs {required} ref audio paths via --ref-audio-paths")
+    if required <= 0 and case.get("requires_ref_audio") and not paths:
+        raise RuntimeError(f"--ref-audio required for case '{args.case}'")
+    if not paths:
+        return None
+    wavs = [_load_waveform(p) for p in paths]
+    return wavs[0] if len(wavs) == 1 else wavs
+
+
+def _resolve_speaker_embedding(args, case: dict, paths: list[str]):
+    if args.speaker_embedding:
+        return _load_speaker_embedding(args.speaker_embedding)
+    if not (case.get("auto_extract_speaker_embeddings") or args.extract_speaker_embeddings) or not paths:
+        return None
+    embs = MingSpeakerEmbeddingExtractor(args.model).extract_many(paths)
+    if not embs:
+        raise RuntimeError("Speaker extraction produced no embeddings")
+    return embs[0] if len(embs) == 1 else torch.stack(embs, dim=0)
+
+
+def _build_prompt_payload(tokenizer, args):
+    case = CASE_DEFAULTS[args.case]
+    paths = _ref_audio_paths(args)
+    prompt_text = args.ref_text if args.ref_text is not None else case.get("prompt_text")
+    if case.get("requires_ref_text") and not prompt_text:
+        raise RuntimeError(f"--ref-text required for case '{args.case}'")
+    prompt_waveform = _resolve_reference_inputs(args, case, paths) if prompt_text is not None else None
+    speaker_embedding = _resolve_speaker_embedding(args, case, paths)
+    use_zero_spk_emb = bool(case.get("use_zero_spk_emb")) and prompt_waveform is None and speaker_embedding is None
+    runtime_controls = {KEY_MAX_DECODE_STEPS: args.max_decode_steps or case["max_decode_steps"]}
+    for key, field in [(KEY_CFG, "cfg"), (KEY_SIGMA, "sigma"), (KEY_TEMPERATURE, "temperature")]:
+        if field in case:
+            runtime_controls[key] = case[field]
+    instruction = (
+        json.loads(args.instruction_json) if args.instruction_json else (args.instructions or case.get("instruction"))
+    )
+    return build_ming_dense_prompt(
+        tokenizer,
+        prompt=args.prompt or case["prompt"],
+        text=args.text or case["text"],
+        runtime_controls=runtime_controls,
+        instruction=instruction,
+        prompt_text=prompt_text,
+        prompt_waveform=prompt_waveform,
+        speaker_embedding=speaker_embedding,
+        use_zero_spk_emb=use_zero_spk_emb,
+    )
+
+
+def main():
+    args = _build_parser().parse_args()
+    _finalize_args(args)
+    tokenizer = AutoTokenizer.from_pretrained(args.model, trust_remote_code=False)
+    prompt_payload = _build_prompt_payload(tokenizer, args)
+    case = CASE_DEFAULTS[args.case]
+    sampling_params_list = build_sampling_params(args.max_decode_steps or case["max_decode_steps"])
+    output_dir = Path(args.output_dir)
+    output_dir.mkdir(parents=True, exist_ok=True)
+    stats_log_file = resolve_stats_log_file(args)
+    summaries = run_generation(args, prompt_payload, sampling_params_list, output_dir, stats_log_file)
+    metadata_json = resolve_metadata_json(args)
+    manifest = build_manifest(args, prompt_payload, stats_log_file, summaries)
+    if metadata_json is not None:
+        Path(metadata_json).write_text(json.dumps(manifest, ensure_ascii=False, indent=2), encoding="utf-8")
+        print(f"Saved run manifest to {metadata_json}")
+
+
+if __name__ == "__main__":
+    os.environ.setdefault("VLLM_WORKER_MULTIPROC_METHOD", "spawn")
+    main()

examples/offline_inference/text_to_speech/ming_tts/runner.py

@@ -0,0 +1,277 @@
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
+
+import asyncio
+import json
+import time
+import uuid
+import wave
+from pathlib import Path
+
+import torch
+from vllm import SamplingParams
+
+from vllm_omni import AsyncOmni, Omni
+from vllm_omni.model_executor.models.ming_tts.config_ming_tts import (
+    KEY_SPEAKER_EMBEDDING,
+    SAMPLE_RATE,
+    TEXT_EOS_TOKEN_ID,
+)
+
+
+def coerce_audio_tensor(audio, *, async_chunk: bool) -> torch.Tensor:
+    if isinstance(audio, list):
+        parts = []
+        for item in audio:
+            tensor = torch.as_tensor(item, dtype=torch.float32).reshape(-1)
+            if tensor.numel() > 0:
+                parts.append(tensor)
+        if not parts:
+            return torch.zeros((0,), dtype=torch.float32)
+        return torch.cat(parts, dim=0)
+
+    return torch.as_tensor(audio, dtype=torch.float32).reshape(-1)
+
+
+def resolve_sr(sr) -> int:
+    if isinstance(sr, list):
+        sr = sr[-1]
+    if hasattr(sr, "item"):
+        return int(sr.item())
+    return int(sr)
+
+
+def extract_sample_rate(multimodal_output: dict) -> int:
+    sr = multimodal_output.get("sr")
+    if sr is None:
+        raise RuntimeError("Expected multimodal_output['sr']")
+    return resolve_sr(sr)
+
+
+def write_wav(path: str, audio: torch.Tensor, sample_rate: int) -> None:
+    audio = audio.clamp(-1.0, 1.0)
+    pcm16 = (audio * 32767.0).round().to(torch.int16).cpu().numpy()
+    with wave.open(path, "wb") as wav_file:
+        wav_file.setnchannels(1)
+        wav_file.setsampwidth(2)
+        wav_file.setframerate(int(sample_rate))
+        wav_file.writeframes(pcm16.tobytes())
+
+
+def request_index(request_id: str | None, fallback: int) -> int:
+    try:
+        return int(request_id)
+    except (TypeError, ValueError):
+        if isinstance(request_id, str):
+            head = request_id.split("_", 1)[0]
+            if head.isdigit():
+                return int(head)
+    return fallback
+
+
+def audio_summary(audio: torch.Tensor, sample_rate: int) -> dict:
+    waveform = audio.detach().cpu().reshape(-1).to(torch.float32)
+    return {
+        "sample_rate": int(sample_rate),
+        "num_samples": int(waveform.numel()),
+        "duration_seconds": float(waveform.numel()) / float(sample_rate),
+        "max_abs_amplitude": float(waveform.abs().max().item()) if waveform.numel() > 0 else 0.0,
+    }
+
+
+def resolve_output_name(output_name: str | None, case: str, index: int, total: int) -> str:
+    if total == 1:
+        return output_name or f"ming_{case}.wav"
+    base = Path(output_name or f"ming_{case}.wav")
+    return f"{base.stem}_{index:05d}{base.suffix or '.wav'}"
+
+
+def resolve_stats_log_file(args) -> str | None:
+    if not args.log_stats:
+        return None
+    if args.stats_log_file:
+        return args.stats_log_file
+    base = Path(args.output_name or f"ming_{args.case}.wav").stem
+    return str(Path(args.output_dir) / f"{base}_pipeline.log")
+
+
+def resolve_metadata_json(args) -> str | None:
+    if args.metadata_json:
+        return args.metadata_json
+    if args.log_stats:
+        base = Path(args.output_name or f"ming_{args.case}.wav").stem
+        return str(Path(args.output_dir) / f"{base}_manifest.json")
+    return None
+
+
+def build_manifest(args, prompt_payload, stats_log_file: str | None, outputs: list[dict]) -> dict:
+    additional_information = {}
+    if isinstance(prompt_payload, dict):
+        additional_information = dict(prompt_payload.get("additional_information", {}))
+    return {
+        "model": args.model,
+        "case": args.case,
+        "streaming": bool(args.streaming),
+        "deploy_config": args.deploy_config,
+        "enforce_eager": bool(args.enforce_eager),
+        "num_prompts": int(args.num_prompts),
+        "log_stats": bool(args.log_stats),
+        "stats_log_file": stats_log_file,
+        "prompt_text": additional_information.get("prompt_text"),
+        "instruction": additional_information.get("instruction"),
+        "speaker_embedding_shape": (
+            list(additional_information[KEY_SPEAKER_EMBEDDING].shape)
+            if KEY_SPEAKER_EMBEDDING in additional_information
+            and hasattr(additional_information[KEY_SPEAKER_EMBEDDING], "shape")
+            else None
+        ),
+        "outputs": outputs,
+        "generated_at_unix": time.time(),
+    }
+
+
+def build_engine_kwargs(args, stats_log_file: str | None) -> dict:
+    kwargs = {
+        "model": args.model,
+        "deploy_config": args.deploy_config,
+        "enforce_eager": args.enforce_eager,
+        "trust_remote_code": args.trust_remote_code,
+        "log_stats": args.log_stats,
+        "stage_init_timeout": args.stage_init_timeout,
+        "init_timeout": args.init_timeout,
+        "batch_timeout": args.batch_timeout,
+        "shm_threshold_bytes": args.shm_threshold_bytes,
+        "worker_backend": args.worker_backend,
+    }
+    if stats_log_file is not None:
+        kwargs["log_file"] = stats_log_file
+    if args.ray_address is not None:
+        kwargs["ray_address"] = args.ray_address
+    return kwargs
+
+
+def build_sampling_params(max_decode_steps: int) -> list[SamplingParams]:
+    return [
+        SamplingParams(
+            temperature=0.0,
+            max_tokens=max_decode_steps + 1,
+            stop_token_ids=[int(TEXT_EOS_TOKEN_ID)],
+        ),
+        SamplingParams(temperature=0.0, max_tokens=1),
+    ]
+
+
+async def run_streaming(args, prompt_payload, sampling_params_list, output_dir: Path, stats_log_file: str | None):
+    engine = AsyncOmni(**build_engine_kwargs(args, stats_log_file))
+    try:
+        all_audio_chunks = []
+        accumulated_samples = 0
+        chunk_idx = 0
+        start_time = time.time()
+        chunk_times = []
+        ttfp_seconds = None
+        final_stage_output = None
+        async for stage_output in engine.generate(
+            prompt=prompt_payload,
+            request_id=str(uuid.uuid4()),
+            sampling_params_list=sampling_params_list,
+        ):
+            final_stage_output = stage_output
+            multimodal_output = stage_output.multimodal_output or {}
+            audio = multimodal_output.get("audio")
+            if audio is None:
+                continue
+
+            finished = stage_output.finished
+            if isinstance(audio, torch.Tensor):
+                if finished:
+                    audio_chunk = audio[accumulated_samples:].float().detach().cpu()
+                else:
+                    audio_chunk = audio.float().detach().cpu()
+            elif isinstance(audio, list):
+                audio_chunk = torch.as_tensor(audio[chunk_idx], dtype=torch.float32).reshape(-1).cpu()
+            else:
+                audio_chunk = torch.as_tensor(audio, dtype=torch.float32).reshape(-1).cpu()
+
+            accumulated_samples += int(audio_chunk.numel())
+            chunk_idx += 1
+            if audio_chunk.numel() > 0:
+                now = time.time()
+                if ttfp_seconds is None:
+                    ttfp_seconds = now - start_time
+                chunk_times.append(now)
+                all_audio_chunks.append(audio_chunk)
+
+        if not all_audio_chunks:
+            raise RuntimeError("Streaming Ming example produced no audio chunks")
+
+        waveform = torch.cat(all_audio_chunks, dim=0)
+        output_name = resolve_output_name(args.output_name, args.case, 0, 1)
+        output_path = str(output_dir / output_name)
+        write_wav(output_path, waveform, SAMPLE_RATE)
+        summary = {
+            "request_id": getattr(final_stage_output, "request_id", None),
+            "stage_id": getattr(final_stage_output, "stage_id", None),
+            "output_path": output_path,
+            "stage_durations": getattr(final_stage_output, "stage_durations", {}),
+            "peak_memory_mb": getattr(final_stage_output, "peak_memory_mb", 0.0),
+            "ttfp_seconds": ttfp_seconds,
+            "mean_inter_chunk_seconds": (
+                sum(t1 - t0 for t0, t1 in zip(chunk_times, chunk_times[1:])) / (len(chunk_times) - 1)
+                if len(chunk_times) > 1
+                else None
+            ),
+        }
+        summary.update(audio_summary(waveform, SAMPLE_RATE))
+        print(f"Saved streaming output to {output_path}")
+        print(json.dumps(summary, ensure_ascii=False, indent=2))
+        return [summary]
+    finally:
+        engine.shutdown()
+
+
+def run_non_streaming(args, prompt_payload, sampling_params_list, output_dir: Path, stats_log_file: str | None):
+    engine = Omni(**build_engine_kwargs(args, stats_log_file))
+    try:
+        outputs = engine.generate(
+            prompts=[prompt_payload for _ in range(args.num_prompts)],
+            sampling_params_list=sampling_params_list,
+            py_generator=False,
+        )
+        summaries = []
+        for fallback_index, output in enumerate(outputs):
+            if output.final_output_type != "audio":
+                continue
+            multimodal_output = output.multimodal_output or {}
+            waveform = coerce_audio_tensor(multimodal_output.get("audio"), async_chunk=False)
+            sample_rate = extract_sample_rate(multimodal_output)
+            output_name = resolve_output_name(
+                args.output_name,
+                args.case,
+                request_index(output.request_id, fallback_index),
+                args.num_prompts,
+            )
+            output_path = str(output_dir / output_name)
+            write_wav(output_path, waveform, sample_rate)
+            summary = {
+                "request_id": output.request_id,
+                "stage_id": output.stage_id,
+                "output_path": output_path,
+                "stage_durations": output.stage_durations,
+                "peak_memory_mb": output.peak_memory_mb,
+            }
+            summary.update(audio_summary(waveform, sample_rate))
+            summaries.append(summary)
+            print(f"Saved output to {output_path}")
+            print(json.dumps(summary, ensure_ascii=False, indent=2))
+        if not summaries:
+            raise RuntimeError("Non-streaming Ming example produced no audio outputs")
+        return summaries
+    finally:
+        engine.close()
+
+
+def run_generation(args, prompt_payload, sampling_params_list, output_dir: Path, stats_log_file: str | None):
+    if args.streaming:
+        return asyncio.run(run_streaming(args, prompt_payload, sampling_params_list, output_dir, stats_log_file))
+    return run_non_streaming(args, prompt_payload, sampling_params_list, output_dir, stats_log_file)

examples/online_serving/text_to_speech/README.md

@@ -16,6 +16,7 @@ For the full list of supported architectures across all modalities, see
 |---|---|---|---|---|---|
 | Fish Speech S2 Pro | `fishaudio/s2-pro` | ✓ (`ref_audio`+`ref_text`) | ✓ (PCM stream) | — | ✓ |
 | GLM-TTS | `zai-org/GLM-TTS` | ✓ (`ref_audio`+`ref_text`, required) | ✓ (PCM stream) | — | ✓ |
+| Ming-omni-tts | `inclusionAI/Ming-omni-tts-0.5B` | ✓ (`ref_audio` / `speaker_embedding`) | ✓ (PCM stream) | IP labels + structured `instructions` | — |
 | Ming-flash-omni-TTS | `Jonathan1909/Ming-flash-omni-2.0` | — (caption-controlled) | — | caption fields (`instructions`) | — |
 | MOSS-TTS-Nano | `OpenMOSS-Team/MOSS-TTS-Nano` | ✓ (`ref_audio` required) | ✓ (PCM stream) | — | ✓ |
 | OmniVoice | `k2-fsa/OmniVoice` | ✓ | — | — | — |
@@ -202,6 +203,51 @@ python fish_speech/gradio_demo.py --api-base http://localhost:8091  # if server
 
 ---
 
+## Ming-omni-tts
+
+Dense 0.5B two-stage TTS served through `/v1/audio/speech`. Ming uses the standard speech endpoint plus structured controls in `instructions`, `voice`, `language`, `ref_audio`, `ref_text`, and `speaker_embedding`.
+
+### Launch
+```bash
+bash examples/online_serving/text_to_speech/ming_tts/run_server.sh
+```
+Equivalent manual command:
+```bash
+vllm-omni serve inclusionAI/Ming-omni-tts-0.5B \
+    --deploy-config vllm_omni/deploy/ming_tts.yaml \
+    --host 0.0.0.0 --port 8091 \
+    --enforce-eager --omni
+```
+
+### Sending requests
+```bash
+python examples/online_serving/text_to_speech/ming_tts/openai_speech_client.py \
+    --text "你好，这是 Ming 在线语音合成测试。"
+```
+
+Structured dialect control:
+```bash
+python examples/online_serving/text_to_speech/ming_tts/openai_speech_client.py \
+    --text "我觉得社会企业同个人都有责任" \
+    --instruction-json '{"方言":"广粤话"}' \
+    --ref-audio /path/to/yue_prompt.wav
+```
+
+Zero-shot cloning:
+```bash
+python examples/online_serving/text_to_speech/ming_tts/openai_speech_client.py \
+    --text "我们的愿景是构建未来服务业的数字化基础设施，为世界带来更多微小而美好的改变。" \
+    --ref-audio /path/to/10002287-00000094.wav \
+    --ref-text "在此奉劝大家别乱打美白针。"
+```
+
+### Notes
+- `run_curl.sh` keeps a small sanity subset; use the Ming README for the broader request cookbook.
+- Online serving is speech-shaped today; music-only `bgm` and text-to-audio `tta` remain offline examples.
+- Full request details live in [`ming_tts/README.md`](ming_tts/README.md).
+
+---
+
 ## Ming-flash-omni-TTS
 
 Standalone talker-only deployment of Ming-flash-omni-2.0. Voice is controlled through caption text passed via `instructions`.

examples/online_serving/text_to_speech/ming_tts/README.md

@@ -0,0 +1,113 @@
+# Ming-omni-tts Online Serving
+
+Serve the dense `inclusionAI/Ming-omni-tts-0.5B` two-stage TTS model through
+the OpenAI-compatible `/v1/audio/speech` endpoint.
+
+## Start Server
+
+```bash
+vllm-omni serve inclusionAI/Ming-omni-tts-0.5B \
+    --deploy-config vllm_omni/deploy/ming_tts.yaml \
+    --omni \
+    --port 8091 \
+    --enforce-eager
+```
+
+Or:
+
+```bash
+cd examples/online_serving/text_to_speech/ming_tts
+./run_server.sh
+```
+
+The tested ROCm environment is summarized in the
+[Ming recipe](https://github.com/vllm-project/vllm-omni/blob/main/recipes/inclusionAI/Ming-omni-tts-0.5B.md).
+
+## Send Requests
+
+The Python client targets `http://localhost:8091/v1` with `api_key=EMPTY`; it
+does not call OpenAI's hosted API.
+
+```bash
+python openai_speech_client.py \
+    --text "你好，这是 Ming 在线语音合成测试。" \
+    --max-new-tokens 200
+```
+
+Style or dialect controls can be plain text or Ming JSON. The upstream
+dialect example also uses `yue_prompt.wav` for speaker conditioning:
+
+```bash
+python openai_speech_client.py \
+    --text "我觉得社会企业同个人都有责任" \
+    --instruction-json '{"方言":"广粤话"}' \
+    --ref-audio /path/to/yue_prompt.wav \
+    --max-new-tokens 200
+```
+
+When `--ref-audio` is supplied without `--ref-text`, the server extracts the
+Ming speaker embedding, matching upstream `use_spk_emb=True`, without using the
+audio as a zero-shot prompt.
+
+Reference-audio cloning:
+
+```bash
+python openai_speech_client.py \
+    --text "我们的愿景是构建未来服务业的数字化基础设施，为世界带来更多微小而美好的改变。" \
+    --ref-audio /path/to/10002287-00000094.wav \
+    --ref-text "在此奉劝大家别乱打美白针。" \
+    --max-new-tokens 200
+```
+
+Podcast-style multi-speaker prompt:
+
+```bash
+python openai_speech_client.py \
+    --text " speaker_1:你可以说一下，就大概说一下，可能虽然我也不知道，我看过那部电影没有。
+ speaker_2:就是那个叫什么，变相一节课的嘛。
+ speaker_1:嗯。
+ speaker_2:一部搞笑的电影。
+ speaker_1:一部搞笑的。" \
+    --ref-audio /path/to/CTS-CN-F2F-2019-11-11-423-012-A.wav \
+    --ref-audio /path/to/CTS-CN-F2F-2019-11-11-423-012-B.wav \
+    --ref-text " speaker_1:并且我们还要进行每个月还要考核 笔试的话还要进行笔试，做个，当服务员还要去笔试了
+ speaker_2:对啊，这真的很奇怪，就是 单纯的因，单纯自己工资不高，只是因为可能人家那个店比较出名一点，就对你苛刻要求"
+```
+
+Streaming PCM:
+
+```bash
+python openai_speech_client.py \
+    --text "你好，这是流式输出测试。" \
+    --stream \
+    --output ming_output.pcm
+```
+
+`run_curl.sh` keeps small smoke checks:
+
+```bash
+./run_curl.sh basic
+REF_AUDIO=/path/to/reference.wav REF_TEXT="在此奉劝大家别乱打美白针。" ./run_curl.sh zero_shot
+./run_curl.sh stream
+```
+
+## Request Fields
+
+| Field | Ming meaning |
+|-------|--------------|
+| `input` | target text |
+| `instructions` | plain style text, or JSON object for structured Ming controls |
+| `voice` | Ming IP voice label unless it resolves to an uploaded speaker |
+| `language` | Ming `方言` control |
+| `ref_audio` | speaker reference; with `ref_text`, also supplies the prompt waveform |
+| `ref_text` | transcript enabling zero-shot or podcast prompt-latent conditioning |
+| `speaker_embedding` | 192-d Ming speaker embedding |
+| `max_new_tokens` | Ming `max_decode_steps` |
+
+## Notes
+
+- `ref_audio` accepts local paths through the client, remote URLs, `file://`,
+  or `data:` URLs.
+- Non-streaming responses return WAV bytes; streaming responses return PCM.
+- Music-only `bgm` generation is offline-only until the API exposes Ming
+  prompt-mode selection.

examples/online_serving/text_to_speech/ming_tts/openai_speech_client.py

@@ -0,0 +1,225 @@
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
+"""OpenAI-compatible client for Ming-omni-tts via /v1/audio/speech.
+
+Examples:
+    python openai_speech_client.py --text "你好，世界"
+    python openai_speech_client.py --text "我觉得社会企业同个人都有责任" \
+        --instruction-json '{"方言":"广粤话"}' --ref-audio yue_prompt.wav --max-new-tokens 200
+    python openai_speech_client.py \
+        --text "我们的愿景是构建未来服务业的数字化基础设施，为世界带来更多微小而美好的改变。" \
+        --ref-audio 10002287-00000094.wav --ref-text "在此奉劝大家别乱打美白针。" \
+        --max-new-tokens 200
+    python openai_speech_client.py --text "speaker_1:你好。 speaker_2:你好。" \
+        --ref-audio speaker_1.wav --ref-audio speaker_2.wav --ref-text "speaker_1:你好。 speaker_2:你好。"
+    python openai_speech_client.py --text "你好，这是流式输出测试。" \
+        --stream --output ming_output.pcm
+"""
+
+import argparse
+import base64
+import json
+import os
+
+import httpx
+
+DEFAULT_API_BASE = "http://localhost:8091"
+DEFAULT_API_KEY = "EMPTY"
+DEFAULT_MODEL = "inclusionAI/Ming-omni-tts-0.5B"
+EXPECTED_SPEAKER_EMBEDDING_DIM = 192
+
+
+def encode_audio_to_base64(audio_path: str) -> str:
+    """Encode a local audio file to a base64 data URL."""
+    if not os.path.exists(audio_path):
+        raise FileNotFoundError(f"Audio file not found: {audio_path}")
+    ext = audio_path.lower().rsplit(".", 1)[-1]
+    mime_map = {
+        "wav": "audio/wav",
+        "mp3": "audio/mpeg",
+        "flac": "audio/flac",
+        "ogg": "audio/ogg",
+        "aac": "audio/aac",
+    }
+    mime_type = mime_map.get(ext, "audio/wav")
+    with open(audio_path, "rb") as f:
+        audio_b64 = base64.b64encode(f.read()).decode("utf-8")
+    return f"data:{mime_type};base64,{audio_b64}"
+
+
+def load_speaker_embedding(path: str) -> list[float]:
+    """Load and validate a 192-d Ming speaker embedding JSON file."""
+    with open(path, encoding="utf-8") as f:
+        data = json.load(f)
+
+    if not isinstance(data, list):
+        raise ValueError("speaker_embedding file must contain a JSON list")
+    if len(data) != EXPECTED_SPEAKER_EMBEDDING_DIM:
+        raise ValueError(
+            f"Ming dense speaker_embedding must have {EXPECTED_SPEAKER_EMBEDDING_DIM} values, got {len(data)}"
+        )
+
+    values = []
+    for index, value in enumerate(data):
+        try:
+            values.append(float(value))
+        except (TypeError, ValueError) as exc:
+            raise ValueError(f"speaker_embedding[{index}] must be a number, got {value!r}") from exc
+    return values
+
+
+def build_instruction_payload(args) -> str | None:
+    """Return a string payload for the API `instructions` field."""
+    if args.instructions and args.instruction_json:
+        raise ValueError("Use either --instructions or --instruction-json, not both")
+    if args.instruction_json:
+        parsed = json.loads(args.instruction_json)
+        return json.dumps(parsed, ensure_ascii=False)
+    return args.instructions
+
+
+def validate_args(args) -> None:
+    """Fail fast on invalid combinations before hitting the server."""
+    if args.ref_text and not args.ref_audio:
+        raise ValueError("--ref-audio is required when --ref-text is provided")
+    if args.speaker_embedding and args.ref_audio and len(args.ref_audio) > 1:
+        raise ValueError("--speaker-embedding cannot be combined with multiple --ref-audio values")
+
+
+def run_tts(args) -> None:
+    """Generate speech via the OpenAI-compatible /v1/audio/speech API."""
+    validate_args(args)
+
+    payload = {
+        "model": args.model,
+        "input": args.text,
+        "response_format": args.response_format,
+    }
+
+    if args.voice:
+        payload["voice"] = args.voice
+    if args.task_type:
+        payload["task_type"] = args.task_type
+    if args.dialect:
+        payload["language"] = args.dialect
+
+    instructions = build_instruction_payload(args)
+    if instructions:
+        payload["instructions"] = instructions
+
+    if args.ref_audio:
+        ref_audio = []
+        for audio in args.ref_audio:
+            if audio.startswith(("http://", "https://", "data:")):
+                ref_audio.append(audio)
+            else:
+                ref_audio.append(encode_audio_to_base64(audio))
+        payload["ref_audio"] = ref_audio[0] if len(ref_audio) == 1 else ref_audio
+    if args.ref_text:
+        payload["ref_text"] = args.ref_text
+    if args.speaker_embedding:
+        payload["speaker_embedding"] = load_speaker_embedding(args.speaker_embedding)
+    if args.max_new_tokens:
+        payload["max_new_tokens"] = args.max_new_tokens
+    if args.stream:
+        payload["stream"] = True
+        payload["response_format"] = "pcm"
+
+    api_url = f"{args.api_base}/v1/audio/speech"
+    headers = {
+        "Content-Type": "application/json",
+        "Authorization": f"Bearer {args.api_key}",
+    }
+
+    print(f"Model: {args.model}")
+    print(f"Text: {args.text}")
+    print(f"Payload keys: {sorted(payload)}")
+
+    if args.stream:
+        output_path = args.output or "ming_output.pcm"
+        with httpx.Client(timeout=300.0) as client:
+            with client.stream("POST", api_url, json=payload, headers=headers) as response:
+                if response.status_code != 200:
+                    print(f"Error: {response.status_code}")
+                    print(response.read().decode())
+                    return
+                with open(output_path, "wb") as f:
+                    for chunk in response.iter_bytes():
+                        f.write(chunk)
+        print(f"Streamed PCM audio to: {output_path}")
+        return
+
+    with httpx.Client(timeout=300.0) as client:
+        response = client.post(api_url, json=payload, headers=headers)
+
+    if response.status_code != 200:
+        print(f"Error: {response.status_code}")
+        print(response.text)
+        return
+
+    try:
+        text = response.content.decode("utf-8")
+        if text.startswith('{"error"'):
+            print(f"Error: {text}")
+            return
+    except UnicodeDecodeError:
+        pass
+
+    output_path = args.output or "ming_output.wav"
+    with open(output_path, "wb") as f:
+        f.write(response.content)
+    print(f"Audio saved to: {output_path}")
+
+
+def main():
+    parser = argparse.ArgumentParser(description="OpenAI-compatible client for Ming-omni-tts via /v1/audio/speech")
+    parser.add_argument("--api-base", default=DEFAULT_API_BASE, help="API base URL")
+    parser.add_argument("--api-key", default=DEFAULT_API_KEY, help="API key")
+    parser.add_argument("--model", "-m", default=DEFAULT_MODEL, help="Model name or path")
+    parser.add_argument("--text", required=True, help="Text to synthesize")
+    parser.add_argument(
+        "--task-type",
+        default=None,
+        choices=["CustomVoice", "VoiceDesign", "Base"],
+        help="Optional compatibility task type. Ming accepts the same field but primarily uses prompt metadata.",
+    )
+    parser.add_argument(
+        "--voice",
+        default=None,
+        help="Maps to Ming `IP` when using built-in character voices, or to an uploaded voice sample name",
+    )
+    parser.add_argument("--dialect", default=None, help="Maps to Ming `方言`")
+    parser.add_argument("--instructions", default=None, help="Free-form Ming instruction string")
+    parser.add_argument(
+        "--instruction-json",
+        default=None,
+        help='Structured Ming instruction JSON, for example \'{"情感":"高兴"}\'',
+    )
+    parser.add_argument(
+        "--ref-audio",
+        action="append",
+        default=None,
+        help="Reference audio path, URL, or data URL. Repeat for podcast-style multi-speaker prompts.",
+    )
+    parser.add_argument("--ref-text", default=None, help="Reference transcript for cloning")
+    parser.add_argument(
+        "--speaker-embedding", default=None, help="Path to a JSON file containing a 192-d speaker embedding"
+    )
+    parser.add_argument("--max-new-tokens", type=int, default=None, help="Override ming_max_decode_steps")
+    parser.add_argument("--stream", action="store_true", help="Enable streaming PCM output")
+    parser.add_argument(
+        "--response-format",
+        default="wav",
+        choices=["wav", "mp3", "flac", "pcm", "aac", "opus"],
+        help="Audio format when not streaming",
+    )
+    parser.add_argument("--output", "-o", default=None, help="Output file path")
+    args = parser.parse_args()
+    try:
+        run_tts(args)
+    except (FileNotFoundError, ValueError, json.JSONDecodeError) as exc:
+        raise SystemExit(f"Error: {exc}") from exc
+
+
+if __name__ == "__main__":
+    main()

examples/online_serving/text_to_speech/ming_tts/run_curl.sh

@@ -0,0 +1,76 @@
+#!/bin/bash
+set -euo pipefail
+
+MODE="${1:-basic}"
+HOST="${HOST:-localhost}"
+PORT="${PORT:-8091}"
+MODEL="${MODEL:-inclusionAI/Ming-omni-tts-0.5B}"
+API_URL="http://${HOST}:${PORT}/v1/audio/speech"
+TEXT="${TEXT:-你好，这是 Ming 在线语音合成测试。}"
+OUTPUT="${OUTPUT:-ming_output.wav}"
+STREAM_OUTPUT="${STREAM_OUTPUT:-ming_output.pcm}"
+REF_AUDIO="${REF_AUDIO:-}"
+REF_TEXT="${REF_TEXT:-}"
+
+post_json() {
+    local payload="$1"
+    local output_path="$2"
+    curl -X POST "$API_URL" \
+        -H "Content-Type: application/json" \
+        -H "Authorization: Bearer EMPTY" \
+        -d "$payload" \
+        --output "$output_path"
+}
+
+case "$MODE" in
+    basic)
+        post_json "{
+            \"model\": \"${MODEL}\",
+            \"input\": \"${TEXT}\",
+            \"response_format\": \"wav\"
+        }" "$OUTPUT"
+        ;;
+    zero_shot)
+        if [ -z "$REF_AUDIO" ] || [ -z "$REF_TEXT" ]; then
+            echo "zero_shot requires REF_AUDIO and REF_TEXT" >&2
+            exit 1
+        fi
+        python - <<'PY' > /tmp/ming_zero_shot_payload.json
+import base64
+import json
+import mimetypes
+import os
+from pathlib import Path
+
+path = Path(os.environ["REF_AUDIO"])
+mime_type = mimetypes.guess_type(path.name)[0] or "audio/wav"
+payload = {
+    "model": os.environ["MODEL"],
+    "input": os.environ["TEXT"],
+    "ref_audio": f"data:{mime_type};base64,{base64.b64encode(path.read_bytes()).decode('utf-8')}",
+    "ref_text": os.environ["REF_TEXT"],
+    "response_format": "wav",
+}
+print(json.dumps(payload, ensure_ascii=False))
+PY
+        curl -X POST "$API_URL" \
+            -H "Content-Type: application/json" \
+            -H "Authorization: Bearer EMPTY" \
+            --data-binary @/tmp/ming_zero_shot_payload.json \
+            --output "$OUTPUT"
+        rm -f /tmp/ming_zero_shot_payload.json
+        ;;
+    stream)
+        post_json "{
+            \"model\": \"${MODEL}\",
+            \"input\": \"${TEXT}\",
+            \"stream\": true,
+            \"response_format\": \"pcm\"
+        }" "$STREAM_OUTPUT"
+        ;;
+    *)
+        echo "Unknown mode: $MODE" >&2
+        echo "Supported sanity checks: basic, zero_shot, stream" >&2
+        exit 1
+        ;;
+esac

examples/online_serving/text_to_speech/ming_tts/run_server.sh

@@ -0,0 +1,25 @@
+#!/bin/bash
+# Launch vLLM-Omni server for Ming-omni-tts.
+#
+# Usage:
+#   ./run_server.sh
+#   PORT=8000 ./run_server.sh
+
+set -e
+
+DIR="$(cd "$(dirname "$0")" && pwd)"
+ROOT="$(cd "$DIR/../../../.." && pwd)"
+
+MODEL="${MODEL:-inclusionAI/Ming-omni-tts-0.5B}"
+PORT="${PORT:-8091}"
+DEPLOY_CONFIG="${DEPLOY_CONFIG:-$ROOT/vllm_omni/deploy/ming_tts.yaml}"
+
+echo "Starting Ming-omni-tts server with model: $MODEL"
+echo "Deploy config: $DEPLOY_CONFIG"
+
+vllm-omni serve "$MODEL" \
+    --deploy-config "$DEPLOY_CONFIG" \
+    --host 0.0.0.0 \
+    --port "$PORT" \
+    --enforce-eager \
+    --omni