[Model] Ming-flash-omni-2.0 Omni-Speech and TTS (#2890)

Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
Signed-off-by: LHXuuu <xulianhao.xlh@antgroup.com>
Co-authored-by: LHXuuu <xulianhao.xlh@antgroup.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>

examples/offline_inference/ming_flash_omni/README.md

@@ -1,75 +1,91 @@
 # Ming-flash-omni 2.0
 
-[Ming-flash-omni-2.0](https://github.com/inclusionAI/Ming) is an omni-modal model supporting text, image, video, and audio understanding, with outputs in text, image, and audio. For now, Ming-flash-omni-2.0 in vLLM-Omni is supported with thinker stage (multi-modal understanding).
+[Ming-flash-omni-2.0](https://github.com/inclusionAI/Ming) is an omni-modal model supporting text, image, video, and audio understanding, with text and speech outputs.
+
+vLLM-Omni supports two deployment modes:
+
+| Mode | Stage config | Output |
+|------|-------------|--------|
+| Thinker only (multimodal understanding) | `ming_flash_omni_thinker.yaml` (default `--omni`) | Text |
+| Thinker + Talker (omni-speech) | `ming_flash_omni.yaml` | Text + Audio |
+
+For standalone TTS (talker only), see [`examples/offline_inference/ming_flash_omni_tts/`](../ming_flash_omni_tts/).
 
 ## Setup
 
 Please refer to the [stage configuration documentation](https://docs.vllm.ai/projects/vllm-omni/en/latest/configuration/stage_configs/) to configure memory allocation appropriately for your hardware setup.
 
+The default `--omni` flag runs thinker only.  For omni-speech, pass the two-stage config explicitly:
+
+```bash
+--stage-configs-path vllm_omni/model_executor/stage_configs/ming_flash_omni.yaml
+```
+
 ## Run examples
 
-### Text-only
+The end-to-end script defaults to built-in assets; pass `--image-path`,
+`--audio-path`, or `--video-path` to override.
+
 ```bash
+# Text-only
 python examples/offline_inference/ming_flash_omni/end2end.py --query-type text
+
+# Image / audio / video / mixed understanding
+python examples/offline_inference/ming_flash_omni/end2end.py --query-type use_image
+python examples/offline_inference/ming_flash_omni/end2end.py --query-type use_audio
+python examples/offline_inference/ming_flash_omni/end2end.py --query-type use_video --num-frames 16
+python examples/offline_inference/ming_flash_omni/end2end.py --query-type use_mixed_modalities \
+    --image-path /path/to/image.jpg --audio-path /path/to/audio.wav
 ```
 
 #### Reasoning (Thinking Mode)
 
-Reasoning (Thinking) mode is enabled via applying "detailed thinking on" when building the system prompt template (in `apply_chat_template`).
-
-In the end2end example, a default problem for thinking mode is provided, as referred to the example usage of Ming's cookbook;
-To utilize it, you have to download the example figure from https://github.com/inclusionAI/Ming/blob/3954fcb880ff5e61ff128bcf7f1ec344d46a6fe3/figures/cases/3_0.png
+Reasoning ("detailed thinking on") is applied by the script when
+`--query-type reasoning` is set. The default prompt matches Ming's cookbook
+and expects the reference figure from the upstream repo — see
+`get_reasoning_query` in `end2end.py`.
 
 ```bash
 python examples/offline_inference/ming_flash_omni/end2end.py -q reasoning --image-path ./3_0.png
 ```
 
-### Image understanding
-```bash
-python examples/offline_inference/ming_flash_omni/end2end.py --query-type use_image
+### Omni-speech (thinker + talker)
 
-# With a local image
-python examples/offline_inference/ming_flash_omni/end2end.py --query-type use_image --image-path /path/to/image.jpg
+To enable spoken output, use the two-stage config and request `audio` (or `text,audio`) modalities.
+The thinker processes your multimodal input, generates text, then the talker synthesises the response as speech.
+
+**Audio-only output** (speech response, no text):
+```bash
+python examples/offline_inference/ming_flash_omni/end2end.py \
+    --query-type text \
+    --stage-configs-path vllm_omni/model_executor/stage_configs/ming_flash_omni.yaml \
+    --modalities audio \
+    --output-dir output_ming_omni_speech
 ```
 
-### Audio understanding
+**Both text and audio output**:
 ```bash
-python examples/offline_inference/ming_flash_omni/end2end.py --query-type use_audio
-
-# With a local audio file
-python examples/offline_inference/ming_flash_omni/end2end.py --query-type use_audio --audio-path /path/to/audio.wav
+python examples/offline_inference/ming_flash_omni/end2end.py \
+    --query-type use_audio \
+    --stage-configs-path vllm_omni/model_executor/stage_configs/ming_flash_omni.yaml \
+    --modalities text,audio \
+    --output-dir output_ming_omni_speech
 ```
 
-### Video understanding
-```bash
-python examples/offline_inference/ming_flash_omni/end2end.py --query-type use_video
+Generated `.wav` files are saved to `--output-dir` (default `output_ming`), one per request.
 
-# With a local video and custom frame count
-python examples/offline_inference/ming_flash_omni/end2end.py --query-type use_video --video-path /path/to/video.mp4 --num-frames 16
-```
-
-### Mixed modalities (image + audio)
-```bash
-python examples/offline_inference/ming_flash_omni/end2end.py --query-type use_mixed_modalities \
-    --image-path /path/to/image.jpg \
-    --audio-path /path/to/audio.wav
-```
-
-If media file paths are not provided, the script uses built-in default assets.
+The stage config allocates thinker on GPUs 0–3 and talker on GPU 3 by default. Adjust `devices` in the YAML to match your hardware.
 
 ### Modality control
-To control output modalities (e.g. text-only output):
-```bash
-python examples/offline_inference/ming_flash_omni/end2end.py --query-type use_audio --modalities text
-```
 
-*For now, only text output is supported*
+| `--modalities` | Thinker output | Talker | Saved files |
+|---------------|----------------|--------|-------------|
+| `text` (default) | Text | Not run | `<id>.txt` |
+| `audio` | Text (internal) | Runs | `<id>.wav` |
+| `text,audio` | Text | Runs | `<id>.txt` + `<id>.wav` |
 
-### Custom stage config
-```bash
-python examples/offline_inference/ming_flash_omni/end2end.py --query-type use_image \
-    --stage-configs-path /path/to/your_config.yaml
-```
+Pass `--stage-configs-path /path/to/your_config.yaml` to any of the commands
+above to override the stage config.
 
 ## Online serving
 

examples/offline_inference/ming_flash_omni/end2end.py

@@ -7,6 +7,7 @@ import time
 from typing import NamedTuple
 
 import numpy as np
+import soundfile as sf
 import vllm
 from PIL import Image
 from transformers import AutoProcessor
@@ -319,7 +320,16 @@ def main(args):
         seed=SEED,
         detokenize=True,
     )
-    sampling_params_list = [thinker_sampling_params]
+    # Talker (ming_tts) uses a custom generation loop (CFM + AudioVAE);
+    # vLLM sampling is a no-op here — max_tokens=1 just satisfies the scheduler.
+    talker_sampling_params = SamplingParams(
+        temperature=0.0,
+        max_tokens=1,
+    )
+    all_sampling_params = [thinker_sampling_params, talker_sampling_params]
+    # Match sampling params to the number of configured stages
+    # (thinker-only yaml → 1, thinker+talker yaml → 2).
+    sampling_params_list = all_sampling_params[: omni.num_stages]
 
     prompts = [query_result.inputs for _ in range(args.num_prompts)]
 
@@ -362,7 +372,19 @@ def main(args):
                 print(f"Failed to write output file {out_txt}: {e}")
 
         elif stage_outputs.final_output_type == "audio":
-            raise NotImplementedError("Add audio example after talker supported.")
+            request_id = output.request_id
+            mm = output.outputs[0].multimodal_output
+            if mm and "audio" in mm:
+                audio = mm["audio"]
+                sr_raw = mm.get("sr", 44100)
+                sample_rate = int(sr_raw.item() if hasattr(sr_raw, "item") else sr_raw)
+                audio_numpy = audio.float().squeeze().cpu().numpy()
+                output_wav = os.path.join(output_dir, f"{request_id}.wav")
+                sf.write(output_wav, audio_numpy, samplerate=sample_rate, format="WAV")
+                print(
+                    f"Request ID: {request_id}, audio saved to {output_wav} "
+                    f"({len(audio_numpy) / sample_rate:.2f}s, {sample_rate}Hz)"
+                )
 
         processed_count += 1
         if profiler_enabled and processed_count >= total_requests:

examples/offline_inference/ming_flash_omni_tts/README.md

@@ -0,0 +1,47 @@
+# Ming-flash-omni Standalone TTS (Offline)
+
+This example runs **Ming-flash-omni-2.0 talker-only** offline inference with:
+
+- `model`: `Jonathan1909/Ming-flash-omni-2.0`
+- `stage config`: `vllm_omni/model_executor/stage_configs/ming_flash_omni_tts.yaml`
+
+It follows the Ming cookbook parameter style:
+
+- `prompt`: `"Please generate speech based on the following description.\n"`
+- `max_decode_steps`: `200`
+- `cfg`: `2.0`
+- `sigma`: `0.25`
+- `temperature`: `0.0`
+
+## Quick Start
+
+```bash
+python examples/offline_inference/ming_flash_omni_tts/end2end.py --case style
+```
+
+## Cases
+
+```bash
+# Style
+python examples/offline_inference/ming_flash_omni_tts/end2end.py --case style
+
+# IP
+python examples/offline_inference/ming_flash_omni_tts/end2end.py --case ip
+
+# Basic (speed/pitch/volume control)
+python examples/offline_inference/ming_flash_omni_tts/end2end.py --case basic
+```
+
+## Useful Arguments
+
+- `--text`: override default text in the selected case
+- `--output`: custom output wav path
+- `--model`: local model path or HF repo id
+- `--stage-configs-path`: custom talker stage config path
+- `--log-stats`: enable runtime stats logs
+
+## Notes
+
+- This directory is for **standalone talker deployment (TTS)**.
+- For Ming thinker multimodal understanding examples, see:
+  `examples/offline_inference/ming_flash_omni/`.

examples/offline_inference/ming_flash_omni_tts/end2end.py

@@ -0,0 +1,129 @@
+"""Offline e2e example for Ming-flash-omni-2.0 standalone talker (TTS)."""
+
+import os
+from typing import Any
+
+import soundfile as sf
+import torch
+from vllm.utils.argparse_utils import FlexibleArgumentParser
+
+os.environ["VLLM_WORKER_MULTIPROC_METHOD"] = "spawn"
+
+from vllm_omni.entrypoints.omni import Omni
+from vllm_omni.inputs.data import OmniTokensPrompt
+from vllm_omni.model_executor.models.ming_flash_omni.prompt_utils import (
+    DEFAULT_PROMPT,
+    create_instruction,
+)
+
+MODEL_NAME = "Jonathan1909/Ming-flash-omni-2.0"
+DEFAULT_STAGE_CONFIG = "vllm_omni/model_executor/stage_configs/ming_flash_omni_tts.yaml"
+
+
+def get_messages(case: str, text_override: str | None) -> dict[str, Any]:
+    if case == "style":
+        text = text_override or "我会一直在这里陪着你，直到你慢慢、慢慢地沉入那个最温柔的梦里……好吗？"
+        instruction = create_instruction(
+            {
+                "风格": "这是一种ASMR耳语，属于一种旨在引发特殊感官体验的创意风格。这个女性使用轻柔的普通话进行耳语，声音气音成分重。音量极低，紧贴麦克风，语速极慢，旨在制造触发听者颅内快感的声学刺激。",
+            }
+        )
+        return {
+            "prompt": DEFAULT_PROMPT,
+            "text": text,
+            "instruction": instruction,
+            "use_zero_spk_emb": True,
+        }
+    if case == "ip":
+        text = text_override or "这款产品的名字，叫变态坑爹牛肉丸。"
+        return {
+            "prompt": DEFAULT_PROMPT,
+            "text": text,
+            "instruction": create_instruction({"IP": "灵小甄"}),
+            "use_zero_spk_emb": True,
+        }
+    if case == "basic":
+        text = text_override or "我们当迎着阳光辛勤耕作，去摘取，去制作，去品尝，去馈赠。"
+        return {
+            "prompt": DEFAULT_PROMPT,
+            "text": text,
+            "instruction": create_instruction({"语速": "快速", "基频": "中", "音量": "中"}),
+            "use_zero_spk_emb": True,
+        }
+    raise ValueError(f"Unknown case: {case}")
+
+
+def save_audio(mm: dict[str, Any], output_path: str) -> None:
+    if not mm or "audio" not in mm:
+        raise RuntimeError("No audio found in model output")
+    audio = mm["audio"]
+    sr_raw = mm.get("sr", 44100)
+    if isinstance(sr_raw, torch.Tensor):
+        sample_rate = int(sr_raw.item())
+    else:
+        sample_rate = int(sr_raw)
+    waveform = audio.squeeze().float().cpu().numpy()
+    sf.write(output_path, waveform, sample_rate)
+    print(f"Saved {output_path} ({len(waveform) / sample_rate:.2f}s, {sample_rate}Hz)")
+
+
+def parse_args():
+    parser = FlexibleArgumentParser(description="Ming-flash-omni standalone talker offline e2e example")
+    parser.add_argument("--model", type=str, default=MODEL_NAME, help="Model name or local path.")
+    parser.add_argument(
+        "--stage-configs-path",
+        type=str,
+        default=DEFAULT_STAGE_CONFIG,
+        help="Path to stage configs yaml for standalone talker deployment.",
+    )
+    parser.add_argument(
+        "--case",
+        type=str,
+        default="style",
+        choices=["style", "ip", "basic"],
+        help="Example case.",
+    )
+    parser.add_argument("--text", type=str, default=None, help="Override default text for the selected case.")
+    parser.add_argument("--output", type=str, default=None, help="Output wav path.")
+    parser.add_argument("--log-stats", action="store_true", default=False, help="Enable stats logging.")
+    parser.add_argument("--init-timeout", type=int, default=600, help="Engine init timeout in seconds.")
+    parser.add_argument("--stage-init-timeout", type=int, default=300, help="Single stage init timeout in seconds.")
+    return parser.parse_args()
+
+
+def main():
+    args = parse_args()
+
+    omni = Omni(
+        model=args.model,
+        stage_configs_path=args.stage_configs_path,
+        trust_remote_code=True,
+        log_stats=args.log_stats,
+        init_timeout=args.init_timeout,
+        stage_init_timeout=args.stage_init_timeout,
+    )
+
+    messages = get_messages(args.case, args.text)
+    decode_args = {
+        # Standalone TTS deployment
+        "ming_task": "instruct",
+        "max_decode_steps": 200,
+        "cfg": 2.0,
+        "sigma": 0.25,
+        "temperature": 0.0,
+    }
+    req = OmniTokensPrompt(
+        prompt_token_ids=[0],
+        additional_information={**messages, **decode_args},
+    )
+
+    outputs = omni.generate(req)
+    mm = outputs[0].outputs[0].multimodal_output
+
+    output_path = args.output or f"output_{args.case}.wav"
+    save_audio(mm, output_path)
+    omni.close()
+
+
+if __name__ == "__main__":
+    main()

examples/online_serving/ming_flash_omni/README.md

@@ -4,38 +4,49 @@
 
 Please refer to [README.md](../../../README.md)
 
+## Deployment modes
+
+| Mode | Launch command | Output |
+|------|---------------|--------|
+| Thinker only (multimodal understanding) | `vllm serve ... --omni` | Text |
+| Thinker + Talker (omni-speech) | `vllm serve ... --omni --stage-configs-path ming_flash_omni.yaml` | Text + Audio |
+
+For standalone TTS (talker only), see [`examples/online_serving/ming_flash_omni_tts/`](../ming_flash_omni_tts/).
+
 ## Run examples (Ming-flash-omni 2.0)
 
 ### Launch the Server
 
+**Thinker only (text output):**
 ```bash
 vllm serve Jonathan1909/Ming-flash-omni-2.0 --omni --port 8091
 ```
 
-If you have custom stage configs file, launch the server with command below
+**Thinker + Talker (omni-speech, text + audio output):**
 ```bash
-vllm serve Jonathan1909/Ming-flash-omni-2.0 --omni --port 8091 --stage-configs-path /path/to/stage_configs_file
+vllm serve Jonathan1909/Ming-flash-omni-2.0 --omni --port 8091 \
+    --stage-configs-path vllm_omni/model_executor/stage_configs/ming_flash_omni.yaml
 ```
 
+Pass `--stage-configs-path /path/to/your_config.yaml` to use a custom stage
+config.
+
 ### Send Multi-modal Request
 
-#### Send request via python
+Shared Python client (supports `text | use_image | use_audio | use_video |
+use_mixed_modalities`; pass `--image-path` / `--audio-path` / `--video-path`
+for local files or URLs, `--modalities text` for output, `--help` for the
+full flag list):
 
 ```bash
-python examples/online_serving/openai_chat_completion_client_for_multimodal_generation.py --model Jonathan1909/Ming-flash-omni-2.0 --query-type use_mixed_modalities --port 8091 --host "localhost" --modalities text
+python examples/online_serving/openai_chat_completion_client_for_multimodal_generation.py \
+    --model Jonathan1909/Ming-flash-omni-2.0 \
+    --query-type use_mixed_modalities \
+    --port 8091 --host localhost \
+    --modalities text
 ```
 
-The Python client supports the following command-line arguments:
-
-- `--query-type` (or `-q`): Query type. Options: `text`, `use_audio`, `use_image`, `use_video`, `use_mixed_modalities`
-- `--video-path` (or `-v`): Path to local video file or URL. If not provided and query-type uses video, uses default video URL. Supports local file paths (automatically encoded to base64) or HTTP/HTTPS URLs. Example: `--video-path /path/to/video.mp4` or `--video-path https://example.com/video.mp4`
-- `--image-path` (or `-i`): Path to local image file or URL. If not provided and query-type uses image, uses default image URL. Supports local file paths (automatically encoded to base64) or HTTP/HTTPS URLs and common image formats: JPEG, PNG, GIF, WebP. Example: `--image-path /path/to/image.jpg` or `--image-path https://example.com/image.png`
-- `--audio-path` (or `-a`): Path to local audio file or URL. If not provided and query-type uses audio, uses default audio URL. Supports local file paths (automatically encoded to base64) or HTTP/HTTPS URLs and common audio formats: MP3, WAV, OGG, FLAC, M4A. Example: `--audio-path /path/to/audio.wav` or `--audio-path https://example.com/audio.mp3`
-- `--prompt` (or `-p`): Custom text prompt/question. If not provided, uses default prompt for the selected query type. Example: `--prompt "What are the main activities shown in this video?"`
-- `--modalities`: Output modalities. For now, only `text` is supported. Example: `--modalities text`
-
-
-#### Send request via curl
+Parameterized curl wrapper in this directory:
 
 ```bash
 bash run_curl_multimodal_generation.sh text
@@ -47,83 +58,17 @@ bash run_curl_multimodal_generation.sh use_mixed_modalities
 
 ## Modality control
 
-Ming-flash-omni 2.0 currently supports text output only (thinker stage).
+| `modalities` | Server config | Output |
+|-------------|--------------|--------|
+| `["text"]` or omitted | Thinker only | Text |
+| `["audio"]` | Thinker + Talker | Audio (speech) |
+| `["text", "audio"]` | Thinker + Talker | Text + Audio |
 
-| Modalities | Output |
-|------------|--------|
-| `["text"]` | Text only |
-| Not specified | Text only (default) |
+For ready-to-copy curl examples (text / audio / multimodal input, SSE
+streaming, reasoning mode), see the recipe at
+[`recipes/inclusionAI/Ming-flash-omni-2.0.md`](../../../recipes/inclusionAI/Ming-flash-omni-2.0.md).
 
-### Using curl
-
-```bash
-curl http://localhost:8091/v1/chat/completions \
-  -H "Content-Type: application/json" \
-  -d '{
-    "model": "Jonathan1909/Ming-flash-omni-2.0",
-    "messages": [
-      {"role": "system", "content": [{"type": "text", "text": "你是一个友好的AI助手。\n\ndetailed thinking off"}]},
-      {"role": "user", "content": "请详细介绍鹦鹉的生活习性。"}
-    ],
-    "modalities": ["text"]
-  }'
-```
-
-### Using OpenAI Python SDK
-
-```python
-from openai import OpenAI
-
-client = OpenAI(base_url="http://localhost:8091/v1", api_key="EMPTY")
-
-response = client.chat.completions.create(
-    model="Jonathan1909/Ming-flash-omni-2.0",
-    messages=[
-        {"role": "system", "content": [{"type": "text", "text": "你是一个友好的AI助手。\n\ndetailed thinking off"}]},
-        {"role": "user", "content": "请详细介绍鹦鹉的生活习性。"},
-    ],
-    modalities=["text"],
-)
-print(response.choices[0].message.content)
-```
-
-### Multi-modal input with OpenAI Python SDK
-
-```python
-from openai import OpenAI
-
-client = OpenAI(base_url="http://localhost:8091/v1", api_key="EMPTY")
-
-response = client.chat.completions.create(
-    model="Jonathan1909/Ming-flash-omni-2.0",
-    messages=[
-        {"role": "system", "content": [{"type": "text", "text": "你是一个友好的AI助手。\n\ndetailed thinking off"}]},
-        {
-            "role": "user",
-            "content": [
-                {"type": "image_url", "image_url": {"url": "https://vllm-public-assets.s3.us-west-2.amazonaws.com/vision_model_images/cherry_blossom.jpg"}},
-                {"type": "text", "text": "Describe this image in detail."},
-            ],
-        },
-    ],
-    modalities=["text"],
-)
-print(response.choices[0].message.content)
-```
-
-## Streaming Output
-
-To enable streaming output:
-
-```bash
-python examples/online_serving/openai_chat_completion_client_for_multimodal_generation.py \
-    --query-type use_image \
-    --model Jonathan1909/Ming-flash-omni-2.0 \
-    --modalities text \
-    --stream
-```
-
-Or with the OpenAI Python SDK:
+## OpenAI Python SDK — streaming
 
 ```python
 from openai import OpenAI
@@ -146,59 +91,5 @@ for chunk in response:
 print()
 ```
 
-Or using curl:
-
-```bash
-curl http://localhost:8091/v1/chat/completions \
-  -H "Content-Type: application/json" \
-  -d '{
-    "model": "Jonathan1909/Ming-flash-omni-2.0",
-    "messages": [
-      {"role": "system", "content": [{"type": "text", "text": "你是一个友好的AI助手。\n\ndetailed thinking off"}]},
-      {"role": "user", "content": "请详细介绍鹦鹉的生活习性。"}
-    ],
-    "modalities": ["text"],
-    "stream": true,
-  }'
-```
-
-
-## Reasoning (Thinking Mode)
-
-To enable reasoning/thinking mode, change `detailed thinking off` to `detailed thinking on` in the system prompt:
-
-### Using curl
-
-```bash
-curl http://localhost:8091/v1/chat/completions \
-  -H "Content-Type: application/json" \
-  -d '{
-    "model": "Jonathan1909/Ming-flash-omni-2.0",
-    "messages": [
-      {"role": "system", "content": [{"type": "text", "text": "你是一个友好的AI助手。\n\ndetailed thinking on"}]},
-      {"role": "user", "content": [
-        {"type": "image_url", "image_url": {"url": "https://example.com/math_problem.png"}},
-        {"type": "text", "text": "Solve this math problem step by step."}
-      ]}
-    ],
-    "modalities": ["text"]
-  }'
-```
-
-### Using OpenAI Python SDK
-
-```python
-from openai import OpenAI
-
-client = OpenAI(base_url="http://localhost:8091/v1", api_key="EMPTY")
-
-response = client.chat.completions.create(
-    model="Jonathan1909/Ming-flash-omni-2.0",
-    messages=[
-        {"role": "system", "content": [{"type": "text", "text": "你是一个友好的AI助手。\n\ndetailed thinking on"}]},
-        {"role": "user", "content": "If a train travels 120 km in 2 hours, what is its average speed?"},
-    ],
-    modalities=["text"],
-)
-print(response.choices[0].message.content)
-```
+The `--stream` flag on the Python client script above shows the same pattern
+driven by the shared multimodal client.

examples/online_serving/ming_flash_omni_tts/README.md

@@ -0,0 +1,54 @@
+# Ming-flash-omni Standalone TTS (Online Serving)
+
+This directory contains online e2e examples for **Ming-flash-omni-2.0 standalone talker deployment**.
+
+Server uses:
+
+- `model`: `Jonathan1909/Ming-flash-omni-2.0`
+- `stage config`: `vllm_omni/model_executor/stage_configs/ming_flash_omni_tts.yaml`
+
+## Launch the Server
+
+```bash
+# from repo root
+bash examples/online_serving/ming_flash_omni_tts/run_server.sh
+```
+
+Equivalent manual command:
+
+```bash
+vllm serve Jonathan1909/Ming-flash-omni-2.0 \
+    --stage-configs-path vllm_omni/model_executor/stage_configs/ming_flash_omni_tts.yaml \
+    --host 0.0.0.0 \
+    --port 8091 \
+    --trust-remote-code \
+    --omni
+```
+
+## Send TTS Request
+
+Python client:
+
+```bash
+python examples/online_serving/ming_flash_omni_tts/speech_client.py \
+    --text "我们当迎着阳光辛勤耕作，去摘取，去制作，去品尝，去馈赠。" \
+    --output ming_online.wav
+```
+
+Long-form `instructions` (e.g. ASMR whisper style) via the client:
+
+```bash
+python examples/online_serving/ming_flash_omni_tts/speech_client.py \
+    --text "我会一直在这里陪着你，直到你慢慢、慢慢地沉入那个最温柔的梦里……好吗？" \
+    --instructions "这是一种ASMR耳语，属于一种旨在引发特殊感官体验的创意风格。这个女性使用轻柔的普通话进行耳语，声音气音成分重。音量极低，紧贴麦克风，语速极慢，旨在制造触发听者颅内快感的声学刺激。" \
+    --output ming_online_asmr.wav
+```
+
+## Notes
+
+- This is the **online serving** counterpart of `examples/offline_inference/ming_flash_omni_tts/`.
+- The server uses `use_zero_spk_emb=True` and the default decode args
+  (`max_decode_steps=200`, `cfg=2.0`, `sigma=0.25`, `temperature=0.0`).
+  For other caption fields (`语速`, `基频`, `IP`, BGM, etc.) or overriding
+  decode args, use the offline e2e example where `additional_information`
+  is set explicitly.

examples/online_serving/ming_flash_omni_tts/run_server.sh

@@ -0,0 +1,25 @@
+#!/bin/bash
+# Launch vLLM-Omni server for Ming-flash-omni-2.0 standalone talker (TTS).
+#
+# Usage:
+#   ./run_server.sh
+#   MODEL=/path/to/local/model ./run_server.sh
+#   PORT=8091 ./run_server.sh
+#   HOST=127.0.0.1 ./run_server.sh   # bind only to loopback
+
+set -e
+
+MODEL="${MODEL:-Jonathan1909/Ming-flash-omni-2.0}"
+HOST="${HOST:-0.0.0.0}"
+PORT="${PORT:-8091}"
+STAGE_CONFIG="${STAGE_CONFIG:-vllm_omni/model_executor/stage_configs/ming_flash_omni_tts.yaml}"
+
+echo "Starting Ming standalone TTS server with model: $MODEL"
+echo "Stage config: $STAGE_CONFIG"
+
+vllm serve "$MODEL" \
+    --stage-configs-path "$STAGE_CONFIG" \
+    --host "$HOST" \
+    --port "$PORT" \
+    --trust-remote-code \
+    --omni

examples/online_serving/ming_flash_omni_tts/speech_client.py

@@ -0,0 +1,93 @@
+"""Client for Ming standalone TTS via /v1/audio/speech endpoint."""
+
+import argparse
+import json
+import sys
+
+import httpx
+
+DEFAULT_API_BASE = "http://localhost:8091"
+DEFAULT_API_KEY = "EMPTY"
+DEFAULT_MODEL = "Jonathan1909/Ming-flash-omni-2.0"
+
+
+def run_tts(args) -> None:
+    payload = {
+        "model": args.model,
+        "input": args.text,
+        "response_format": args.response_format,
+    }
+
+    instructions = args.instructions
+    if args.instruction_json:
+        if instructions:
+            sys.exit("--instructions and --instruction-json are mutually exclusive")
+
+        try:
+            parsed = json.loads(args.instruction_json)
+        except json.JSONDecodeError as exc:
+            sys.exit(f"--instruction-json must be valid JSON: {exc}")
+        if not isinstance(parsed, dict):
+            sys.exit("--instruction-json must decode to a JSON object")
+        # Re-encode with ensure_ascii=False so UTF-8 Chinese keys/values
+        # arrive at the server intact rather than as \\uXXXX escapes.
+        instructions = json.dumps(parsed, ensure_ascii=False)
+    if instructions:
+        payload["instructions"] = instructions
+
+    print(f"Model: {args.model}")
+    print(f"Text: {args.text}")
+    print("Generating audio...")
+
+    api_url = f"{args.api_base}/v1/audio/speech"
+    headers = {
+        "Content-Type": "application/json",
+        "Authorization": f"Bearer {args.api_key}",
+    }
+
+    with httpx.Client(timeout=300.0) as client:
+        response = client.post(api_url, json=payload, headers=headers)
+
+    if response.status_code != 200:
+        print(f"Error: {response.status_code}")
+        print(response.text)
+        return
+
+    output_path = args.output or "ming_tts_output.wav"
+    with open(output_path, "wb") as f:
+        f.write(response.content)
+    print(f"Audio saved to: {output_path}")
+
+
+def main():
+    parser = argparse.ArgumentParser(description="Ming standalone TTS speech client")
+    parser.add_argument("--api-base", default=DEFAULT_API_BASE, help="API base URL")
+    parser.add_argument("--api-key", default=DEFAULT_API_KEY, help="API key")
+    parser.add_argument("--model", "-m", default=DEFAULT_MODEL, help="Model name or local path")
+    parser.add_argument("--text", required=True, help="Text to synthesize")
+    parser.add_argument(
+        "--response-format",
+        default="wav",
+        choices=["wav", "mp3", "flac", "pcm", "aac", "opus"],
+        help="Audio format (default: wav)",
+    )
+    parser.add_argument("--output", "-o", default=None, help="Output file path")
+    parser.add_argument(
+        "--instructions",
+        default=None,
+        help="Free-form style description (mapped to caption 风格 on the server).",
+    )
+    parser.add_argument(
+        "--instruction-json",
+        default=None,
+        help=(
+            "Structured caption JSON forwarded as `instructions`. Accepts Ming "
+            "caption keys: 方言, 风格, 语速, 基频, 音量, 情感, IP, 说话人, BGM. "
+        ),
+    )
+    args = parser.parse_args()
+    run_tts(args)
+
+
+if __name__ == "__main__":
+    main()