[Model] Ming-flash-omni-2.0 Omni-Speech and TTS (#2890)

Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
Signed-off-by: LHXuuu <xulianhao.xlh@antgroup.com>
Co-authored-by: LHXuuu <xulianhao.xlh@antgroup.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
This commit is contained in:
authored and GitHub committed 2026-04-23 12:04:52 +08:00
1 parent f327cd1c89
commit 3a32cd6780
32 files changed
+4620 -261

No files matched your search

+37 -146
View File
@@ -4,38 +4,49 @@
Please refer to [README.md](../../../README.md)
## Deployment modes
| Mode | Launch command | Output |
|------|---------------|--------|
| Thinker only (multimodal understanding) | `vllm serve ... --omni` | Text |
| Thinker + Talker (omni-speech) | `vllm serve ... --omni --stage-configs-path ming_flash_omni.yaml` | Text + Audio |
For standalone TTS (talker only), see [`examples/online_serving/ming_flash_omni_tts/`](../ming_flash_omni_tts/).
## Run examples (Ming-flash-omni 2.0)
### Launch the Server
**Thinker only (text output):**
```bash
vllm serve Jonathan1909/Ming-flash-omni-2.0 --omni --port 8091
```
If you have custom stage configs file, launch the server with command below
**Thinker + Talker (omni-speech, text + audio output):**
```bash
vllm serve Jonathan1909/Ming-flash-omni-2.0 --omni --port 8091 --stage-configs-path /path/to/stage_configs_file
vllm serve Jonathan1909/Ming-flash-omni-2.0 --omni --port 8091 \
--stage-configs-path vllm_omni/model_executor/stage_configs/ming_flash_omni.yaml
```
Pass `--stage-configs-path /path/to/your_config.yaml` to use a custom stage
config.
### Send Multi-modal Request
#### Send request via python
Shared Python client (supports `text | use_image | use_audio | use_video |
use_mixed_modalities`; pass `--image-path` / `--audio-path` / `--video-path`
for local files or URLs, `--modalities text` for output, `--help` for the
full flag list):
```bash
python examples/online_serving/openai_chat_completion_client_for_multimodal_generation.py --model Jonathan1909/Ming-flash-omni-2.0 --query-type use_mixed_modalities --port 8091 --host "localhost" --modalities text
python examples/online_serving/openai_chat_completion_client_for_multimodal_generation.py \
--model Jonathan1909/Ming-flash-omni-2.0 \
--query-type use_mixed_modalities \
--port 8091 --host localhost \
--modalities text
```
The Python client supports the following command-line arguments:
- `--query-type` (or `-q`): Query type. Options: `text`, `use_audio`, `use_image`, `use_video`, `use_mixed_modalities`
- `--video-path` (or `-v`): Path to local video file or URL. If not provided and query-type uses video, uses default video URL. Supports local file paths (automatically encoded to base64) or HTTP/HTTPS URLs. Example: `--video-path /path/to/video.mp4` or `--video-path https://example.com/video.mp4`
- `--image-path` (or `-i`): Path to local image file or URL. If not provided and query-type uses image, uses default image URL. Supports local file paths (automatically encoded to base64) or HTTP/HTTPS URLs and common image formats: JPEG, PNG, GIF, WebP. Example: `--image-path /path/to/image.jpg` or `--image-path https://example.com/image.png`
- `--audio-path` (or `-a`): Path to local audio file or URL. If not provided and query-type uses audio, uses default audio URL. Supports local file paths (automatically encoded to base64) or HTTP/HTTPS URLs and common audio formats: MP3, WAV, OGG, FLAC, M4A. Example: `--audio-path /path/to/audio.wav` or `--audio-path https://example.com/audio.mp3`
- `--prompt` (or `-p`): Custom text prompt/question. If not provided, uses default prompt for the selected query type. Example: `--prompt "What are the main activities shown in this video?"`
- `--modalities`: Output modalities. For now, only `text` is supported. Example: `--modalities text`
#### Send request via curl
Parameterized curl wrapper in this directory:
```bash
bash run_curl_multimodal_generation.sh text
@@ -47,83 +58,17 @@ bash run_curl_multimodal_generation.sh use_mixed_modalities
## Modality control
Ming-flash-omni 2.0 currently supports text output only (thinker stage).
| `modalities` | Server config | Output |
|-------------|--------------|--------|
| `["text"]` or omitted | Thinker only | Text |
| `["audio"]` | Thinker + Talker | Audio (speech) |
| `["text", "audio"]` | Thinker + Talker | Text + Audio |
| Modalities | Output |
|------------|--------|
| `["text"]` | Text only |
| Not specified | Text only (default) |
For ready-to-copy curl examples (text / audio / multimodal input, SSE
streaming, reasoning mode), see the recipe at
[`recipes/inclusionAI/Ming-flash-omni-2.0.md`](../../../recipes/inclusionAI/Ming-flash-omni-2.0.md).
### Using curl
```bash
curl http://localhost:8091/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "Jonathan1909/Ming-flash-omni-2.0",
"messages": [
{"role": "system", "content": [{"type": "text", "text": "你是一个友好的AI助手。\n\ndetailed thinking off"}]},
{"role": "user", "content": "请详细介绍鹦鹉的生活习性。"}
],
"modalities": ["text"]
}'
```
### Using OpenAI Python SDK
```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8091/v1", api_key="EMPTY")
response = client.chat.completions.create(
model="Jonathan1909/Ming-flash-omni-2.0",
messages=[
{"role": "system", "content": [{"type": "text", "text": "你是一个友好的AI助手。\n\ndetailed thinking off"}]},
{"role": "user", "content": "请详细介绍鹦鹉的生活习性。"},
],
modalities=["text"],
)
print(response.choices[0].message.content)
```
### Multi-modal input with OpenAI Python SDK
```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8091/v1", api_key="EMPTY")
response = client.chat.completions.create(
model="Jonathan1909/Ming-flash-omni-2.0",
messages=[
{"role": "system", "content": [{"type": "text", "text": "你是一个友好的AI助手。\n\ndetailed thinking off"}]},
{
"role": "user",
"content": [
{"type": "image_url", "image_url": {"url": "https://vllm-public-assets.s3.us-west-2.amazonaws.com/vision_model_images/cherry_blossom.jpg"}},
{"type": "text", "text": "Describe this image in detail."},
],
},
],
modalities=["text"],
)
print(response.choices[0].message.content)
```
## Streaming Output
To enable streaming output:
```bash
python examples/online_serving/openai_chat_completion_client_for_multimodal_generation.py \
--query-type use_image \
--model Jonathan1909/Ming-flash-omni-2.0 \
--modalities text \
--stream
```
Or with the OpenAI Python SDK:
## OpenAI Python SDK — streaming
```python
from openai import OpenAI
@@ -146,59 +91,5 @@ for chunk in response:
print()
```
Or using curl:
```bash
curl http://localhost:8091/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "Jonathan1909/Ming-flash-omni-2.0",
"messages": [
{"role": "system", "content": [{"type": "text", "text": "你是一个友好的AI助手。\n\ndetailed thinking off"}]},
{"role": "user", "content": "请详细介绍鹦鹉的生活习性。"}
],
"modalities": ["text"],
"stream": true,
}'
```
## Reasoning (Thinking Mode)
To enable reasoning/thinking mode, change `detailed thinking off` to `detailed thinking on` in the system prompt:
### Using curl
```bash
curl http://localhost:8091/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "Jonathan1909/Ming-flash-omni-2.0",
"messages": [
{"role": "system", "content": [{"type": "text", "text": "你是一个友好的AI助手。\n\ndetailed thinking on"}]},
{"role": "user", "content": [
{"type": "image_url", "image_url": {"url": "https://example.com/math_problem.png"}},
{"type": "text", "text": "Solve this math problem step by step."}
]}
],
"modalities": ["text"]
}'
```
### Using OpenAI Python SDK
```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8091/v1", api_key="EMPTY")
response = client.chat.completions.create(
model="Jonathan1909/Ming-flash-omni-2.0",
messages=[
{"role": "system", "content": [{"type": "text", "text": "你是一个友好的AI助手。\n\ndetailed thinking on"}]},
{"role": "user", "content": "If a train travels 120 km in 2 hours, what is its average speed?"},
],
modalities=["text"],
)
print(response.choices[0].message.content)
```
The `--stream` flag on the Python client script above shows the same pattern
driven by the shared multimodal client.
@@ -0,0 +1,54 @@
# Ming-flash-omni Standalone TTS (Online Serving)
This directory contains online e2e examples for **Ming-flash-omni-2.0 standalone talker deployment**.
Server uses:
- `model`: `Jonathan1909/Ming-flash-omni-2.0`
- `stage config`: `vllm_omni/model_executor/stage_configs/ming_flash_omni_tts.yaml`
## Launch the Server
```bash
# from repo root
bash examples/online_serving/ming_flash_omni_tts/run_server.sh
```
Equivalent manual command:
```bash
vllm serve Jonathan1909/Ming-flash-omni-2.0 \
--stage-configs-path vllm_omni/model_executor/stage_configs/ming_flash_omni_tts.yaml \
--host 0.0.0.0 \
--port 8091 \
--trust-remote-code \
--omni
```
## Send TTS Request
Python client:
```bash
python examples/online_serving/ming_flash_omni_tts/speech_client.py \
--text "我们当迎着阳光辛勤耕作,去摘取,去制作,去品尝,去馈赠。" \
--output ming_online.wav
```
Long-form `instructions` (e.g. ASMR whisper style) via the client:
```bash
python examples/online_serving/ming_flash_omni_tts/speech_client.py \
--text "我会一直在这里陪着你,直到你慢慢、慢慢地沉入那个最温柔的梦里……好吗?" \
--instructions "这是一种ASMR耳语,属于一种旨在引发特殊感官体验的创意风格。这个女性使用轻柔的普通话进行耳语,声音气音成分重。音量极低,紧贴麦克风,语速极慢,旨在制造触发听者颅内快感的声学刺激。" \
--output ming_online_asmr.wav
```
## Notes
- This is the **online serving** counterpart of `examples/offline_inference/ming_flash_omni_tts/`.
- The server uses `use_zero_spk_emb=True` and the default decode args
(`max_decode_steps=200`, `cfg=2.0`, `sigma=0.25`, `temperature=0.0`).
For other caption fields (`语速`, `基频`, `IP`, BGM, etc.) or overriding
decode args, use the offline e2e example where `additional_information`
is set explicitly.
+25
View File
@@ -0,0 +1,25 @@
#!/bin/bash
# Launch vLLM-Omni server for Ming-flash-omni-2.0 standalone talker (TTS).
#
# Usage:
# ./run_server.sh
# MODEL=/path/to/local/model ./run_server.sh
# PORT=8091 ./run_server.sh
# HOST=127.0.0.1 ./run_server.sh # bind only to loopback
set -e
MODEL="${MODEL:-Jonathan1909/Ming-flash-omni-2.0}"
HOST="${HOST:-0.0.0.0}"
PORT="${PORT:-8091}"
STAGE_CONFIG="${STAGE_CONFIG:-vllm_omni/model_executor/stage_configs/ming_flash_omni_tts.yaml}"
echo "Starting Ming standalone TTS server with model: $MODEL"
echo "Stage config: $STAGE_CONFIG"
vllm serve "$MODEL" \
--stage-configs-path "$STAGE_CONFIG" \
--host "$HOST" \
--port "$PORT" \
--trust-remote-code \
--omni
@@ -0,0 +1,93 @@
"""Client for Ming standalone TTS via /v1/audio/speech endpoint."""
import argparse
import json
import sys
import httpx
DEFAULT_API_BASE = "http://localhost:8091"
DEFAULT_API_KEY = "EMPTY"
DEFAULT_MODEL = "Jonathan1909/Ming-flash-omni-2.0"
def run_tts(args) -> None:
payload = {
"model": args.model,
"input": args.text,
"response_format": args.response_format,
}
instructions = args.instructions
if args.instruction_json:
if instructions:
sys.exit("--instructions and --instruction-json are mutually exclusive")
try:
parsed = json.loads(args.instruction_json)
except json.JSONDecodeError as exc:
sys.exit(f"--instruction-json must be valid JSON: {exc}")
if not isinstance(parsed, dict):
sys.exit("--instruction-json must decode to a JSON object")
# Re-encode with ensure_ascii=False so UTF-8 Chinese keys/values
# arrive at the server intact rather than as \\uXXXX escapes.
instructions = json.dumps(parsed, ensure_ascii=False)
if instructions:
payload["instructions"] = instructions
print(f"Model: {args.model}")
print(f"Text: {args.text}")
print("Generating audio...")
api_url = f"{args.api_base}/v1/audio/speech"
headers = {
"Content-Type": "application/json",
"Authorization": f"Bearer {args.api_key}",
}
with httpx.Client(timeout=300.0) as client:
response = client.post(api_url, json=payload, headers=headers)
if response.status_code != 200:
print(f"Error: {response.status_code}")
print(response.text)
return
output_path = args.output or "ming_tts_output.wav"
with open(output_path, "wb") as f:
f.write(response.content)
print(f"Audio saved to: {output_path}")
def main():
parser = argparse.ArgumentParser(description="Ming standalone TTS speech client")
parser.add_argument("--api-base", default=DEFAULT_API_BASE, help="API base URL")
parser.add_argument("--api-key", default=DEFAULT_API_KEY, help="API key")
parser.add_argument("--model", "-m", default=DEFAULT_MODEL, help="Model name or local path")
parser.add_argument("--text", required=True, help="Text to synthesize")
parser.add_argument(
"--response-format",
default="wav",
choices=["wav", "mp3", "flac", "pcm", "aac", "opus"],
help="Audio format (default: wav)",
)
parser.add_argument("--output", "-o", default=None, help="Output file path")
parser.add_argument(
"--instructions",
default=None,
help="Free-form style description (mapped to caption 风格 on the server).",
)
parser.add_argument(
"--instruction-json",
default=None,
help=(
"Structured caption JSON forwarded as `instructions`. Accepts Ming "
"caption keys: 方言, 风格, 语速, 基频, 音量, 情感, IP, 说话人, BGM. "
),
)
args = parser.parse_args()
run_tts(args)
if __name__ == "__main__":
main()