anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

[Docs] Remove stale stage config references (#6270)

Browse Source 
Signed-off-by: zwhzzz0821 <2831474076@qq.com>
This commit is contained in:
Yukim1 authored and GitHub committed 2026-08-20 07:09:12 +08:00
1 parent efaf4276f9
commit 6f55a06fab
85 files changed
+566 -794
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

examples/offline_inference/covo_audio/README.md
+3 -2
View File
@@ -2,7 +2,8 @@
## Setup
Please refer to the [stage configuration documentation](https://docs.vllm.ai/projects/vllm-omni/en/latest/configuration/stage_configs/) to configure memory allocation appropriately for your hardware setup.
Use `--deploy-config` for deployment overrides such as stage memory allocation.
See the [pipeline and deploy configuration documentation](https://docs.vllm.ai/projects/vllm-omni/en/latest/configuration/stage_configs/).
> **Note**
> Covo-Audio code2wav requires `torchdiffeq`. Install it with: `pip install torchdiffeq`
@@ -41,7 +42,7 @@ python end2end.py -m /path/to/Covo-Audio-Chat --output-dir ./my_output
| `--sampling-rate` | | `16000` | Sampling rate for audio loading (Hz) |
| `--output-dir` | | `./output_audio` | Output directory for generated files |
| `--num-prompts` | | `1` | Number of prompts to generate |
| `--stage-configs-path` | | (auto) | Path to stage configs YAML file |
| `--deploy-config` | | (auto) | Optional path to a deploy YAML override |
| `--log-stats` | | `false` | Enable detailed statistics logging |
| `--stage-init-timeout` | | `300` | Stage initialization timeout (seconds) |
| `--batch-timeout` | | `5` | Batching timeout (seconds) |
examples/offline_inference/dynin_omni/README.md
+1 -1
View File
@@ -107,4 +107,4 @@ python <REPO_ROOT>/examples/offline_inference/dynin_omni/end2end.py \
- If you want to force local config resolution, pass `--dynin-config-path <PATH_TO_DYNIN_OMNI_YAML>`.
- If you see the warning
`max_num_batched_tokens (32768) exceeds max_num_seqs * max_model_len (4096)`,
reduce `max_num_batched_tokens` in stage config (for example, `4096` in CI config).
reduce `max_num_batched_tokens` in the deploy config (for example, `4096` in the CI config).
examples/offline_inference/hunyuan_image3/README.md
+10 -11
View File
@@ -7,17 +7,17 @@ YAMLs under `vllm_omni/deploy/`.
| File | Topology | Default use |
| :--- | :--- | :--- |
| `vllm_omni/deploy/hunyuan_image3.yaml` | AR + DiT | Default for `text2img` and `img2img`. |
| `vllm_omni/deploy/hunyuan_image_3_moe.yaml` | AR + DiT | Default for `text2img` and `img2img`. |
| `vllm_omni/deploy/hunyuan_image3_ar.yaml` | AR only | Default for `img2text` and `text2text`. |
| `vllm_omni/deploy/hunyuan_image3_dit.yaml` | DiT only | Standalone diffusion stage. Pass it explicitly with `--deploy-config`. |
The example chooses a deploy config automatically when `--deploy-config` and
`--stage-configs-path` are both omitted:
The example chooses a deploy config automatically when `--deploy-config` is
omitted:
| `--modality` | `mode` passed to Omni | Default deploy |
| :--- | :--- | :--- |
| `text2img` | `text-to-image` | `hunyuan_image3.yaml` |
| `img2img` | `image-editing` | `hunyuan_image3.yaml` |
| `text2img` | `text-to-image` | `hunyuan_image_3_moe.yaml` |
| `img2img` | `image-editing` | `hunyuan_image_3_moe.yaml` |
| `img2text` | `image-to-text` | `hunyuan_image3_ar.yaml` |
| `text2text` | `text-to-text` | `hunyuan_image3_ar.yaml` |
@@ -35,8 +35,8 @@ path; the image endpoints infer the image task from the endpoint and payload.
| Online scenario | Server deploy | Request |
| :--- | :--- | :--- |
| Text to image | `--deploy-config vllm_omni/deploy/hunyuan_image3.yaml` | `POST /v1/images/generations`, or `POST /v1/chat/completions` with `"modalities": ["image"]`. |
| Image editing | `--deploy-config vllm_omni/deploy/hunyuan_image3.yaml` | `POST /v1/images/edits`. |
| Text to image | `--deploy-config vllm_omni/deploy/hunyuan_image_3_moe.yaml` | `POST /v1/images/generations`, or `POST /v1/chat/completions` with `"modalities": ["image"]`. |
| Image editing | `--deploy-config vllm_omni/deploy/hunyuan_image_3_moe.yaml` | `POST /v1/images/edits`. |
| Image/text to text | `--deploy-config vllm_omni/deploy/hunyuan_image3_ar.yaml` | `POST /v1/chat/completions` for text output, for example with `"modalities": ["text"]`. |
| DiT-only image generation | `--deploy-config vllm_omni/deploy/hunyuan_image3_dit.yaml` | `POST /v1/images/generations`. |
@@ -96,7 +96,7 @@ Override the default full AR + DiT deploy explicitly:
python examples/offline_inference/hunyuan_image3/end2end.py \
--model tencent/HunyuanImage-3.0-Instruct \
--modality text2img \
--deploy-config vllm_omni/deploy/hunyuan_image3.yaml \
--deploy-config vllm_omni/deploy/hunyuan_image_3_moe.yaml \
--prompts "A cute cat"
```
@@ -117,8 +117,7 @@ python end2end.py --modality text2img \
| Argument | Description |
| :--- | :--- |
| `--deploy-config` | Preferred config path for unified deploy YAMLs. |
| `--stage-configs-path` | Legacy stage config path, kept only for compatibility. Prefer `--deploy-config`. |
| `--deploy-config` | Optional path for unified deploy YAML overrides. |
| `--additional-config` | JSON object forwarded to diffusion worker `additional_config`. |
| `--modality` | Offline-only convenience flag. One of `text2img`, `img2img`, `img2text`, `text2text`. It selects prompt formatting, internal `mode`, and default deploy config for this script. Online serving uses `--deploy-config` plus the endpoint and, for chat completions, request `modalities` instead. |
| `--steps` | Number of diffusion inference steps for image generation. |
@@ -132,7 +131,7 @@ python end2end.py --modality text2img \
- `hunyuan_image3_ar.yaml` is a 4-card AR-only text/comprehension deploy.
- `hunyuan_image3_dit.yaml` is a single-stage DiT deploy with `stage_id: 0`.
- The old HunyuanImage3 YAMLs under `model_executor/stage_configs/` and `platforms/*/stage_configs/` have been folded into the deploy YAMLs.
- Platform-specific overrides are included in the unified deploy YAMLs.
## Prompt Format
examples/offline_inference/mimo_audio/README.md
+2 -2
View File
@@ -19,7 +19,7 @@ MiMo-Audio provides multiple task variants for audio understanding and generatio
## Setup
Please refer to the [stage configuration documentation](https://docs.vllm.ai/projects/vllm-omni/en/latest/configuration/stage_configs/) to configure memory allocation appropriately for your hardware setup.
See the [pipeline and deploy configuration documentation](https://docs.vllm.ai/projects/vllm-omni/en/latest/configuration/stage_configs/) to configure memory allocation for your hardware.
### Environment Variables
@@ -189,7 +189,7 @@ Note: This task uses hardcoded message lists in the script.
### Other
- If the model or stage config fails to load, check [stage configuration documentation](https://docs.vllm.ai/projects/vllm-omni/en/latest/configuration/stage_configs/) for memory and GPU settings.
- If the model or deploy config fails to load, check the [pipeline and deploy configuration documentation](https://docs.vllm.ai/projects/vllm-omni/en/latest/configuration/stage_configs/) for memory and GPU settings.
- For errors when reading/writing WAV (e.g. unsupported format), ensure input files are standard WAV/MP3 and that `soundfile` is linked to a working libsndfile (see above).
## Notes
examples/offline_inference/ming_flash_omni/README.md
+1 -1
View File
@@ -14,7 +14,7 @@ For standalone TTS (talker only), see the [Ming-flash-omni-TTS section in the Te
## Setup
Please refer to the [stage configuration documentation](https://docs.vllm.ai/projects/vllm-omni/en/latest/configuration/stage_configs/) to configure memory allocation appropriately for your hardware setup.
See the [pipeline and deploy configuration documentation](https://docs.vllm.ai/projects/vllm-omni/en/latest/configuration/stage_configs/) to configure memory allocation for your hardware.
When no `--deploy-config` is passed, the model registry auto-loads the full thinker+talker `vllm_omni/deploy/ming_flash_omni.yaml` (See [Omni-Speech](#omni-speech-thinker--talker)).
examples/offline_inference/minicpmo/README.md
+1 -1
View File
@@ -8,7 +8,7 @@ Two-stage pipeline: **thinker** (multimodal understanding) → **talker + Token2
- Install talker deps: `pip install stepaudio2-minicpmo`
- `--trust-remote-code` is always passed by `end2end.py` (required for MiniCPMO)
- See [stage configuration docs](https://docs.vllm.ai/projects/vllm-omni/en/latest/configuration/stage_configs/) for memory tuning
- See [pipeline and deploy configuration docs](https://docs.vllm.ai/projects/vllm-omni/en/latest/configuration/stage_configs/) for memory tuning
## Run examples
examples/offline_inference/qwen2_5_omni/README.md
+1 -1
View File
@@ -1,7 +1,7 @@
# Qwen2.5-Omni: Offline inference
## Setup
Please refer to the [stage configuration documentation](https://docs.vllm.ai/projects/vllm-omni/en/latest/configuration/stage_configs/) to configure memory allocation appropriately for your hardware setup.
See the [pipeline and deploy configuration documentation](https://docs.vllm.ai/projects/vllm-omni/en/latest/configuration/stage_configs/) to configure memory allocation for your hardware.
## Run examples
examples/offline_inference/qwen3_omni/README.md
+6 -5
View File
@@ -1,7 +1,8 @@
# Qwen3-Omni: Offline inference
## Setup
Please refer to the [stage configuration documentation](https://docs.vllm.ai/projects/vllm-omni/en/latest/configuration/stage_configs/) to configure memory allocation appropriately for your hardware setup.
Use `--deploy-config` for deployment overrides such as stage memory allocation.
See the [pipeline and deploy configuration documentation](https://docs.vllm.ai/projects/vllm-omni/en/latest/configuration/stage_configs/).
## Run examples
@@ -70,8 +71,8 @@ For true stage-level concurrency -- where downstream stages (Talker, Code2Wav)
start **before** the upstream stage (Thinker) finishes -- use the async_chunk
example. This requires:
1. A deploy config YAML with ``async_chunk: true`` (e.g.
``qwen3_omni_moe.yaml``).
1. A deploy YAML with ``async_chunk: true`` (for example, an overlay based on
``vllm_omni/deploy/qwen3_omni_moe.yaml``).
2. Hardware that matches the config (e.g. 2x H100 for the default 3-stage
config).
@@ -97,11 +98,11 @@ bash run_multiple_prompts_async_chunk.sh --max-in-flight 4
python end2end_async_chunk.py --query-type text --modalities text
```
#### Custom stage config
#### Custom deploy config
```bash
python end2end_async_chunk.py \
--query-type use_audio \
--deploy-config /path/to/your_deploy_config.yaml
--deploy-config /path/to/your_async_chunk.yaml
```
> **Note**: The synchronous ``end2end.py`` (using ``Omni``) is still the
examples/offline_inference/qwen3_omni/end2end_async_chunk.py
+2 -2
View File
@@ -386,8 +386,8 @@ async def run_all(args):
# argparse defaults do not silently override deploy YAML values.
async_omni = AsyncOmni.from_cli_args(args, model=args.model)
# Use default sampling params from stage config (they are pre-configured
# in the YAML for each stage).
# Use default sampling params from the resolved pipeline and deploy
# config.
#
# NOTE: Since we do not set the sampling params directly, .generate in
# will automatically set the output kind to delta, since this is what
examples/offline_inference/step_audio2/README.md
+1 -6
View File
@@ -29,7 +29,7 @@ Step-Audio2 is a two-stage audio model:
**Notes:**
- Single GPU mode requires high VRAM due to both stages sharing memory
- Multi GPU mode separates Stage 0 (Thinker) and Stage 1 (Token2Wav) across GPUs
- VRAM usage can be adjusted via `gpu_memory_utilization` in stage config
- VRAM usage can be adjusted via `gpu_memory_utilization` in the deploy config
## Performance Benchmark
@@ -197,11 +197,6 @@ This mode:
### Advanced Options
```bash
# Use custom stage configuration
python end2end.py --query-type audio_to_text \
--model stepfun-ai/Step-Audio-2-mini \
--stage-configs-path /path/to/custom_config.yaml
# Use custom deploy configuration
python end2end.py --query-type audio_to_text \
--model stepfun-ai/Step-Audio-2-mini \
examples/offline_inference/text_to_speech/README.md
+4 -4
View File
@@ -156,7 +156,7 @@ python examples/offline_inference/text_to_speech/fish_speech/end2end.py \
--text "Hello, this is a streaming test." \
--streaming
```
Streaming requires `async_chunk: true` in the stage config.
Streaming requires `async_chunk: true` in the deploy config.
### Notes
- Output: 44.1 kHz mono WAV.
@@ -371,7 +371,7 @@ python examples/offline_inference/text_to_speech/qwen3_tts/end2end.py \
--streaming \
--output-dir /tmp/out_stream
```
Streaming requires `async_chunk: true` in the stage config.
Streaming requires `async_chunk: true` in the deploy config.
### Word Timestamps
Word-level timestamps are currently a serving-path feature: launch
@@ -380,13 +380,13 @@ Word-level timestamps are currently a serving-path feature: launch
example is not provided in this release.
### Batched decoding
The Code2Wav stage supports batched decoding through the SpeechTokenizer. Pass multiple prompts via `--txt-prompts` and set `--batch-size` accordingly. To raise `max_num_seqs` on either stage, point `--stage-configs-path` at a stage configs YAML with the desired values (see `vllm_omni/model_executor/stage_configs/` for templates):
The Code2Wav stage supports batched decoding through the SpeechTokenizer. Pass multiple prompts via `--txt-prompts` and set `--batch-size` accordingly. To raise `max_num_seqs` on either stage, create an overlay based on `vllm_omni/deploy/qwen3_tts.yaml` and pass it with `--deploy-config`:
```bash
python examples/offline_inference/text_to_speech/qwen3_tts/end2end.py \
--query-type CustomVoice \
--txt-prompts examples/offline_inference/text_to_speech/qwen3_tts/benchmark_prompts.txt \
--batch-size 4 \
--stage-configs-path /path/to/qwen3_tts_batched.yaml
--deploy-config /path/to/qwen3_tts_batched.yaml
```
`--batch-size` must match a CUDA-graph capture size (1, 2, 4, 8, 16…).
examples/offline_inference/text_to_speech/ming_tts/README.md
+1 -1
View File
@@ -131,7 +131,7 @@ python examples/offline_inference/text_to_speech/ming_tts/end2end.py \
--enforce-eager
```
`--streaming` uses `AsyncOmni` and the async_chunk stage config. It currently
`--streaming` uses `AsyncOmni` and the async chunk deploy config. It currently
supports one prompt per process invocation; use blocking mode for
`--num-prompts > 1`.
examples/offline_inference/text_to_speech/voxtral_tts/end2end.py
+1 -1
View File
@@ -304,7 +304,7 @@ def parse_args() -> Namespace:
"--cfg-alpha",
type=float,
default=None,
help="CFG alpha for flow-matching guidance (default: use value from stage config, typically 1.2).",
help="CFG alpha for flow-matching guidance (default: use the deploy config value, typically 1.2).",
)
parser.add_argument(
"--quantization",
examples/online_serving/mimo_audio/openai_chat_completion_client_for_multimodal_generation.py
+1 -1
View File
@@ -328,7 +328,7 @@ def run_multimodal_generation(args) -> None:
extra_body = {
"sampling_params_list": sampling_params_list
# Optional, it has a default setting in stage_configs of the corresponding model.
# Optional; defaults come from the resolved pipeline and deploy config.
}
# Build messages list (filter None so concurrent tasks get valid structure)
examples/online_serving/openai_chat_completion_client_for_multimodal_generation.py
+1 -1
View File
@@ -415,7 +415,7 @@ def run_multimodal_generation(args, client: OpenAI) -> None:
audio_path=audio_path,
)
extra_body = {
# Optional, it has default settings in stage configs. you can override them here.
# Optional; defaults come from the resolved pipeline and deploy config.
}
if args.query_type == "use_audio_in_video":
extra_body["mm_processor_kwargs"] = {"use_audio_in_video": True}
examples/online_serving/personaplex/README.md
+1 -1
View File
@@ -38,7 +38,7 @@ full-duplex control plane:
CUDA_VISIBLE_DEVICES=0 python -m vllm_omni.entrypoints.cli.main serve \
/path/to/personaplex-7b-v1 \
--omni \
--stage-configs-path vllm_omni/deploy/personaplex.yaml
--deploy-config vllm_omni/deploy/personaplex.yaml
```
Validate the actual `/v1/realtime?duplex=1` scheduler path with paced 24 kHz
examples/online_serving/qwen2_5_omni/run_curl_multimodal_generation.sh
+1 -1
View File
@@ -51,7 +51,7 @@ code2wav_sampling_params='{
"detokenize": true,
"repetition_penalty": 1.1
}'
# Above is optional, it has a default setting in stage_configs of the corresponding model.
# The block above is optional; defaults come from the resolved pipeline and deploy config.
# Define URLs for assets
MARY_HAD_LAMB_AUDIO_URL="https://vllm-public-assets.s3.us-west-2.amazonaws.com/multimodal_asset/mary_had_lamb.ogg"
examples/online_serving/qwen3_omni/README.md
+1 -1
View File
@@ -513,7 +513,7 @@ The script supports the following arguments:
vllm serve Qwen/Qwen3-Omni-30B-A3B-Instruct --omni --port 8091
```
If you have custom stage configs file:
If you have a custom deploy config file:
```bash
vllm serve Qwen/Qwen3-Omni-30B-A3B-Instruct --omni --port 8091 --deploy-config /path/to/deploy_config_file
```
examples/online_serving/qwen3_omni/run_curl_multimodal_generation.sh
+1 -1
View File
@@ -50,7 +50,7 @@ code2wav_sampling_params='{
"detokenize": true,
"repetition_penalty": 1.1
}'
# Above is optional, it has a default setting in stage_configs of the corresponding model.
# The block above is optional; defaults come from the resolved pipeline and deploy config.
# Define URLs for assets
MARY_HAD_LAMB_AUDIO_URL="https://vllm-public-assets.s3.us-west-2.amazonaws.com/multimodal_asset/mary_had_lamb.ogg"
examples/online_serving/replica_data_parallel/bench_replica_dp.py
+1 -1
View File
@@ -15,7 +15,7 @@ so an output is a deterministic function of its own input and nothing else.
Measuring replica scaling
-------------------------
This client is replica-agnostic. Vary the *server's* replica count
(`runtime.num_replicas` in the stage config; see run_server.sh) and keep the
(`stages[].num_replicas` in the deploy config; see run_server.sh) and keep the
client workload fixed, then compare throughput:
N=1 -> baseline
examples/online_serving/step_audio2/README.md
+1 -1
View File
@@ -189,5 +189,5 @@ Async chunk reduces TTFP by **67%** by streaming audio token chunks from Thinker
- Check server logs for errors
### Out of memory
- Reduce `gpu_memory_utilization` in stage configs
- Reduce `gpu_memory_utilization` in the deploy config
- Use a smaller batch size
examples/online_serving/text_to_speech/README.md
+1 -1
View File
@@ -570,7 +570,7 @@ curl -X POST http://localhost:8091/v1/audio/speech \
"response_format": "pcm"
}' --no-buffer | play -t raw -r 24000 -e signed -b 16 -c 1 -
```
Raw PCM streaming requires `stream_format="audio"`, `response_format="pcm"`, and `async_chunk: true` on the stage config (default in `qwen3_tts.yaml`). `speed` is not supported when streaming.
Raw PCM streaming requires `stream_format="audio"`, `response_format="pcm"`, and `async_chunk: true` in the deploy config (default in `qwen3_tts.yaml`). `speed` is not supported when streaming.
### Streaming WebSocket
The `/v1/audio/speech/stream` endpoint accepts text incrementally and synthesizes the buffered text as one continuous request on `input.done`: