anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

[Refactor] Remove legacy --stage-configs-path from the serve CLI (#5647)

Browse Source 
Signed-off-by: zwhzzz0821 <2831474076@qq.com>
This commit is contained in:
Yukim1 authored and GitHub committed 2026-08-03 23:41:52 +08:00
1 parent 76de642c54
commit 7a2007cc17
29 files changed
+113 -108
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/cli/README.md
+1 -7
View File
@@ -18,18 +18,12 @@ Specify the port:
vllm serve Qwen/Qwen2.5-Omni-7B --omni --port 8091
```
For a migrated model, load a custom deploy configuration with `--deploy-config`:
Load a custom deploy configuration with `--deploy-config`:
```bash
vllm serve Qwen/Qwen2.5-Omni-7B --omni --deploy-config /path/to/deploy_config.yaml
```
The deprecated `--stage-configs-path` flag is retained for models that still use the legacy `stage_args` schema:
```bash
vllm serve ByteDance-Seed/BAGEL-7B-MoT --omni --stage-configs-path /path/to/legacy_stage_config.yaml
```
## bench
Run benchmark tests for online serving throughput.
docs/cli/serve.md
+1 -2
View File
@@ -8,7 +8,6 @@ The stage-based CLI is designed for deployments that require launching each pipe
- For **migrated models** that utilize the bundled deployment YAML configurations located in
`vllm_omni/deploy/`, the `--deploy-config` flag is only required to override the default configuration. By default, executing `vllm serve MODEL --omni ...`
automatically loads the bundled deployment configuration.
- For custom legacy `stage_args` YAMLs, pass the file with `--stage-configs-path`.
Example: Initializing Stage 0 (Orchestrator and API Server):
The commands below show a common device mapping where Stage 0 uses GPU 0 and
@@ -32,7 +31,7 @@ CUDA_VISIBLE_DEVICES=1 vllm serve Qwen/Qwen3-Omni-30B-A3B-Instruct --omni \
--omni-master-port 26000
```
When utilizing a custom deployment YAML based on the new schema, append `--deploy-config /path/to/override.yaml` to each command execution. Conversely, for legacy models, substitute this parameter with `--stage-configs-path /path/to/stage_configs.yaml`.
When utilizing a custom deployment YAML, append `--deploy-config /path/to/override.yaml` to each command execution.
In the standard execution paradigm, the `--stage-overrides` argument is utilized to apply stage-specific configurations from a single CLI command.
However, under the **stage-based CLI** paradigm, where each process strictly encapsulates a single stage, it is recommended to specify tuning parameters directly via discrete command-line flags for the respective stage, rather than constructing a composite `--stage-overrides` JSON string.
docs/configuration/composable_parallel.md
-15
View File
@@ -34,12 +34,8 @@ Two additional `StrategySpec` slots — `layer_hook_specs` and `kernel_specs`
| Flag | Description |
|------|-------------|
| `--strategy-config PATH` | Path to a `strategy.yaml`. Only takes effect on registry-based deploy paths (`--deploy-config` or the bundled default deploy YAML). Its derived sizing is overlaid onto the merged stages *before* CLI overrides. |
| `--stage-configs-path PATH` | **Legacy.** Loads the old `stage_args` YAML schema for un-migrated models. Passing `--strategy-config` together with this path is silently inapplicable — vLLM-Omni emits a warning and the strategy is ignored (see `vllm_omni/entrypoints/utils.py`, lines 372–380). |
| `--omni-lb-policy POLICY` | Orchestrator-wide load-balancer policy. Strategy-derived from any `stage_replica` axis; an explicit CLI value wins and must match the derived one or `AsyncOmniEngine` raises `Conflicting load-balancer policy` at construction. |
!!! warning
`--strategy-config` cannot be used with `--stage-configs-path` (the legacy deploy path). If your model has not been migrated to the registry-based pipeline yet, the strategy is silently dropped with a warning. Migrate to a registry-based model or use `--deploy-config` to apply a strategy.
## Sub-schemas
### `axis`
@@ -216,17 +212,6 @@ Strategy keys MUST be `model_stage` *names* (strings), the same labels stage con
!!! warning "Mixing CLI overrides with a strategy on the same axis"
A strategy is meant to be the single writer for the axes it declares. If a CLI override (`--tensor-parallel-size 2`, `--stage-overrides '{"0": {"tensor_parallel_size": 2}}'`) sets the same axis, the **CLI value wins** and a warning is emitted from `_reconcile_strategy_with_cli`. The device-layout guard then re-runs against the effective layout, so a CLI override that breaks `tp * dp * pp * num_replicas` will still fail fast at config time.
!!! warning "Passing `--strategy-config` against the legacy `--stage-configs-path`"
Composable parallel only applies on the registry-based deploy path. If a model still resolves through the legacy `stage_args` YAML (loaded via `--stage-configs-path`), `--strategy-config` is silently inapplicable and emits:
```text
--strategy-config (<path>) was provided but model '<model>' resolves via the
legacy stage_configs YAML path, which does not support composable-parallel
strategies; the strategy is ignored. Use a registry-based model to apply it.
```
Migrate the model to the new deploy schema (see `configuration/stage_configs.md`) before applying a strategy.
!!! warning "Conflicting `omni_lb_policy` across stages"
`omni_lb_policy` is a single pipeline-wide knob. If two `stage_replica` axes in different stages derive different policies, `apply_strategy_specs` raises:
docs/configuration/pd_disaggregation.md
+1 -1
View File
@@ -158,7 +158,7 @@ runtime:
```bash
vllm serve Qwen/Qwen3-Omni-30B-A3B-Instruct --omni --port 8091 \
--stage-configs-path /path/to/qwen3_omni_pd.yaml
--deploy-config /path/to/qwen3_omni_pd.yaml
```
## Operational Notes
docs/configuration/stage_configs.md
+6 -14
View File
@@ -1,9 +1,9 @@
# Deploy and legacy stage configurations
# Pipeline and deploy configurations
In vLLM-Omni, a model's `PipelineConfig` defines its fixed stage topology, while a deploy configuration controls how those stages run. Models that have not migrated to this split still use legacy stage configurations with a `stage_args` schema.
In vLLM-Omni, a model's `PipelineConfig` defines its fixed stage topology, while a deploy configuration controls how those stages run.
!!! note
Default deploy config YAMLs (for example, `vllm_omni/deploy/qwen2_5_omni.yaml`, `vllm_omni/deploy/qwen3_omni_moe.yaml`, and `vllm_omni/deploy/qwen3_tts.yaml`) are bundled and loaded automatically when neither `--stage-configs-path` nor `--deploy-config` is provided — the model registry resolves the right pipeline + deploy YAML by `model_type`. Custom legacy `stage_args` YAMLs are still accepted via `--stage-configs-path`.
Default deploy config YAMLs (for example, `vllm_omni/deploy/qwen2_5_omni.yaml`, `vllm_omni/deploy/qwen3_omni_moe.yaml`, and `vllm_omni/deploy/qwen3_tts.yaml`) are bundled and loaded automatically when `--deploy-config` is omitted. The model registry resolves the right pipeline and deploy YAML by `model_type`.
## Deploy configuration schema
@@ -86,10 +86,9 @@ stages:
| Flag | Description |
|------|-------------|
| `--deploy-config PATH` | Load a new-schema deploy YAML. It is mutually exclusive with `--stage-configs-path`. **Optional** — when omitted, the bundled `vllm_omni/deploy/<model_type>.yaml` is auto-loaded by the model registry. |
| `--deploy-config PATH` | Load a deploy YAML. **Optional** — when omitted, the bundled `vllm_omni/deploy/<model_type>.yaml` is auto-loaded by the model registry. |
| `--stage-overrides JSON` | Per-stage JSON overrides, e.g. `'{"0":{"gpu_memory_utilization":0.5}}'`. Per-stage values always win over global flags. |
| `--async-chunk` / `--no-async-chunk` | Flip the deploy YAML's `async_chunk:` bool. Unset (default) leaves the YAML value in force. |
| `--stage-configs-path` | **Deprecated.** Load a legacy `stage_args` YAML. New-schema YAMLs passed to this flag are auto-detected for compatibility, but should use `--deploy-config`. It is mutually exclusive with `--deploy-config` and will be removed in a future release. |
### Stage-Based CLI Paradigm
@@ -119,8 +118,7 @@ CUDA_VISIBLE_DEVICES=1 vllm serve Qwen/Qwen3-Omni-30B-A3B-Instruct --omni \
--omni-master-port 26000
```
When instantiating a custom deployment YAML conforming to the updated schema, append the `--deploy-config /path/to/override.yaml` directive
to all node invocations. For legacy architectures (e.g., BAGEL) configured via deprecated `stage_args:` schemas, continue to specify the relevant configuration via `--stage-configs-path /path/to/config.yaml`.
When instantiating a custom deployment YAML, append the `--deploy-config /path/to/override.yaml` directive to all node invocations.
In the context of standard initialization architectures, utilizing the `--stage-overrides` parameter operates as the optimal methodology
for delineating stage-specific tuning from the CLI interface:
@@ -220,8 +218,7 @@ Therefore, as a core part of vLLM-Omni, a model's pipeline and deployment config
To override specific parameters, explicitly inject the customized configuration schema
in both online and offline instantiation flows. Use the `--deploy-config` flag
when loading a deploy configuration, reserving the `--stage-configs-path` parameter
exclusively to maintain compatibility with legacy `stage_args` YAML constructs.
when loading a deploy configuration.
Examples:
@@ -236,11 +233,6 @@ For online serving:
vllm serve Qwen/Qwen2.5-Omni-7B --omni --port 8091 --deploy-config /path/to/deploy_config.yaml
```
Legacy online serving:
```bash
vllm serve ByteDance-Seed/BAGEL-7B-MoT --omni --port 8091 --stage-configs-path /path/to/stage_configs_file
```
!!! important
We are actively iterating on the definition of deployment configurations, and we welcome feedback from users and developers.
docs/contributing/model/adding_diffusion_model.md
+1 -1
View File
@@ -653,7 +653,7 @@ python end2end.py --model your-org/your-model-name --modality text2img --prompts
Mirror BAGEL’s online serving setup:
- Server launcher: `examples/online_serving/your_model_name/run_server.sh`
- Wrap `vllm serve ... --omni --port ...` (and `--stage-configs-path ...` if needed)
- Wrap `vllm serve ... --omni --port ...` (and `--deploy-config ...` if needed)
- Client: `examples/online_serving/your_model_name/openai_chat_client.py`
- Send requests to `POST /v1/chat/completions`
- Support multimodal inputs (e.g., base64 image) if your model needs it
docs/contributing/model/adding_omni_model.md
+2 -2
View File
@@ -48,8 +48,8 @@ vllm_omni/deploy/
```
New in-tree models should register their `PipelineConfig` in
`vllm_omni/config/pipeline_registry.py`; use `--stage-configs-path` only for
custom legacy `stage_args` YAMLs.
`vllm_omni/config/pipeline_registry.py`; use `--deploy-config` for custom
deployment YAMLs.
## Step-by-Step Implementation
docs/contributing/model/adding_tts_model.md
+2 -3
View File
@@ -449,9 +449,8 @@ Key configuration fields:
!!! note
New in-tree models should define frozen topology in `models/<model>/pipeline.py`,
register it in `vllm_omni/config/pipeline_registry.py`, and put deployment
knobs in `vllm_omni/deploy/<model>.yaml`. Legacy `stage_args` YAMLs are still
accepted for custom configs via `--stage-configs-path`, but should not be used
for new bundled defaults.
knobs in `vllm_omni/deploy/<model>.yaml`. Use `--deploy-config` to load a
custom deployment YAML.
### Batch mode
docs/design/qwen3_omni_tts_performance_optimization.md
+2 -3
View File
@@ -419,8 +419,7 @@ Notes:
```bash
vllm serve Qwen/Qwen3-Omni-30B-A3B-Instruct \
--omni \
--port 8091 \
--stage-configs-path vllm_omni/model_executor/stage_configs/qwen3_omni_moe_async_chunk.yaml
--port 8091
```
#### 3) Key config knobs
@@ -494,7 +493,7 @@ The default config (`qwen3_tts.yaml`) enables the full optimization stack:
vllm-omni serve Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice \
--omni \
--port 8000 \
--stage-configs-path vllm_omni/model_executor/stage_configs/qwen3_tts_no_async_chunk.yaml
--no-async-chunk
```
#### 3) Key config knobs
docs/user_guide/examples/online_serving/bagel.md
+1 -1
View File
@@ -40,7 +40,7 @@ bash run_server_stage_cli.sh --stage 0
bash run_server_stage_cli.sh --stage 1
```
To use a custom deploy YAML (note: `--stage-configs-path` is deprecated in favor of `--deploy-config`):
To use a custom deploy YAML:
```bash
vllm serve ByteDance-Seed/BAGEL-7B-MoT --omni --port 8091 \
docs/user_guide/examples/online_serving/glm_image.md
+2 -2
View File
@@ -11,8 +11,8 @@ memory, sampling params) live in `vllm_omni/deploy/glm_image.yaml`.
vllm serve zai-org/GLM-Image --omni --port 8091
```
The config system auto-detects the pipeline from the model's `model_index.json` — no
manual `--stage-configs-path` or `--deploy-config` needed.
The config system auto-detects the pipeline from the model's `model_index.json`, so
no manual `--deploy-config` is needed.
By default, stage 0 (AR) runs on GPU 0 and stage 1 (Diffusion) on GPU 1. To colocate
both stages on a single GPU, override per stage:
docs/user_guide/examples/online_serving/qwen2_5_omni.md
+5 -5
View File
@@ -15,9 +15,9 @@ Please refer to [README.md](https://github.com/vllm-project/vllm-omni/tree/main/
vllm serve Qwen/Qwen2.5-Omni-7B --omni --port 8091
```
If you have custom stage configs file, launch the server with command below
To use a custom deploy config, launch the server with the command below:
```bash
vllm serve Qwen/Qwen2.5-Omni-7B --omni --port 8091 --stage-configs-path /path/to/stage_configs_file
vllm serve Qwen/Qwen2.5-Omni-7B --omni --port 8091 --deploy-config /path/to/deploy_config.yaml
```
### Send Multi-modal Request
@@ -182,7 +182,7 @@ The script supports the following arguments:
- `--model`: Model name/path (default: Qwen/Qwen2.5-Omni-7B)
- `--server-port`: Port for vLLM server (default: 8091)
- `--gradio-port`: Port for Gradio demo (default: 7861)
- `--stage-configs-path`: Path to custom stage configs YAML file (optional)
- `--deploy-config`: Path to a custom deploy config YAML file (optional)
- `--server-host`: Host for vLLM server (default: 0.0.0.0)
- `--gradio-ip`: IP for Gradio demo (default: 127.0.0.1)
- `--share`: Share Gradio demo publicly (creates a public link)
@@ -195,9 +195,9 @@ The script supports the following arguments:
vllm serve Qwen/Qwen2.5-Omni-7B --omni --port 8091
```
If you have custom stage configs file:
To use a custom deploy config:
```bash
vllm serve Qwen/Qwen2.5-Omni-7B --omni --port 8091 --stage-configs-path /path/to/stage_configs_file
vllm serve Qwen/Qwen2.5-Omni-7B --omni --port 8091 --deploy-config /path/to/deploy_config.yaml
```
**Step 2: Run the Gradio demo**
docs/user_guide/fault_injection_reliability_matrix.md
+4 -4
View File
@@ -69,19 +69,19 @@ The behaviors and conclusions above are summarized from current fault injection
### Example: `SIGTERM` fault injection log (Qwen3-Omni)
The following full log is from a real run where a `SIGTERM` signal was sent to the serve root process.
The following log is based on a real run where a `SIGTERM` signal was sent to the serve root process. Its command line has been normalized to the current `--deploy-config` option.
```text
Launching OmniServer with: /workspace/.venv/bin/python3 -m vllm_omni.entrypoints.cli.main serve Qwen/Qwen3-Omni-30B-A3B-Instruct --host 127.0.0.1 --port 60675 --omni --async-chunk --stage-init-timeout 600 --init-timeout 900 --log-stats --stage-configs-path vllm-omni/vllm_omni/deploy/qwen3_omni_moe.yaml
Launching OmniServer with: /workspace/.venv/bin/python3 -m vllm_omni.entrypoints.cli.main serve Qwen/Qwen3-Omni-30B-A3B-Instruct --host 127.0.0.1 --port 60675 --omni --async-chunk --stage-init-timeout 600 --init-timeout 900 --log-stats --deploy-config vllm-omni/vllm_omni/deploy/qwen3_omni_moe.yaml
Server ready on 127.0.0.1:60675 (OmniServer startup took 206.023s)
OmniServer started successfully
[reliability][process-kill] current_server_proc pid=556855 name=python3 cmdline=/workspace/.venv/bin/python3 -m vllm_omni.entrypoints.cli.main serve Qwen/Qwen3-Omni-30B-A3B-Instruct --host 127.0.0.1 --port 60675 --omni --async-chunk --stage-init-timeout 600 --init-timeout 900 --log-stats --stage-configs-path vllm-omni/vllm_omni/deploy/qwen3_omni_moe.yaml
[reliability][process-kill] current_server_proc pid=556855 name=python3 cmdline=/workspace/.venv/bin/python3 -m vllm_omni.entrypoints.cli.main serve Qwen/Qwen3-Omni-30B-A3B-Instruct --host 127.0.0.1 --port 60675 --omni --async-chunk --stage-init-timeout 600 --init-timeout 900 --log-stats --deploy-config vllm-omni/vllm_omni/deploy/qwen3_omni_moe.yaml
[reliability][process-kill] current_server_proc pid=557118 name=python3 cmdline=/workspace/.venv/bin/python3 -c from multiprocessing.resource_tracker import main;main(57)
[reliability][process-kill] current_server_proc pid=557119 name=VLLM::StageEngineCoreProc_noid_replica0_DP0 cmdline=VLLM::StageEngineCoreProc_noid_replica0_DP0
[reliability][process-kill] current_server_proc pid=557122 name=VLLM::StageEngineCoreProc_noid_replica0_DP0 cmdline=VLLM::StageEngineCoreProc_noid_replica0_DP0
[reliability][process-kill] current_server_proc pid=558201 name=VLLM::StageEngineCoreProc_noid_replica0_DP0 cmdline=VLLM::StageEngineCoreProc_noid_replica0_DP0
[reliability][process-kill] root-kill pid=556855 name=python3 signal=SIGTERM cmdline=/workspace/.venv/bin/python3 -m vllm_omni.entrypoints.cli.main serve Qwen/Qwen3-Omni-30B-A3B-Instruct --host 127.0.0.1 --port 60675 --omni --async-chunk --stage-init-timeout 600 --init-timeout 900 --log-stats --stage-configs-path vllm-omni/vllm_omni/deploy/qwen3_omni_moe.yaml
[reliability][process-kill] root-kill pid=556855 name=python3 signal=SIGTERM cmdline=/workspace/.venv/bin/python3 -m vllm_omni.entrypoints.cli.main serve Qwen/Qwen3-Omni-30B-A3B-Instruct --host 127.0.0.1 --port 60675 --omni --async-chunk --stage-init-timeout 600 --init-timeout 900 --log-stats --deploy-config vllm-omni/vllm_omni/deploy/qwen3_omni_moe.yaml
[0;36m(APIServer pid=556855)[0;0m INFO 05-19 12:24:51 [omni_base.py:463] [AsyncOmni] Shutting down
[0;36m(APIServer pid=556855)[0;0m INFO 05-19 12:24:51 [async_omni_engine.py:2397] [AsyncOmniEngine] Shutting down Orchestrator
[0;36m(APIServer pid=556855)[0;0m INFO 05-19 12:24:51 [orchestrator.py:351] [Orchestrator] Received shutdown signal