[Config Refactor][2/N] Pipeline + Deploy Config Schema (#2383)

Signed-off-by: lishunyang <lishunyang12@163.com>
Signed-off-by: reidliu41 <reid201711@gmail.com>
Signed-off-by: Alex Brooks <albrooks@redhat.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: xiaohajiayou <75477391+xiaohajiayou@users.noreply.github.com>
Co-authored-by: Alex Brooks <albrooks@redhat.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>

examples/online_serving/qwen3_omni/README.md

@@ -12,19 +12,159 @@ Please refer to [README.md](../../../README.md)
 vllm serve Qwen/Qwen3-Omni-30B-A3B-Instruct --omni --port 8091
 ```
 
-If you want to open async chunking for qwen3-omni, launch the server with command below
-
-```bash
-vllm serve Qwen/Qwen3-Omni-30B-A3B-Instruct --omni --port 8091 --stage-configs-path vllm_omni/model_executor/stage_configs/qwen3_omni_moe_async_chunk.yaml
-```
+The default deploy config at `vllm_omni/deploy/qwen3_omni_moe.yaml` is loaded
+automatically by the model registry — no `--deploy-config` flag needed for the
+common case. Async-chunk streaming is **enabled by default** in the bundled config.
+NPU / ROCm / XPU per-platform deltas are merged in automatically from the
+`platforms:` section of the same YAML.
 
 **Note:** The OpenAI-style **`/v1/realtime`** WebSocket (streaming PCM audio in, audio + transcription out) is **not supported** when `async_chunk` is enabled. Use the default omni layout or a stage config with `async_chunk: false` for realtime sessions.
 
-If you have custom stage configs file, launch the server with command below
+If you have a custom deploy YAML, point at it explicitly:
+
 ```bash
-vllm serve Qwen/Qwen3-Omni-30B-A3B-Instruct --omni --port 8091 --stage-configs-path /path/to/stage_configs_file
+vllm serve Qwen/Qwen3-Omni-30B-A3B-Instruct --omni --port 8091 \
+    --deploy-config /path/to/your_deploy_config.yaml
 ```
 
+### Tuning deployment parameters
+
+Most engine knobs (`max_num_batched_tokens`, `max_model_len`, `enforce_eager`,
+`gpu_memory_utilization`, `tensor_parallel_size`, …) can be tuned without
+editing the YAML. There are three layers, in increasing specificity:
+
+#### 1. Global CLI flags (apply to every stage)
+
+```bash
+# Tighter memory budget on a smaller GPU
+vllm serve Qwen/Qwen3-Omni-30B-A3B-Instruct --omni --port 8091 \
+    --gpu-memory-utilization 0.85
+
+# Disable cudagraphs (e.g. for debugging)
+vllm serve Qwen/Qwen3-Omni-30B-A3B-Instruct --omni --port 8091 \
+    --enforce-eager
+
+# Reduce context length
+vllm serve Qwen/Qwen3-Omni-30B-A3B-Instruct --omni --port 8091 \
+    --max-model-len 32768
+
+# Toggle prefix caching on every stage (yaml default: off)
+vllm serve Qwen/Qwen3-Omni-30B-A3B-Instruct --omni --port 8091 \
+    --enable-prefix-caching
+# ...or force it off if the yaml turned it on
+vllm serve Qwen/Qwen3-Omni-30B-A3B-Instruct --omni --port 8091 \
+    --no-enable-prefix-caching
+
+# Toggle pipeline-wide async chunked streaming between stages
+# (yaml default for qwen3_omni_moe: on)
+vllm serve Qwen/Qwen3-Omni-30B-A3B-Instruct --omni --port 8091 \
+    --no-async-chunk
+```
+
+For the TTS counterpart (synchronous codec variant), see
+[qwen3_tts README](../qwen3_tts/README.md#sync-vs-async-chunk-mode).
+
+Explicit CLI flags **override** the deploy YAML (which itself overrides the
+parser defaults). If you don't pass a flag, the YAML value wins.
+
+> **Note on `--no-async-chunk`**: Flips the deploy yaml's `async_chunk:`
+> bool. Pipelines that implement alternate processor functions for
+> chunked vs end-to-end modes (e.g. qwen3_tts code2wav) dispatch
+> automatically based on that bool — no extra flag or variant yaml is
+> needed.
+
+> ⚠️ **For multi-stage models that share GPUs (qwen3_omni_moe by default
+> shares cuda:1 between stages 1 and 2), avoid using global memory flags.**
+> A global `--gpu-memory-utilization 0.85` would apply to every stage and
+> oversubscribe the shared device. Use per-stage overrides instead — see
+> below.
+
+#### 2. Per-stage overrides via `--stage-overrides` (recommended for memory)
+
+```bash
+# Lower stage 1's memory budget; leave others at the YAML default
+vllm serve Qwen/Qwen3-Omni-30B-A3B-Instruct --omni --port 8091 \
+    --stage-overrides '{
+        "1": {"gpu_memory_utilization": 0.5},
+        "2": {"max_num_batched_tokens": 65536}
+    }'
+```
+
+Per-stage values are always treated as explicit and beat YAML defaults for
+the named stage. Other stages keep their YAML values.
+
+#### 3. Custom deploy YAML
+
+When per-stage overrides get long, write a small overlay YAML that inherits
+from the bundled default:
+
+```yaml
+# my_qwen3_omni_overrides.yaml
+base_config: /path/to/vllm_omni/deploy/qwen3_omni_moe.yaml
+
+stages:
+  - stage_id: 0
+    max_num_batched_tokens: 65536
+    enforce_eager: true
+  - stage_id: 1
+    gpu_memory_utilization: 0.5
+  - stage_id: 2
+    max_model_len: 8192
+```
+
+Then start the server with `--deploy-config my_qwen3_omni_overrides.yaml`.
+The `base_config:` line tells the loader to inherit everything else (stages,
+connectors, edges, platforms section) from the bundled production YAML, so
+you only need to spell out the deltas.
+
+#### 4. Multi-node deployment (cross-host transfer connector)
+
+The bundled `qwen3_omni_moe.yaml` uses `SharedMemoryConnector` between stages,
+which only works when all stages run on the same physical host. For
+**cross-node** deployments, write a small overlay YAML that swaps in a
+network-capable connector (e.g. `MooncakeStoreConnector`) and re-points each
+stage's connector wiring at it. The connector spec carries your own server
+addresses — there is no checked-in default because every cluster is
+different.
+
+```yaml
+# my_qwen3_omni_multinode.yaml
+base_config: /path/to/vllm_omni/deploy/qwen3_omni_moe.yaml
+
+connectors:
+  mooncake_connector:
+    name: MooncakeStoreConnector
+    extra:
+      host: "127.0.0.1"
+      metadata_server: "http://YOUR_METADATA_HOST:8080/metadata"
+      master: "YOUR_MASTER_HOST:50051"
+      segment: 512000000    # 512 MB transfer segment
+      localbuf: 64000000    # 64 MB local buffer
+      proto: "tcp"
+
+stages:
+  - stage_id: 0
+    output_connectors:
+      to_stage_1: mooncake_connector
+  - stage_id: 1
+    input_connectors:
+      from_stage_0: mooncake_connector
+    output_connectors:
+      to_stage_2: mooncake_connector
+  - stage_id: 2
+    input_connectors:
+      from_stage_1: mooncake_connector
+```
+
+Then launch with `--deploy-config my_qwen3_omni_multinode.yaml`. Same
+pattern works for Qwen2.5-Omni — replace `base_config:` with the path to
+`vllm_omni/deploy/qwen2_5_omni.yaml`.
+
+> ⚠️ Replace `YOUR_METADATA_HOST` / `YOUR_MASTER_HOST` with the actual
+> mooncake server addresses for your cluster. The `base_config:` overlay
+> inherits all stage budgets, devices, and edges from the bundled prod
+> YAML — you only need to spell out the connector swap.
+
 ### Send Multi-modal Request
 
 Get into the example folder
@@ -285,7 +425,7 @@ The script supports the following arguments:
 - `--model`: Model name/path (default: Qwen/Qwen3-Omni-30B-A3B-Instruct)
 - `--server-port`: Port for vLLM server (default: 8091)
 - `--gradio-port`: Port for Gradio demo (default: 7861)
-- `--stage-configs-path`: Path to custom stage configs YAML file (optional)
+- `--deploy-config`: Path to custom deploy config YAML file (optional)
 - `--server-host`: Host for vLLM server (default: 0.0.0.0)
 - `--gradio-ip`: IP for Gradio demo (default: 127.0.0.1)
 - `--share`: Share Gradio demo publicly (creates a public link)
@@ -300,7 +440,7 @@ vllm serve Qwen/Qwen3-Omni-30B-A3B-Instruct --omni --port 8091
 
 If you have custom stage configs file:
 ```bash
-vllm serve Qwen/Qwen3-Omni-30B-A3B-Instruct --omni --port 8091 --stage-configs-path /path/to/stage_configs_file
+vllm serve Qwen/Qwen3-Omni-30B-A3B-Instruct --omni --port 8091 --deploy-config /path/to/deploy_config_file
 ```
 
 **Step 2: Run the Gradio demo**

examples/online_serving/qwen3_tts/README.md

@@ -43,7 +43,7 @@ Then open http://localhost:7860 in your browser.
 
 ### Launch the Server
 
-The default stage config is located at `vllm_omni/model_executor/stage_configs/qwen3_tts.yaml`. For other platforms (e.g., NPU), refer to `vllm_omni/platforms/npu/stage_configs/qwen3_tts.yaml`.
+The default deploy config is located at `vllm_omni/deploy/qwen3_tts.yaml` and is loaded automatically by the model registry — no `--deploy-config` flag needed for default use. Platform-specific deltas (NPU, ROCm, XPU) are merged in automatically from the `platforms:` block of the same YAML based on the detected runtime.
 
 ```bash
 # CustomVoice model (predefined speakers)
@@ -70,6 +70,22 @@ vllm serve Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice \
     --port 8091
 ```
 
+#### Sync vs async-chunk mode
+
+Qwen3-TTS supports both **chunked streaming** (default, lower latency) and
+**synchronous end-to-end** modes from the same deploy YAML. The bundled
+`qwen3_tts.yaml` ships with `async_chunk: true`; flip with `--no-async-chunk`
+and the pipeline automatically dispatches to the end-to-end codec processor:
+
+```bash
+vllm serve Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice --omni --port 8091 \
+    --no-async-chunk
+```
+
+No variant YAML or extra flag is needed — the `StagePipelineConfig` on each
+stage declares both processor functions and the runtime picks based on the
+`async_chunk:` bool.
+
 Alternatively, use the convenience script:
 ```bash
 ./run_server.sh                  # Default: CustomVoice model

examples/online_serving/qwen3_tts/batch_speech_client.py

@@ -5,11 +5,13 @@ A particularly useful scenario is voice cloning: set ref_audio once at the
 batch level and generate many utterances in the cloned voice without repeating
 the reference for each item.
 
-Start the server (with batch-optimized config for best throughput):
+Start the server (with batch-optimized stage settings for best throughput):
 
     vllm serve Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice \
-        --stage-configs-path vllm_omni/model_executor/stage_configs/qwen3_tts_batch.yaml \
-        --trust-remote-code
+        --omni \
+        --trust-remote-code \
+        --stage-overrides '{"0":{"max_num_seqs":4,"gpu_memory_utilization":0.2},
+                            "1":{"max_num_seqs":4,"gpu_memory_utilization":0.2}}'
 
 Examples:
     # Batch with a predefined voice

examples/online_serving/qwen3_tts/run_gradio_demo.sh

@@ -127,7 +127,7 @@ echo "Starting vLLM server..."
 LOG_FILE="/tmp/vllm_tts_server_${SERVER_PORT}.log"
 
 vllm-omni serve "$MODEL" \
-    --stage-configs-path vllm_omni/model_executor/stage_configs/qwen3_tts.yaml \
+    --deploy-config vllm_omni/deploy/qwen3_tts.yaml \
     --host "$SERVER_HOST" \
     --port "$SERVER_PORT" \
     --gpu-memory-utilization 0.9 \

examples/online_serving/qwen3_tts/run_server.sh

@@ -31,7 +31,7 @@ esac
 echo "Starting Qwen3-TTS server with model: $MODEL"
 
 vllm-omni serve "$MODEL" \
-    --stage-configs-path vllm_omni/model_executor/stage_configs/qwen3_tts.yaml \
+    --deploy-config vllm_omni/deploy/qwen3_tts.yaml \
     --host 0.0.0.0 \
     --port 8091 \
     --gpu-memory-utilization 0.9 \