[Docs] Remove stale stage config references (#6270)

Signed-off-by: zwhzzz0821 <2831474076@qq.com>
This commit is contained in:
Yukim1 authored and GitHub committed 2026-08-20 07:09:12 +08:00
1 parent efaf4276f9
commit 6f55a06fab
85 files changed
+566 -794

No files matched your search

+2 -2
View File
@@ -335,8 +335,8 @@ The single resolution boundary is implemented by
`StageConfigFactory.create_from_model()` and
`VllmOmniConfig.from_pipeline_config()` in
[`vllm_omni/config`](https://github.com/vllm-project/vllm-omni/tree/main/vllm_omni/config).
The legacy `stage_args` YAML path remains only for models that have not yet
migrated to `PipelineConfig` and `DeployConfig`.
The legacy `stage_args` YAML path has been removed. Model topology now resolves
through `PipelineConfig`, with runtime overrides supplied by `DeployConfig`.
In the typed path, each stage configuration derives from
`BaseVllmOmniStageConfig` and is specialized as
+23 -25
View File
@@ -211,38 +211,39 @@ In sequential mode, each stage must wait for the previous stage to complete enti
## Configuration
Enable async_chunk in stage configuration YAML:
Enable async chunk mode in the deploy YAML:
```yaml
async_chunk: true
stage_args:
- stage_id: 0
engine_args:
custom_process_next_stage_input_func: vllm_omni.model_executor.stage_input_processors.qwen3_omni.thinker2talker_async_chunk
- stage_id: 1
engine_args:
custom_process_next_stage_input_func: vllm_omni.model_executor.stage_input_processors.qwen3_omni.talker2code2wav_async_chunk
```
### Stage Configuration
The registered `PipelineConfig` owns the async handoff processor functions.
For Qwen3-Omni these are `thinker2talker_async_chunk` and
`talker2code2wav_async_chunk`; deploy YAML only selects the mode and runtime
sizing.
### Configuration ownership
- `async_chunk: bool`: Enable/disable async chunk mode
- `custom_process_next_stage_input_func: str`: Path to custom chunk processing function; receives `(transfer_manager, pooling_output, request)`. For qwen3-omni: `thinker2talker_async_chunk`, `talker2code2wav_async_chunk`
- `stage_connector_config: dict`: Connector configuration
- `worker_type: str`: Model type, e.g. `"ar"` or `"generation"` (used by OmniChunkTransferAdapter for mode-specific payload handling)
- `async_chunk_process_next_stage_input_func: str`: Pipeline-owned chunk processor path
- `execution_type: StageExecutionType`: Pipeline-owned runtime family
- `connectors: dict`: Deploy-owned connector definitions
- `max_num_seqs: int`: Maximum number of sequences for concurrent processing in the stage
### Connector Configuration
```yaml
connectors:
- from_stage: 0
to_stage: 1
spec:
name: SharedMemoryConnector
extra:
stage_id: 0
connector_of_shared_memory:
name: SharedMemoryConnector
stages:
- stage_id: 0
output_connectors:
to_stage_1: connector_of_shared_memory
- stage_id: 1
input_connectors:
from_stage_0: connector_of_shared_memory
```
### Code2Wav Batch Configuration
@@ -250,13 +251,10 @@ connectors:
For optimal performance with async_chunk, the code2wav stage should be configured with batching:
```yaml
stage_args:
stages:
- stage_id: 2 # code2wav stage
runtime:
devices: "1"
engine_args:
model_stage: code2wav
max_num_seqs: 64 # Enables batched audio generation
devices: "1"
max_num_seqs: 64 # Enables batched audio generation
```
## Related Files
@@ -59,19 +59,18 @@ This `metadata` must be passed through the control plane so `get()` can locate t
## Configuration Model
Define connectors in runtime:
Define connectors at the top level of the deploy YAML:
```yaml
runtime:
connectors:
connector_of_shared_memory:
name: SharedMemoryConnector
connectors:
connector_of_shared_memory:
name: SharedMemoryConnector
```
Wire stages to connectors:
```yaml
stage_args:
stages:
- stage_id: 0
output_connectors:
to_stage_1: connector_of_shared_memory
@@ -32,26 +32,25 @@ mooncake_master \
## Configuration
Define the connector in runtime:
Define the connector at the top level of the deploy YAML:
```yaml
runtime:
connectors:
connector_of_mooncake:
name: MooncakeStoreConnector
extra:
host: "127.0.0.1"
metadata_server: "http://<MASTER_IP>:8080/metadata"
master: "<MASTER_IP>:50051"
segment: 512000000
localbuf: 64000000
proto: "tcp"
connectors:
connector_of_mooncake:
name: MooncakeStoreConnector
extra:
host: "127.0.0.1"
metadata_server: "http://<MASTER_IP>:8080/metadata"
master: "<MASTER_IP>:50051"
segment: 512000000
localbuf: 64000000
proto: "tcp"
```
Wire stages to the connector:
```yaml
stage_args:
stages:
- stage_id: 0
output_connectors:
to_stage_1: connector_of_mooncake
@@ -21,26 +21,25 @@ InfiniBand/RoCE NICs).
## Configuration
Define the connector in runtime:
Define the connector at the top level of the deploy YAML:
```yaml
runtime:
connectors:
rdma_connector:
name: MooncakeTransferEngineConnector
extra:
host: "auto" # Auto-detect local RDMA IP
zmq_port: 50051 # ZMQ base port (see "Port Offset Scheme" below)
protocol: "rdma" # "rdma" or "tcp"
device_name: "" # RDMA device (e.g., "mlx5_0"), empty for auto-detect
memory_pool_size: 4294967296 # 4 GB (CPU); use 2147483648 (2 GB) for GPU
memory_pool_device: "cpu" # "cpu" for pinned memory (recommended), "cuda" for GPUDirect RDMA
connectors:
rdma_connector:
name: MooncakeTransferEngineConnector
extra:
host: "auto" # Auto-detect local RDMA IP
zmq_port: 50051 # ZMQ base port (see "Port Offset Scheme" below)
protocol: "rdma" # "rdma" or "tcp"
device_name: "" # RDMA device (e.g., "mlx5_0"), empty for auto-detect
memory_pool_size: 4294967296 # 4 GB (CPU); use 2147483648 (2 GB) for GPU
memory_pool_device: "cpu" # "cpu" for pinned memory (recommended), "cuda" for GPUDirect RDMA
```
Wire stages to the connector:
```yaml
stage_args:
stages:
- stage_id: 0
output_connectors:
to_stage_1: rdma_connector
@@ -415,7 +414,7 @@ A receiver connector:
- allocates receive buffers from its own pool
- requests metadata or transfer service from the sender
The role is not inferred dynamically. It is injected by the stage configuration layer:
The role is not inferred dynamically. It is injected from the deploy connector wiring:
- incoming edge for a stage -> `role="receiver"`
- outgoing edge for a stage -> `role="sender"`
@@ -13,10 +13,9 @@ segment name is returned in metadata.
## Configuration
```yaml
runtime:
connectors:
connector_of_shared_memory:
name: SharedMemoryConnector
connectors:
connector_of_shared_memory:
name: SharedMemoryConnector
```
## Notes
@@ -65,23 +65,22 @@ dscli stop --worker_address "${WORKER_IP}:31501"
## Configuration
Define the connector in runtime:
Define the connector at the top level of the deploy YAML:
```yaml
runtime:
connectors:
connector_of_yuanrong:
name: YuanrongConnector
extra:
host: "127.0.0.1"
port: 31501
get_sub_timeout_ms: 1000
connectors:
connector_of_yuanrong:
name: YuanrongConnector
extra:
host: "127.0.0.1"
port: 31501
get_sub_timeout_ms: 1000
```
Wire stages to the connector:
```yaml
stage_args:
stages:
- stage_id: 0
output_connectors:
to_stage_1: connector_of_yuanrong
@@ -94,26 +94,23 @@ For Ascend P2P, configure the connector with `protocol: "ascend"` and an NPU
memory pool:
```yaml
runtime:
enabled: true
defaults:
connectors:
yuanrong_te_connector:
name: YuanrongTransferEngineConnector
extra:
host: "auto"
zmq_port: 50051
rpc_port: "auto"
protocol: "ascend"
device_name: "auto"
memory_pool_size: 1073741824
memory_pool_device: "npu"
edges:
- from: 0
to: 1
window_size: -1
max_inflight: 1
connectors:
yuanrong_te_connector:
name: YuanrongTransferEngineConnector
extra:
host: "auto"
zmq_port: 50051
rpc_port: "auto"
protocol: "ascend"
device_name: "auto"
memory_pool_size: 1073741824
memory_pool_device: "npu"
edges:
- from: 0
to: 1
window_size: -1
```
Important fields:
@@ -141,31 +138,27 @@ For AR 4-way TP sending KV cache to DiT 4-way TP, set the stage TP topology in
both stage `omni_kv_config` blocks:
```yaml
stage_args:
stages:
- stage_id: 0
runtime:
devices: "0,1,2,3"
engine_args:
tensor_parallel_size: 4
omni_kv_config:
need_send_cache: true
rank_mapping:
from_tp: 4
to_tp: 4
devices: "0,1,2,3"
tensor_parallel_size: 4
omni_kv_config:
need_send_cache: true
rank_mapping:
from_tp: 4
to_tp: 4
output_connectors:
to_stage_1: yuanrong_te_connector
- stage_id: 1
runtime:
devices: "4,5,6,7"
engine_args:
parallel_config:
tensor_parallel_size: 4
omni_kv_config:
need_recv_cache: true
rank_mapping:
from_tp: 4
to_tp: 4
devices: "4,5,6,7"
parallel_config:
tensor_parallel_size: 4
omni_kv_config:
need_recv_cache: true
rank_mapping:
from_tp: 4
to_tp: 4
input_connectors:
from_stage_0: yuanrong_te_connector
```
+1 -1
View File
@@ -41,7 +41,7 @@ With this in mind, consider the set of blocks in a 2D layout, where the row repr
### Example
!!! note "Note 3"
Prefix caching in vLLM-Omni currently is only supported on AutoRegressive stages with one kv-cache group. It can be enabled/disabled per-stage via the `enable_prefix_caching` parameter in the model's stage config.
Prefix caching in vLLM-Omni currently is only supported on AutoRegressive stages with one kv-cache group. Configure it with the pipeline-wide `enable_prefix_caching` field in the deploy config.
The way in which vLLM-Omni ties into vLLM's prefix caching is best understood by example. Say that we have the following:
+1 -1
View File
@@ -42,7 +42,7 @@ and execution behavior to stable runtime contracts.
### MODEL-INV-001: Registration is explicit
**Rule:** A model integration MUST declare how its model class, loader, input
processor, and stage configuration are selected.
processor, and pipeline configuration are selected.
### MODEL-INV-002: Model code does not route stages
+3 -4
View File
@@ -10,9 +10,8 @@ owners:
primary_code_paths:
- vllm_omni/config/**
- vllm_omni/deploy/**
- vllm_omni/model_executor/stage_configs/**
related_code_paths:
- vllm_omni/platforms/*/stage_configs/**
- vllm_omni/model_executor/models/*/pipeline.py
related_code_paths: []
depends_on: []
validation_paths:
- tests/config/**
@@ -95,7 +94,7 @@ parallel input is effective for a diffusion stage.
## Safe-change guide while deferred
Changes should trace every affected producer to its runtime consumer and test
the structured, legacy, CLI, and deployment paths that are actually supported.
the structured pipeline, CLI, and deployment paths that are actually supported.
For inherited vLLM fields, tests should separately cover constructor behavior
and effective `EngineArgs` projection. Do not infer a stable contract from
this placeholder page.
@@ -111,7 +111,7 @@ This post walks through each optimization in the same order they are typically e
For both Qwen3-Omni and Qwen3-TTS, batching is a pipeline-level optimization:
- Requests are grouped per stage using `runtime.max_batch_size`
- Requests are grouped per stage using `stages[].max_num_seqs`
- Each stage executes batch inference with its own scheduler/worker
- Stage outputs are routed to downstream stages with per-request mapping preserved
@@ -171,7 +171,7 @@ At concurrency 10, batching alone brings Qwen3-TTS RTF from 2.19 (slower than re
In decode-heavy serving, repeatedly launching many small kernels from CPU can become a visible overhead. CUDA Graph reduces this overhead by capturing and replaying stable execution graphs.
In stage configs, this is represented by `enforce_eager: false` for stages where graph capture is desired (Thinker/Talker), while Code2Wav keeps eager mode depending on stage behavior.
In deploy configs, this is represented by `enforce_eager: false` for stages where graph capture is desired (Thinker/Talker), while Code2Wav keeps eager mode depending on stage behavior.
### CUDA Graph results on top of batching
@@ -320,7 +320,7 @@ In the async chunk pipeline, the standard `codec_chunk_frames` is 25 (each chunk
**Dynamic initial chunk sizing (default behavior):**
Rather than using a fixed initial chunk size, vLLM-Omni dynamically selects it based on current server load. The initial chunk size is chosen from power-of-2 steps [2, 4, 8, 16] based on load factor (`active_requests / max_batch_size`):
Rather than using a fixed initial chunk size, vLLM-Omni dynamically selects it based on current server load. The initial chunk size is chosen from power-of-2 steps [2, 4, 8, 16] based on load factor (`active_requests / max_num_seqs`):
| Server load | Initial chunk frames | Rationale |
| --- | --- | --- |
@@ -349,16 +349,15 @@ This overrides the dynamic calculation for that request.
**Config (server-side):**
```yaml
runtime:
connectors:
connector_of_shared_memory:
name: SharedMemoryConnector
extra:
codec_streaming: true
codec_chunk_frames: 25 # standard chunk size (~2s of audio)
codec_left_context_frames: 25
# initial chunk is computed dynamically by default
# set initial_codec_chunk_frames: 2 to force a fixed value
connectors:
connector_of_shared_memory:
name: SharedMemoryConnector
extra:
codec_streaming: true
codec_chunk_frames: 25 # standard chunk size (~2s of audio)
codec_left_context_frames: 25
# initial chunk is computed dynamically by default
# set initial_codec_chunk_frames: 2 to force a fixed value
```
The 64 ms TTFP result reported above for Qwen3-TTS at concurrency 1 uses the dynamic initial chunk, which picks `initial_codec_chunk_frames=2` at low load. At higher concurrency the dynamic sizing increases the initial chunk to maintain decode efficiency.
@@ -408,7 +407,7 @@ vllm serve Qwen/Qwen3-Omni-30B-A3B-Instruct \
Notes:
- `runtime.max_batch_size` controls stage-level batching.
- `stages[].max_num_seqs` controls stage-level batching.
- Thinker/Talker commonly use `enforce_eager: false` for CUDA Graph paths.
- Code2Wav often remains eager (`enforce_eager: true`) depending on runtime behavior.
- Qwen3-Omni defaults `VLLM_USE_FLASHINFER_MOE_FP16=0`. The Triton has been more stable & faster
@@ -426,33 +425,26 @@ vllm serve Qwen/Qwen3-Omni-30B-A3B-Instruct \
```yaml
async_chunk: true
stage_args:
stages:
- stage_id: 0 # thinker
runtime:
max_batch_size: 64
engine_args:
enforce_eager: false
max_num_batched_tokens: 32768
custom_process_next_stage_input_func: >-
vllm_omni.model_executor.stage_input_processors.qwen3_omni.thinker2talker_async_chunk
max_num_seqs: 64
enforce_eager: false
max_num_batched_tokens: 32768
- stage_id: 1 # talker
runtime:
max_batch_size: 64
engine_args:
enforce_eager: false
max_num_batched_tokens: 32768
custom_process_next_stage_input_func: >-
vllm_omni.model_executor.stage_input_processors.qwen3_omni.talker2code2wav_async_chunk
max_num_seqs: 64
enforce_eager: false
max_num_batched_tokens: 32768
- stage_id: 2 # code2wav
runtime:
max_batch_size: 64
engine_args:
enforce_eager: true
max_num_batched_tokens: 51200
max_num_seqs: 64
enforce_eager: true
max_num_batched_tokens: 51200
```
The async handoff processors are fixed by the registered Qwen3-Omni
`PipelineConfig`; deploy YAML only selects `async_chunk` and runtime sizing.
#### Reproduce Qwen3-Omni benchmarks
```bash
@@ -483,7 +475,7 @@ vllm-omni serve Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice \
The default config (`qwen3_tts.yaml`) enables the full optimization stack:
- Batching with `max_batch_size: 10` on the Talker stage
- Batching with `max_num_seqs: 10` on the Talker stage
- CUDA Graph on the Talker (`enforce_eager: false`)
- Async chunk with streaming transport
@@ -500,33 +492,29 @@ vllm-omni serve Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice \
```yaml
async_chunk: true
stage_args:
stages:
- stage_id: 0 # Talker (AR decoder)
runtime:
max_batch_size: 10
engine_args:
enforce_eager: false
max_num_batched_tokens: 512
custom_process_next_stage_input_func: >-
vllm_omni.model_executor.stage_input_processors.qwen3_tts.talker2code2wav_async_chunk
max_num_seqs: 10
enforce_eager: false
max_num_batched_tokens: 512
- stage_id: 1 # Code2Wav (vocoder)
runtime:
max_batch_size: 1
engine_args:
enforce_eager: true
max_num_batched_tokens: 8192
max_num_seqs: 1
enforce_eager: true
max_num_batched_tokens: 8192
runtime:
connectors:
connector_of_shared_memory:
name: SharedMemoryConnector
extra:
codec_streaming: true
codec_chunk_frames: 25
codec_left_context_frames: 25
connectors:
connector_of_shared_memory:
name: SharedMemoryConnector
extra:
codec_streaming: true
codec_chunk_frames: 25
codec_left_context_frames: 25
```
The async Talker-to-Code2Wav processor is defined by the registered Qwen3-TTS
pipeline rather than the deploy YAML.
#### Reproduce Qwen3-TTS benchmarks
```bash