[Docs] Remove stale stage config references (#6270)

Signed-off-by: zwhzzz0821 <2831474076@qq.com>
This commit is contained in:
Yukim1 authored and GitHub committed 2026-08-20 07:09:12 +08:00
1 parent efaf4276f9
commit 6f55a06fab
85 files changed
+566 -794

No files matched your search

+23 -25
View File
@@ -211,38 +211,39 @@ In sequential mode, each stage must wait for the previous stage to complete enti
## Configuration
Enable async_chunk in stage configuration YAML:
Enable async chunk mode in the deploy YAML:
```yaml
async_chunk: true
stage_args:
- stage_id: 0
engine_args:
custom_process_next_stage_input_func: vllm_omni.model_executor.stage_input_processors.qwen3_omni.thinker2talker_async_chunk
- stage_id: 1
engine_args:
custom_process_next_stage_input_func: vllm_omni.model_executor.stage_input_processors.qwen3_omni.talker2code2wav_async_chunk
```
### Stage Configuration
The registered `PipelineConfig` owns the async handoff processor functions.
For Qwen3-Omni these are `thinker2talker_async_chunk` and
`talker2code2wav_async_chunk`; deploy YAML only selects the mode and runtime
sizing.
### Configuration ownership
- `async_chunk: bool`: Enable/disable async chunk mode
- `custom_process_next_stage_input_func: str`: Path to custom chunk processing function; receives `(transfer_manager, pooling_output, request)`. For qwen3-omni: `thinker2talker_async_chunk`, `talker2code2wav_async_chunk`
- `stage_connector_config: dict`: Connector configuration
- `worker_type: str`: Model type, e.g. `"ar"` or `"generation"` (used by OmniChunkTransferAdapter for mode-specific payload handling)
- `async_chunk_process_next_stage_input_func: str`: Pipeline-owned chunk processor path
- `execution_type: StageExecutionType`: Pipeline-owned runtime family
- `connectors: dict`: Deploy-owned connector definitions
- `max_num_seqs: int`: Maximum number of sequences for concurrent processing in the stage
### Connector Configuration
```yaml
connectors:
- from_stage: 0
to_stage: 1
spec:
name: SharedMemoryConnector
extra:
stage_id: 0
connector_of_shared_memory:
name: SharedMemoryConnector
stages:
- stage_id: 0
output_connectors:
to_stage_1: connector_of_shared_memory
- stage_id: 1
input_connectors:
from_stage_0: connector_of_shared_memory
```
### Code2Wav Batch Configuration
@@ -250,13 +251,10 @@ connectors:
For optimal performance with async_chunk, the code2wav stage should be configured with batching:
```yaml
stage_args:
stages:
- stage_id: 2 # code2wav stage
runtime:
devices: "1"
engine_args:
model_stage: code2wav
max_num_seqs: 64 # Enables batched audio generation
devices: "1"
max_num_seqs: 64 # Enables batched audio generation
```
## Related Files
@@ -59,19 +59,18 @@ This `metadata` must be passed through the control plane so `get()` can locate t
## Configuration Model
Define connectors in runtime:
Define connectors at the top level of the deploy YAML:
```yaml
runtime:
connectors:
connector_of_shared_memory:
name: SharedMemoryConnector
connectors:
connector_of_shared_memory:
name: SharedMemoryConnector
```
Wire stages to connectors:
```yaml
stage_args:
stages:
- stage_id: 0
output_connectors:
to_stage_1: connector_of_shared_memory
@@ -32,26 +32,25 @@ mooncake_master \
## Configuration
Define the connector in runtime:
Define the connector at the top level of the deploy YAML:
```yaml
runtime:
connectors:
connector_of_mooncake:
name: MooncakeStoreConnector
extra:
host: "127.0.0.1"
metadata_server: "http://<MASTER_IP>:8080/metadata"
master: "<MASTER_IP>:50051"
segment: 512000000
localbuf: 64000000
proto: "tcp"
connectors:
connector_of_mooncake:
name: MooncakeStoreConnector
extra:
host: "127.0.0.1"
metadata_server: "http://<MASTER_IP>:8080/metadata"
master: "<MASTER_IP>:50051"
segment: 512000000
localbuf: 64000000
proto: "tcp"
```
Wire stages to the connector:
```yaml
stage_args:
stages:
- stage_id: 0
output_connectors:
to_stage_1: connector_of_mooncake
@@ -21,26 +21,25 @@ InfiniBand/RoCE NICs).
## Configuration
Define the connector in runtime:
Define the connector at the top level of the deploy YAML:
```yaml
runtime:
connectors:
rdma_connector:
name: MooncakeTransferEngineConnector
extra:
host: "auto" # Auto-detect local RDMA IP
zmq_port: 50051 # ZMQ base port (see "Port Offset Scheme" below)
protocol: "rdma" # "rdma" or "tcp"
device_name: "" # RDMA device (e.g., "mlx5_0"), empty for auto-detect
memory_pool_size: 4294967296 # 4 GB (CPU); use 2147483648 (2 GB) for GPU
memory_pool_device: "cpu" # "cpu" for pinned memory (recommended), "cuda" for GPUDirect RDMA
connectors:
rdma_connector:
name: MooncakeTransferEngineConnector
extra:
host: "auto" # Auto-detect local RDMA IP
zmq_port: 50051 # ZMQ base port (see "Port Offset Scheme" below)
protocol: "rdma" # "rdma" or "tcp"
device_name: "" # RDMA device (e.g., "mlx5_0"), empty for auto-detect
memory_pool_size: 4294967296 # 4 GB (CPU); use 2147483648 (2 GB) for GPU
memory_pool_device: "cpu" # "cpu" for pinned memory (recommended), "cuda" for GPUDirect RDMA
```
Wire stages to the connector:
```yaml
stage_args:
stages:
- stage_id: 0
output_connectors:
to_stage_1: rdma_connector
@@ -415,7 +414,7 @@ A receiver connector:
- allocates receive buffers from its own pool
- requests metadata or transfer service from the sender
The role is not inferred dynamically. It is injected by the stage configuration layer:
The role is not inferred dynamically. It is injected from the deploy connector wiring:
- incoming edge for a stage -> `role="receiver"`
- outgoing edge for a stage -> `role="sender"`
@@ -13,10 +13,9 @@ segment name is returned in metadata.
## Configuration
```yaml
runtime:
connectors:
connector_of_shared_memory:
name: SharedMemoryConnector
connectors:
connector_of_shared_memory:
name: SharedMemoryConnector
```
## Notes
@@ -65,23 +65,22 @@ dscli stop --worker_address "${WORKER_IP}:31501"
## Configuration
Define the connector in runtime:
Define the connector at the top level of the deploy YAML:
```yaml
runtime:
connectors:
connector_of_yuanrong:
name: YuanrongConnector
extra:
host: "127.0.0.1"
port: 31501
get_sub_timeout_ms: 1000
connectors:
connector_of_yuanrong:
name: YuanrongConnector
extra:
host: "127.0.0.1"
port: 31501
get_sub_timeout_ms: 1000
```
Wire stages to the connector:
```yaml
stage_args:
stages:
- stage_id: 0
output_connectors:
to_stage_1: connector_of_yuanrong
@@ -94,26 +94,23 @@ For Ascend P2P, configure the connector with `protocol: "ascend"` and an NPU
memory pool:
```yaml
runtime:
enabled: true
defaults:
connectors:
yuanrong_te_connector:
name: YuanrongTransferEngineConnector
extra:
host: "auto"
zmq_port: 50051
rpc_port: "auto"
protocol: "ascend"
device_name: "auto"
memory_pool_size: 1073741824
memory_pool_device: "npu"
edges:
- from: 0
to: 1
window_size: -1
max_inflight: 1
connectors:
yuanrong_te_connector:
name: YuanrongTransferEngineConnector
extra:
host: "auto"
zmq_port: 50051
rpc_port: "auto"
protocol: "ascend"
device_name: "auto"
memory_pool_size: 1073741824
memory_pool_device: "npu"
edges:
- from: 0
to: 1
window_size: -1
```
Important fields:
@@ -141,31 +138,27 @@ For AR 4-way TP sending KV cache to DiT 4-way TP, set the stage TP topology in
both stage `omni_kv_config` blocks:
```yaml
stage_args:
stages:
- stage_id: 0
runtime:
devices: "0,1,2,3"
engine_args:
tensor_parallel_size: 4
omni_kv_config:
need_send_cache: true
rank_mapping:
from_tp: 4
to_tp: 4
devices: "0,1,2,3"
tensor_parallel_size: 4
omni_kv_config:
need_send_cache: true
rank_mapping:
from_tp: 4
to_tp: 4
output_connectors:
to_stage_1: yuanrong_te_connector
- stage_id: 1
runtime:
devices: "4,5,6,7"
engine_args:
parallel_config:
tensor_parallel_size: 4
omni_kv_config:
need_recv_cache: true
rank_mapping:
from_tp: 4
to_tp: 4
devices: "4,5,6,7"
parallel_config:
tensor_parallel_size: 4
omni_kv_config:
need_recv_cache: true
rank_mapping:
from_tp: 4
to_tp: 4
input_connectors:
from_stage_0: yuanrong_te_connector
```
+1 -1
View File
@@ -41,7 +41,7 @@ With this in mind, consider the set of blocks in a 2D layout, where the row repr
### Example
!!! note "Note 3"
Prefix caching in vLLM-Omni currently is only supported on AutoRegressive stages with one kv-cache group. It can be enabled/disabled per-stage via the `enable_prefix_caching` parameter in the model's stage config.
Prefix caching in vLLM-Omni currently is only supported on AutoRegressive stages with one kv-cache group. Configure it with the pipeline-wide `enable_prefix_caching` field in the deploy config.
The way in which vLLM-Omni ties into vLLM's prefix caching is best understood by example. Say that we have the following: