[Config Refactor][2/N] Pipeline + Deploy Config Schema (#2383)
Signed-off-by: lishunyang <lishunyang12@163.com> Signed-off-by: reidliu41 <reid201711@gmail.com> Signed-off-by: Alex Brooks <albrooks@redhat.com> Co-authored-by: reidliu41 <reid201711@gmail.com> Co-authored-by: xiaohajiayou <75477391+xiaohajiayou@users.noreply.github.com> Co-authored-by: Alex Brooks <albrooks@redhat.com> Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
This commit is contained in:
104 files changed
+3959
-3814
No files matched your search
@@ -231,8 +231,7 @@ vllm_omni/ tests/
|
||||
│ ├── test_qwen3_omni_expansion.py
|
||||
│ ├── test_mimo_audio.py
|
||||
│ ├── test_image_gen_edit.py
|
||||
│ ├── test_images_generations_lora.py
|
||||
│ └── stage_configs/
|
||||
│ └── test_images_generations_lora.py
|
||||
└── offline_inference/ ✅
|
||||
├── test_qwen2_5_omni.py
|
||||
├── test_qwen3_omni.py
|
||||
@@ -248,11 +247,12 @@ vllm_omni/ tests/
|
||||
├── test_diffusion_layerwise_offload.py
|
||||
├── test_diffusion_lora.py
|
||||
├── test_sequence_parallel.py
|
||||
└── stage_configs/
|
||||
├── qwen2_5_omni_ci.yaml
|
||||
├── qwen3_omni_ci.yaml
|
||||
├── bagel_*.yaml
|
||||
└── npu/, rocm/, etc.
|
||||
└── stage_configs/ (legacy schema, still
|
||||
├── bagel_*.yaml present for unmigrated
|
||||
└── npu/, rocm/, etc. models)
|
||||
|
||||
# Migrated models (qwen3_omni_moe, qwen2_5_omni, qwen3_tts) live under
|
||||
# vllm_omni/deploy/ instead — see docs/configuration/stage_configs.md.
|
||||
```
|
||||
|
||||
|
||||
|
||||
@@ -135,8 +135,7 @@ vllm_omni/ tests/
|
||||
│ ├── test_qwen3_omni_expansion.py
|
||||
│ ├── test_mimo_audio.py
|
||||
│ ├── test_image_gen_edit.py
|
||||
│ ├── test_images_generations_lora.py
|
||||
│ └── stage_configs/
|
||||
│ └── test_images_generations_lora.py
|
||||
└── offline_inference/ ✅
|
||||
├── test_qwen2_5_omni.py
|
||||
├── test_qwen3_omni.py
|
||||
@@ -153,11 +152,12 @@ vllm_omni/ tests/
|
||||
├── test_diffusion_lora.py
|
||||
├── test_sequence_parallel.py
|
||||
├── test_qwen_image_edit_expansion.py
|
||||
└── stage_configs/
|
||||
├── qwen2_5_omni_ci.yaml
|
||||
├── qwen3_omni_ci.yaml
|
||||
├── bagel_*.yaml
|
||||
└── stage_configs/ (legacy schema, still present
|
||||
├── bagel_*.yaml for unmigrated models)
|
||||
└── npu/, rocm/, etc.
|
||||
|
||||
# Migrated models (qwen3_omni_moe, qwen2_5_omni, qwen3_tts) live under
|
||||
# vllm_omni/deploy/ instead — see docs/configuration/stage_configs.md.
|
||||
examples/ tests
|
||||
│ └── examples
|
||||
├── online_serving/ → ├── online_serving/
|
||||
@@ -229,6 +229,7 @@ from tests.conftest import (
|
||||
generate_synthetic_video,
|
||||
merge_base64_and_convert_to_text,
|
||||
)
|
||||
from tests.utils import get_deploy_config_path
|
||||
from vllm_omni.platforms import current_omni_platform
|
||||
|
||||
# Edit: model name and stage config path
|
||||
@@ -236,7 +237,7 @@ models = ["Qwen/Qwen3-Omni-30B-A3B-Instruct"]
|
||||
|
||||
#If you use the default configuration file, you can directly use the following address.
|
||||
def get_default_config():
|
||||
return str(Path(__file__).parent.parent / "stage_configs" / "qwen3_omni_ci.yaml")
|
||||
return get_deploy_config_path("ci/qwen3_omni_moe.yaml")
|
||||
|
||||
#If you need to modify the configuration file, you can use modify_stage_config.
|
||||
def get_chunk_config():
|
||||
|
||||
@@ -313,7 +313,7 @@ The registry uses lazy loading, so the model class is imported only when needed.
|
||||
|
||||
## Stage Configuration
|
||||
|
||||
Create a YAML configuration file in `vllm_omni/model_executor/stage_configs/`. For a complete example, see the [Qwen3-Omni configuration file](gh-file:vllm_omni/model_executor/stage_configs/qwen3_omni_moe.yaml).
|
||||
Create a YAML configuration file in `vllm_omni/deploy/`. For a complete example, see the [Qwen3-Omni configuration file](gh-file:vllm_omni/deploy/qwen3_omni_moe.yaml).
|
||||
|
||||
### Key Configuration Fields
|
||||
|
||||
@@ -614,7 +614,7 @@ For a complete reference implementation, see:
|
||||
- **Thinker**: `vllm_omni/model_executor/models/qwen3_omni/qwen3_omni_moe_thinker.py`
|
||||
- **Talker**: `vllm_omni/model_executor/models/qwen3_omni/qwen3_omni_moe_talker.py`
|
||||
- **Code2Wav**: `vllm_omni/model_executor/models/qwen3_omni/qwen3_omni_code2wav.py`
|
||||
- **Stage config**: `vllm_omni/model_executor/stage_configs/qwen3_omni_moe.yaml`
|
||||
- **Stage config**: `vllm_omni/deploy/qwen3_omni_moe.yaml`
|
||||
- **Input processors**: `vllm_omni/model_executor/stage_input_processors/qwen3_omni.py`
|
||||
- **Registry**: `vllm_omni/model_executor/models/registry.py`
|
||||
- **Testing**: `vllm_omni/tests/e2e/offline_inference/test_qwen3_omni.py`
|
||||
|
||||
@@ -120,8 +120,18 @@ vllm_omni/model_executor/stage_configs/
|
||||
| `models/qwen3_tts/qwen3_tts.py` | Unified model class |
|
||||
| `models/qwen3_tts/qwen3_tts_code_predictor_vllm.py` | Stage 0 - optimized AR |
|
||||
| `models/qwen3_tts/qwen3_tts_code2wav.py` | Stage 1 - decoder |
|
||||
| `stage_configs/qwen3_tts.yaml` | Stage config (async_chunk enabled) |
|
||||
| `stage_configs/qwen3_tts_batch.yaml` | Batch mode config |
|
||||
| `deploy/qwen3_tts.yaml` (new schema) | Deploy config (async_chunk enabled) — paired with `models/qwen3_tts/pipeline.py` for the frozen topology |
|
||||
|
||||
> **Chunked vs end-to-end modes**: `qwen3_tts` registers a single
|
||||
> pipeline whose stage 1 declares alternate processor functions — an
|
||||
> `async_chunk_process_next_stage_input_func` (per-chunk streaming, used
|
||||
> when `deploy.async_chunk=True`) and a `sync_process_input_func`
|
||||
> (batch-end, used when `deploy.async_chunk=False`). The loader selects
|
||||
> one at merge time based on the bool, so `--no-async-chunk` alone
|
||||
> switches modes — no variant yaml or variant pipeline registration is
|
||||
> needed. Pipelines that only make sense in one mode (e.g.
|
||||
> `qwen3_omni_moe` is always chunked) can keep using the unconditional
|
||||
> `custom_process_*` fields.
|
||||
| `stage_input_processors/qwen3_tts.py` | Stage transition processors |
|
||||
|
||||
## Step-by-Step Implementation
|
||||
@@ -574,7 +584,8 @@ Adding a TTS model to vLLM-Omni involves:
|
||||
| `models/qwen3_tts/qwen3_tts.py` | Unified model class |
|
||||
| `models/qwen3_tts/qwen3_tts_code_predictor_vllm.py` | AR stage with vLLM fused ops |
|
||||
| `models/qwen3_tts/qwen3_tts_code2wav.py` | Decoder stage with `chunked_decode_streaming()` |
|
||||
| `stage_configs/qwen3_tts.yaml` | Stage configuration |
|
||||
| `models/qwen3_tts/pipeline.py` | Frozen pipeline topology (registered at import time) |
|
||||
| `deploy/qwen3_tts.yaml` | Deploy config (user-editable, async_chunk + SharedMemoryConnector) |
|
||||
| `stage_input_processors/qwen3_tts.py` | Stage transition processors |
|
||||
|
||||
For more information, see:
|
||||
|
||||
@@ -127,10 +127,11 @@ Multi-stage omni serving:
|
||||
```bash
|
||||
vllm serve Qwen/Qwen2.5-Omni-7B \
|
||||
--omni \
|
||||
--stage-configs-path qwen2_5_omni.yaml \
|
||||
--port 8091
|
||||
```
|
||||
|
||||
(The default deploy config at `vllm_omni/deploy/qwen2_5_omni.yaml` is loaded automatically. Pass `--deploy-config /path/to/custom.yaml` to override.)
|
||||
|
||||
Single-stage diffusion serving with torch profiler:
|
||||
|
||||
```bash
|
||||
|
||||
Reference in new issue
Block a user