anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

[Docs] Remove stale stage config references (#6270)

Browse Source 
Signed-off-by: zwhzzz0821 <2831474076@qq.com>
This commit is contained in:
Yukim1 authored and GitHub committed 2026-08-20 07:09:12 +08:00
1 parent efaf4276f9
commit 6f55a06fab
85 files changed
+566 -794
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/contributing/DOCS_GUIDE.md
+1 -1
View File
@@ -46,7 +46,7 @@ class Omni:
Args:
model: Model name or path
stage_configs: Optional stage configurations
deploy_config: Optional path to a deploy configuration
**kwargs: Additional arguments passed to the engine
Example:
docs/contributing/ci/test_examples/l4_performance_tests.inc.md
+6 -6
View File
@@ -31,7 +31,7 @@ Pass **`--test-config-file`** to run one JSON file as-is, or omit it for the bul
],
"server_params": {
"model": "Qwen/Qwen3-Omni-30B-A3B-Instruct",
"stage_config_name": "qwen3_omni.yaml"
"stage_config_name": "qwen3_omni_moe.yaml"
},
"benchmark_params": [
{
@@ -155,7 +155,7 @@ See also [Markers for Tests](#markers-for-tests) for registered hardware markers
| Parameter | Required | Example | Description |
| ----------------- | -------- | ---------------------------------- | ----------------------------- |
| model | Yes | "Qwen/Qwen3-Omni-30B-A3B-Instruct" | Model name or path |
| stage_config_name | Yes | "qwen3_omni.yaml" | Stage configuration file name |
| stage_config_name | Yes | "qwen3_omni_moe.yaml" | Deploy configuration file name |
*Dynamic Configuration (update/delete)*
@@ -171,15 +171,15 @@ Supports incremental modifications based on the basic configuration:
```
"update": {
"async_chunk": true, // Enable asynchronous chunk processing
"stage_args": {
"stages": {
"0": {
"engine_args.custom_process_next_stage_input_func": "vllm_omni.model_executor.stage_input_processors.qwen3_omni.thinker2talker_async_chunk"
"max_num_seqs": 64
}
}
},
"delete": {
"stage_args": {
"2": ["custom_process_input_func"] // Delete this configuration for stage 2
"stages": {
"2": ["max_num_batched_tokens"] // Delete this deploy override for stage 2
}
}
```
docs/contributing/ci/test_writing_guide.md
+6 -6
View File
@@ -17,7 +17,7 @@ Summary:
- **Reliability:** `tests/dfx/reliability/`
- Do **not** add new top-level `tests/` directories unrelated to a component (or to `e2e` / `dfx` / `helpers` / `examples` / `buildkite`)
Deploy YAMLs: `vllm_omni/deploy/` via `tests.helpers.stage_config.get_deploy_config_path` (see [stage configs](../../configuration/stage_configs.md)).
Deploy YAMLs: `vllm_omni/deploy/` via `tests.helpers.stage_config.get_deploy_config_path` (see [pipeline and deploy configs](../../configuration/stage_configs.md)).
## Markers for Tests
@@ -581,7 +581,7 @@ When you want to add L5-level stability test cases, add or extend the appropriat
"test_name": "test_qwen3_omni_stability",
"server_params": {
"model": "Qwen/Qwen3-Omni-30B-A3B-Instruct",
"stage_config_name": "qwen3_omni.yaml"
"stage_config_name": "qwen3_omni_moe.yaml"
},
"benchmark_params": [
{
@@ -607,7 +607,7 @@ When you want to add L5-level stability test cases, add or extend the appropriat
| Field | Required | Description |
| ---------------- | -------- | --------------------------------------------------------------------------- |
| test_name | Yes | Unique identifier for the stability test case |
| server_params | Yes | Server-side configuration parameters (model, stage configuration, etc.) |
| server_params | Yes | Server-side configuration parameters (model, deploy configuration, etc.) |
| benchmark_params | Yes | Stability benchmark running parameters (supports multiple configurations) |
##### server_params Configuration
@@ -617,7 +617,7 @@ When you want to add L5-level stability test cases, add or extend the appropriat
| Parameter | Required | Example | Description |
| ----------------- | -------- | ---------------------------------- | ----------------------------------- |
| model | Yes | "Qwen/Qwen3-Omni-30B-A3B-Instruct" | Model name or path |
| stage_config_name | Yes | "qwen3_omni.yaml" | Stage configuration file name |
| stage_config_name | Yes | "qwen3_omni_moe.yaml" | Deploy configuration file name |
###### Dynamic Configuration (update/delete)
@@ -697,8 +697,8 @@ pytest -s -v tests/dfx/reliability/test_reliability_hunyuan_image.py -m slow
##### Adding a new model suite
1. Add `test_reliability_<model>.py` under `tests/dfx/reliability/`.
2. Define **`RELIABILITY_SCENARIOS`** and pass them through **`create_reliability_omni_server_params()`** with the correct deploy or e2e stage-config directory (same pattern as existing files).
3. Reuse **`helpers`** for OOM / kill / raw HTTP; prefer **`assert_fault_exception()`** and **`resolve_oom_device_spec()`** from `tests/dfx/conftest.py` for consistent device selection vs stage YAML.
2. Define **`RELIABILITY_SCENARIOS`** and pass them through **`create_reliability_omni_server_params()`** with the correct deploy or e2e deploy-config directory (same pattern as existing files).
3. Reuse **`helpers`** for OOM / kill / raw HTTP; prefer **`assert_fault_exception()`** and **`resolve_oom_device_spec()`** from `tests/dfx/conftest.py` for consistent device selection vs deploy YAML.
4. Register **`slow`** (and **`hardware_test`** if needed); extend **`.buildkite/cuda/test-weekly.yml`** when the suite should run in weekly L5.
</details>
docs/contributing/model/adding_omni_model.md
+25 -14
View File
@@ -9,7 +9,7 @@ This guide walks through the process of adding a new multi-stage model to vLLM-O
3. [Step-by-Step Implementation](#step-by-step-implementation)
4. [Key Components](#key-components)
5. [Model Registration](#model-registration)
6. [Stage Configuration](#stage-configuration)
6. [Pipeline and Deploy Configuration](#pipeline-and-deploy-configuration)
7. [Stage Input Processors](#stage-input-processors)
8. [Testing](#testing)
9. [Adding a Model Recipe](#adding-a-model-recipe)
@@ -316,18 +316,28 @@ _OMNI_MODELS = {
The registry uses lazy loading, so the model class is imported only when needed.
## Stage Configuration
## Pipeline and Deploy Configuration
Create a YAML configuration file in `vllm_omni/deploy/`. For a complete example, see the [Qwen3-Omni configuration file](gh-file:vllm_omni/deploy/qwen3_omni_moe.yaml).
Define fixed topology in `vllm_omni/model_executor/models/<model>/pipeline.py`
with `PipelineConfig` and `StagePipelineConfig`, then register it in
`vllm_omni/config/pipeline_registry.py`. For a complete example, see the
[Qwen3-Omni pipeline](gh-file:vllm_omni/model_executor/models/qwen3_omni/pipeline.py).
### Key Configuration Fields
### Pipeline topology fields
- **`model_stage`**: Which stage to run ("thinker", "talker", "code2wav", etc.)
- **`model_arch`**: The model architecture name (must match registry)
- **`engine_input_source`**: List of stage IDs that provide input to this stage
- **`custom_process_input_func`**: Function to process inputs from previous stages
- **`final_output`**: Whether this stage produces the final output (True/False)
- **`final_output_type`**: Type of final output ("text", "audio", "image", etc.)
- **`model_stage`**: Logical stage name ("thinker", "talker", "code2wav", etc.)
- **`execution_type`**: Runtime family (`LLM_AR`, `LLM_GENERATION`, or `DIFFUSION`)
- **`model_arch`**: Stage architecture name, when different from the pipeline default
- **`input_sources`**: Upstream stage IDs that provide input to this stage
- **`custom_process_input_func`**: Function that processes inputs from previous stages
- **`final_output`**: Whether this stage produces user-facing output
- **`final_output_type`**: Output modality ("text", "audio", "image", etc.)
- **`owns_tokenizer`**: Whether this stage owns the pipeline tokenizer
Create the matching runtime defaults in `vllm_omni/deploy/<model>.yaml` and set
`PipelineConfig.default_deploy_config_name` to that file. The deploy YAML owns
GPU placement, memory sizing, parallelism, connectors, and sampling defaults;
it must not redefine topology fields.
## Stage Input Processors
@@ -374,7 +384,7 @@ The stage transition process follows these steps:
2. **Transition Detection**: The orchestrator checks if there's a next stage and calls `process_engine_inputs()` on it
3. **Input Processing**: The stage input processor configured in stage YAML (under `vllm_omni/model_executor/stage_input_processors/`) handles the transition:
3. **Input Processing**: The stage input processor registered by the pipeline (implemented under `vllm_omni/model_executor/stage_input_processors/`) handles the transition:
```python
def process_engine_inputs(
self, stage_list: list[Any], prompt: OmniTokensPrompt | TextPrompt = None
@@ -577,7 +587,7 @@ Adding a new model to vLLM-Omni involves:
1. **Create model directory structure** with stage implementations
2. **Implement unified model class** that orchestrates stages
3. **Register model** in `registry.py`
4. **Create stage configuration** YAML file
4. **Define pipeline topology and deploy defaults** in `pipeline.py` and `vllm_omni/deploy/`
5. **Implement stage input processors** for stage transitions
6. **Write tests** to verify functionality
7. **Add model recipe** to the [vllm-project/recipes](https://github.com/vllm-project/recipes) repository (see [Adding a Model Recipe](#adding-a-model-recipe) section)
@@ -590,7 +600,8 @@ For a complete reference implementation, see:
- **Thinker**: `vllm_omni/model_executor/models/qwen3_omni/qwen3_omni_moe_thinker.py`
- **Talker**: `vllm_omni/model_executor/models/qwen3_omni/qwen3_omni_moe_talker.py`
- **Code2Wav**: `vllm_omni/model_executor/models/qwen3_omni/qwen3_omni_code2wav.py`
- **Stage config**: `vllm_omni/deploy/qwen3_omni_moe.yaml`
- **Pipeline topology**: `vllm_omni/model_executor/models/qwen3_omni/pipeline.py`
- **Deploy config**: `vllm_omni/deploy/qwen3_omni_moe.yaml`
- **Input processors**: `vllm_omni/model_executor/stage_input_processors/qwen3_omni.py`
- **Registry**: `vllm_omni/model_executor/models/registry.py`
- **Testing**: `vllm_omni/tests/e2e/offline_inference/test_qwen3_omni.py`
@@ -598,4 +609,4 @@ For a complete reference implementation, see:
For more information, see:
- [Architecture Overview](../../design/architecture_overview.md)
- [Supported Models](../../models/supported_models.md)
- [Stage Configuration Guide](../../configuration/stage_configs.md)
- [Pipeline and Deploy Configuration Guide](../../configuration/stage_configs.md)
docs/contributing/model/adding_tts_model.md
+92 -103
View File
@@ -20,7 +20,7 @@ pattern, refer to MOSS-TTS-Nano.
4. [Step-by-Step Implementation](#step-by-step-implementation)
5. [Key Components](#key-components)
6. [Model Registration](#model-registration)
7. [Stage Configuration](#stage-configuration)
7. [Pipeline and Deploy Configuration](#pipeline-and-deploy-configuration)
8. [Stage Input Processors](#stage-input-processors)
9. [Online Serving Integration](#online-serving-integration)
10. [Single-Stage Models](#single-stage-models)
@@ -427,112 +427,93 @@ _OMNI_MODELS = {
The registry uses lazy loading - model classes are only imported when needed.
## Stage Configuration
## Pipeline and Deploy Configuration
Each stage has a `worker_type` that determines how it is scheduled:
Define fixed topology in `models/<model>/pipeline.py`. The execution type
selects the scheduler and worker family:
- `worker_type: ar` - autoregressive stage, uses `OmniARScheduler` with PagedAttention
- `worker_type: generation` - non-AR stage (e.g. decoder), uses `OmniGenerationScheduler`
- `StageExecutionType.LLM_AR` uses the autoregressive runtime with PagedAttention.
- `StageExecutionType.LLM_GENERATION` uses the generation runtime for stages such as audio decoders.
Key configuration fields:
Key topology fields belong to `StagePipelineConfig`:
| Field | Description |
|-------|-------------|
| `model_stage` | Which stage to initialize (`ar_stage`, `decoder`, etc.) |
| `model_arch` | Architecture name, must match `registry.py` |
| `engine_input_source` | List of upstream stage IDs that provide input (e.g. `[0]`) |
| `engine_output_type` | Output type: `latent` for intermediate, `audio` for final |
| `custom_process_next_stage_input_func` | Async chunk processor function path (streaming only) |
| `final_output` | Whether this stage produces the final user-facing output |
| `final_output_type` | Type of final output (`audio`, `text`, etc.) |
| `model_stage` | Logical stage name (`ar_stage`, `decoder`, etc.). |
| `execution_type` | Runtime family for this stage. |
| `input_sources` | Upstream stage IDs that provide input. |
| `model_arch` | Stage-specific architecture name, when it differs from the pipeline default. |
| `engine_output_type` | Output representation such as `latent` or `audio`. |
| `custom_process_next_stage_input_func` | Processor for full-payload stage handoff. |
| `async_chunk_process_next_stage_input_func` | Processor for streaming chunk handoff. |
| `final_output` / `final_output_type` | Whether this stage produces user-facing output and its modality. |
| `owns_tokenizer` | Whether this stage supplies the pipeline tokenizer. |
!!! note
New in-tree models should define frozen topology in `models/<model>/pipeline.py`,
register it in `vllm_omni/config/pipeline_registry.py`, and put deployment
knobs in `vllm_omni/deploy/<model>.yaml`. Use `--deploy-config` to load a
custom deployment YAML.
For example, a two-stage TTS topology can be defined as:
### Batch mode
```python
from vllm_omni.config.stage_config import (
PipelineConfig,
StageExecutionType,
StagePipelineConfig,
)
```yaml
# stage_configs/your_model_name.yaml
_PROC = "vllm_omni.model_executor.stage_input_processors.your_model_name"
stage_args:
- stage_id: 0
stage_type: llm
runtime:
devices: "0"
engine_args:
model_stage: ar_stage
max_num_seqs: 64
model_arch: YourTTSModelForConditionalGeneration
worker_type: ar
scheduler_cls: vllm_omni.core.sched.omni_ar_scheduler.OmniARScheduler
engine_output_type: latent
default_sampling_params:
temperature: 0.9
top_k: 50
max_tokens: 2048
- stage_id: 1
stage_type: llm
runtime:
devices: "0"
engine_args:
model_stage: decoder
model_arch: YourTTSModelForConditionalGeneration
worker_type: generation
scheduler_cls: vllm_omni.core.sched.omni_generation_scheduler.OmniGenerationScheduler
engine_output_type: audio
engine_input_source: [0]
final_output: true
final_output_type: audio
YOUR_TTS_PIPELINE = PipelineConfig(
model_type="your_model_name",
default_deploy_config_name="your_model_name.yaml",
model_arch="YourTTSModelForConditionalGeneration",
stages=(
StagePipelineConfig(
stage_id=0,
model_stage="ar_stage",
execution_type=StageExecutionType.LLM_AR,
input_sources=(),
owns_tokenizer=True,
engine_output_type="latent",
custom_process_next_stage_input_func=f"{_PROC}.ar2decoder",
async_chunk_process_next_stage_input_func=f"{_PROC}.ar2decoder_async_chunk",
),
StagePipelineConfig(
stage_id=1,
model_stage="decoder",
execution_type=StageExecutionType.LLM_GENERATION,
input_sources=(0,),
model_arch="YourTTSDecoder",
engine_output_type="audio",
final_output=True,
final_output_type="audio",
),
),
)
```
### Streaming mode (async_chunk)
Add `async_chunk: true` at the top level and specify `custom_process_next_stage_input_func`
on Stage 0 to define how intermediate outputs are chunked and forwarded:
Register the pipeline in `vllm_omni/config/pipeline_registry.py`. Runtime
placement and sizing belong in `vllm_omni/deploy/your_model_name.yaml`:
```yaml
# stage_configs/your_model_name_async_chunk.yaml
async_chunk: true
stage_args:
stages:
- stage_id: 0
stage_type: llm
runtime:
devices: "0"
engine_args:
model_stage: ar_stage
max_num_seqs: 64
model_arch: YourTTSModelForConditionalGeneration
worker_type: ar
scheduler_cls: vllm_omni.core.sched.omni_ar_scheduler.OmniARScheduler
engine_output_type: latent
custom_process_next_stage_input_func: >
vllm_omni.model_executor.stage_input_processors.your_model_name.ar2decoder_async_chunk
devices: "0"
max_num_seqs: 64
default_sampling_params:
temperature: 0.9
top_k: 50
max_tokens: 2048
- stage_id: 1
stage_type: llm
runtime:
devices: "0"
engine_args:
model_stage: decoder
model_arch: YourTTSModelForConditionalGeneration
worker_type: generation
scheduler_cls: vllm_omni.core.sched.omni_generation_scheduler.OmniGenerationScheduler
engine_output_type: audio
engine_input_source: [0]
final_output: true
final_output_type: audio
devices: "0"
max_num_seqs: 1
```
Set `async_chunk: false` for end-to-end handoff. The registered pipeline
selects the full-payload or async chunk processor; processor paths and topology
must not be moved into the deploy YAML. Use `--deploy-config` to load a custom
deployment overlay.
## Stage Input Processors
Stage input processors convert Stage 0 outputs into Stage 1 inputs. Create yours in
@@ -832,32 +813,40 @@ vllm_omni/deploy/your_model_name.yaml
No stage input processor is needed.
### Stage config
### Pipeline and deploy config
Use a single stage with `worker_type: ar`. The `is_comprehension: true` field and the
top-level `async_chunk: false` are required — omitting them causes silent
misclassification in the serving layer. Set `max_num_seqs` to at least 4 for
concurrent production use.
Use a single `LLM_AR` stage that owns the tokenizer and produces final audio:
```python
YOUR_TTS_PIPELINE = PipelineConfig(
model_type="your_model_name",
default_deploy_config_name="your_model_name.yaml",
model_arch="YourModelForCausalLM",
stages=(
StagePipelineConfig(
stage_id=0,
model_stage="your_model_stage_key",
execution_type=StageExecutionType.LLM_AR,
input_sources=(),
owns_tokenizer=True,
engine_output_type="audio",
final_output=True,
final_output_type="audio",
),
),
)
```
The deploy YAML controls runtime behavior. Set `max_num_seqs` to at least 4
for concurrent production use:
```yaml
# stage_configs/your_model_name.yaml
async_chunk: false
stage_args:
stages:
- stage_id: 0
stage_type: llm
is_comprehension: true # required for serving_speech.py dispatch
runtime:
devices: "0"
engine_args:
model_stage: your_model_stage_key
model_arch: YourModelForCausalLM
worker_type: ar
scheduler_cls: vllm_omni.core.sched.omni_ar_scheduler.OmniARScheduler
engine_output_type: audio
max_num_seqs: 4 # min 4 for concurrent requests; default 1 causes gaps
final_output: true
final_output_type: audio
devices: "0"
max_num_seqs: 4
```
### Generator-based streaming pattern
@@ -1031,4 +1020,4 @@ For more information, see:
- [Architecture Overview](../../design/architecture_overview.md)
- [Async Chunk Design](../../design/feature/async_chunk.md)
- [Stage Configuration Guide](../../configuration/stage_configs.md)
- [Pipeline and Deploy Configuration Guide](../../configuration/stage_configs.md)