anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

[Docs] Remove stale stage config references (#6270)

Browse Source 
Signed-off-by: zwhzzz0821 <2831474076@qq.com>
This commit is contained in:
Yukim1 authored and GitHub committed 2026-08-20 07:09:12 +08:00
1 parent efaf4276f9
commit 6f55a06fab
85 files changed
+566 -794
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/contributing/model/adding_tts_model.md
+92 -103
View File
@@ -20,7 +20,7 @@ pattern, refer to MOSS-TTS-Nano.
4. [Step-by-Step Implementation](#step-by-step-implementation)
5. [Key Components](#key-components)
6. [Model Registration](#model-registration)
7. [Stage Configuration](#stage-configuration)
7. [Pipeline and Deploy Configuration](#pipeline-and-deploy-configuration)
8. [Stage Input Processors](#stage-input-processors)
9. [Online Serving Integration](#online-serving-integration)
10. [Single-Stage Models](#single-stage-models)
@@ -427,112 +427,93 @@ _OMNI_MODELS = {
The registry uses lazy loading - model classes are only imported when needed.
## Stage Configuration
## Pipeline and Deploy Configuration
Each stage has a `worker_type` that determines how it is scheduled:
Define fixed topology in `models/<model>/pipeline.py`. The execution type
selects the scheduler and worker family:
- `worker_type: ar` - autoregressive stage, uses `OmniARScheduler` with PagedAttention
- `worker_type: generation` - non-AR stage (e.g. decoder), uses `OmniGenerationScheduler`
- `StageExecutionType.LLM_AR` uses the autoregressive runtime with PagedAttention.
- `StageExecutionType.LLM_GENERATION` uses the generation runtime for stages such as audio decoders.
Key configuration fields:
Key topology fields belong to `StagePipelineConfig`:
| Field | Description |
|-------|-------------|
| `model_stage` | Which stage to initialize (`ar_stage`, `decoder`, etc.) |
| `model_arch` | Architecture name, must match `registry.py` |
| `engine_input_source` | List of upstream stage IDs that provide input (e.g. `[0]`) |
| `engine_output_type` | Output type: `latent` for intermediate, `audio` for final |
| `custom_process_next_stage_input_func` | Async chunk processor function path (streaming only) |
| `final_output` | Whether this stage produces the final user-facing output |
| `final_output_type` | Type of final output (`audio`, `text`, etc.) |
| `model_stage` | Logical stage name (`ar_stage`, `decoder`, etc.). |
| `execution_type` | Runtime family for this stage. |
| `input_sources` | Upstream stage IDs that provide input. |
| `model_arch` | Stage-specific architecture name, when it differs from the pipeline default. |
| `engine_output_type` | Output representation such as `latent` or `audio`. |
| `custom_process_next_stage_input_func` | Processor for full-payload stage handoff. |
| `async_chunk_process_next_stage_input_func` | Processor for streaming chunk handoff. |
| `final_output` / `final_output_type` | Whether this stage produces user-facing output and its modality. |
| `owns_tokenizer` | Whether this stage supplies the pipeline tokenizer. |
!!! note
New in-tree models should define frozen topology in `models/<model>/pipeline.py`,
register it in `vllm_omni/config/pipeline_registry.py`, and put deployment
knobs in `vllm_omni/deploy/<model>.yaml`. Use `--deploy-config` to load a
custom deployment YAML.
For example, a two-stage TTS topology can be defined as:
### Batch mode
```python
from vllm_omni.config.stage_config import (
PipelineConfig,
StageExecutionType,
StagePipelineConfig,
)
```yaml
# stage_configs/your_model_name.yaml
_PROC = "vllm_omni.model_executor.stage_input_processors.your_model_name"
stage_args:
- stage_id: 0
stage_type: llm
runtime:
devices: "0"
engine_args:
model_stage: ar_stage
max_num_seqs: 64
model_arch: YourTTSModelForConditionalGeneration
worker_type: ar
scheduler_cls: vllm_omni.core.sched.omni_ar_scheduler.OmniARScheduler
engine_output_type: latent
default_sampling_params:
temperature: 0.9
top_k: 50
max_tokens: 2048
- stage_id: 1
stage_type: llm
runtime:
devices: "0"
engine_args:
model_stage: decoder
model_arch: YourTTSModelForConditionalGeneration
worker_type: generation
scheduler_cls: vllm_omni.core.sched.omni_generation_scheduler.OmniGenerationScheduler
engine_output_type: audio
engine_input_source: [0]
final_output: true
final_output_type: audio
YOUR_TTS_PIPELINE = PipelineConfig(
model_type="your_model_name",
default_deploy_config_name="your_model_name.yaml",
model_arch="YourTTSModelForConditionalGeneration",
stages=(
StagePipelineConfig(
stage_id=0,
model_stage="ar_stage",
execution_type=StageExecutionType.LLM_AR,
input_sources=(),
owns_tokenizer=True,
engine_output_type="latent",
custom_process_next_stage_input_func=f"{_PROC}.ar2decoder",
async_chunk_process_next_stage_input_func=f"{_PROC}.ar2decoder_async_chunk",
),
StagePipelineConfig(
stage_id=1,
model_stage="decoder",
execution_type=StageExecutionType.LLM_GENERATION,
input_sources=(0,),
model_arch="YourTTSDecoder",
engine_output_type="audio",
final_output=True,
final_output_type="audio",
),
),
)
```
### Streaming mode (async_chunk)
Add `async_chunk: true` at the top level and specify `custom_process_next_stage_input_func`
on Stage 0 to define how intermediate outputs are chunked and forwarded:
Register the pipeline in `vllm_omni/config/pipeline_registry.py`. Runtime
placement and sizing belong in `vllm_omni/deploy/your_model_name.yaml`:
```yaml
# stage_configs/your_model_name_async_chunk.yaml
async_chunk: true
stage_args:
stages:
- stage_id: 0
stage_type: llm
runtime:
devices: "0"
engine_args:
model_stage: ar_stage
max_num_seqs: 64
model_arch: YourTTSModelForConditionalGeneration
worker_type: ar
scheduler_cls: vllm_omni.core.sched.omni_ar_scheduler.OmniARScheduler
engine_output_type: latent
custom_process_next_stage_input_func: >
vllm_omni.model_executor.stage_input_processors.your_model_name.ar2decoder_async_chunk
devices: "0"
max_num_seqs: 64
default_sampling_params:
temperature: 0.9
top_k: 50
max_tokens: 2048
- stage_id: 1
stage_type: llm
runtime:
devices: "0"
engine_args:
model_stage: decoder
model_arch: YourTTSModelForConditionalGeneration
worker_type: generation
scheduler_cls: vllm_omni.core.sched.omni_generation_scheduler.OmniGenerationScheduler
engine_output_type: audio
engine_input_source: [0]
final_output: true
final_output_type: audio
devices: "0"
max_num_seqs: 1
```
Set `async_chunk: false` for end-to-end handoff. The registered pipeline
selects the full-payload or async chunk processor; processor paths and topology
must not be moved into the deploy YAML. Use `--deploy-config` to load a custom
deployment overlay.
## Stage Input Processors
Stage input processors convert Stage 0 outputs into Stage 1 inputs. Create yours in
@@ -832,32 +813,40 @@ vllm_omni/deploy/your_model_name.yaml
No stage input processor is needed.
### Stage config
### Pipeline and deploy config
Use a single stage with `worker_type: ar`. The `is_comprehension: true` field and the
top-level `async_chunk: false` are required — omitting them causes silent
misclassification in the serving layer. Set `max_num_seqs` to at least 4 for
concurrent production use.
Use a single `LLM_AR` stage that owns the tokenizer and produces final audio:
```python
YOUR_TTS_PIPELINE = PipelineConfig(
model_type="your_model_name",
default_deploy_config_name="your_model_name.yaml",
model_arch="YourModelForCausalLM",
stages=(
StagePipelineConfig(
stage_id=0,
model_stage="your_model_stage_key",
execution_type=StageExecutionType.LLM_AR,
input_sources=(),
owns_tokenizer=True,
engine_output_type="audio",
final_output=True,
final_output_type="audio",
),
),
)
```
The deploy YAML controls runtime behavior. Set `max_num_seqs` to at least 4
for concurrent production use:
```yaml
# stage_configs/your_model_name.yaml
async_chunk: false
stage_args:
stages:
- stage_id: 0
stage_type: llm
is_comprehension: true # required for serving_speech.py dispatch
runtime:
devices: "0"
engine_args:
model_stage: your_model_stage_key
model_arch: YourModelForCausalLM
worker_type: ar
scheduler_cls: vllm_omni.core.sched.omni_ar_scheduler.OmniARScheduler
engine_output_type: audio
max_num_seqs: 4 # min 4 for concurrent requests; default 1 causes gaps
final_output: true
final_output_type: audio
devices: "0"
max_num_seqs: 4
```
### Generator-based streaming pattern
@@ -1031,4 +1020,4 @@ For more information, see:
- [Architecture Overview](../../design/architecture_overview.md)
- [Async Chunk Design](../../design/feature/async_chunk.md)
- [Stage Configuration Guide](../../configuration/stage_configs.md)
- [Pipeline and Deploy Configuration Guide](../../configuration/stage_configs.md)