[Docs] Remove stale stage config references (#6270)
Signed-off-by: zwhzzz0821 <2831474076@qq.com>
This commit is contained in:
1 parent
efaf4276f9
commit
6f55a06fab
85 files changed
+566
-794
No files matched your search
@@ -207,18 +207,16 @@ outputs = omni.generate(
|
||||
)
|
||||
```
|
||||
|
||||
## Stage Configuration
|
||||
## Deploy Configuration
|
||||
|
||||
Use the equivalent engine arguments in a deployment YAML:
|
||||
|
||||
```yaml
|
||||
stage_args:
|
||||
stages:
|
||||
- stage_id: 0
|
||||
stage_type: diffusion
|
||||
engine_args:
|
||||
step_execution: false
|
||||
max_num_seqs: 4
|
||||
request_batch_max_wait_ms: 20
|
||||
step_execution: false
|
||||
max_num_seqs: 4
|
||||
request_batch_max_wait_ms: 20
|
||||
```
|
||||
|
||||
For step execution, set `step_execution: true` and remove
|
||||
|
||||
@@ -24,7 +24,7 @@ stages:
|
||||
|
||||
For an experimental whole-transformer compile scope, set
|
||||
`--diffusion-compile-granularity full` or use
|
||||
`diffusion_compile_granularity: full` in the stage configuration. Full scope may
|
||||
`diffusion_compile_granularity: full` in the deploy configuration. Full scope may
|
||||
still contain graph breaks; it does not force one graph. It is rejected when
|
||||
HSDP, sequence parallelism, CPU offload, or layerwise offload is enabled. Use
|
||||
regional scope with those features.
|
||||
|
||||
@@ -22,7 +22,7 @@ MiMo-Audio provides multiple task variants for audio understanding and generatio
|
||||
|
||||
## Setup
|
||||
|
||||
Please refer to the [stage configuration documentation](https://docs.vllm.ai/projects/vllm-omni/en/latest/configuration/stage_configs/) to configure memory allocation appropriately for your hardware setup.
|
||||
See the [pipeline and deploy configuration documentation](https://docs.vllm.ai/projects/vllm-omni/en/latest/configuration/stage_configs/) to configure memory allocation for your hardware.
|
||||
|
||||
### Environment Variables
|
||||
|
||||
@@ -188,7 +188,7 @@ Note: This task uses hardcoded message lists in the script.
|
||||
|
||||
### Other
|
||||
|
||||
- If the model or stage config fails to load, check [stage configuration documentation](https://docs.vllm.ai/projects/vllm-omni/en/latest/configuration/stage_configs/) for memory and GPU settings.
|
||||
- If the model or deploy config fails to load, check the [pipeline and deploy configuration documentation](https://docs.vllm.ai/projects/vllm-omni/en/latest/configuration/stage_configs/) for memory and GPU settings.
|
||||
- For errors when reading/writing WAV (e.g. unsupported format), ensure input files are standard WAV/MP3 and that `soundfile` is linked to a working libsndfile (see above).
|
||||
|
||||
## Notes
|
||||
|
||||
@@ -4,7 +4,7 @@ Source <https://github.com/vllm-project/vllm-omni/tree/main/examples/offline_inf
|
||||
|
||||
|
||||
## Setup
|
||||
Please refer to the [stage configuration documentation](https://docs.vllm.ai/projects/vllm-omni/en/latest/configuration/stage_configs/) to configure memory allocation appropriately for your hardware setup.
|
||||
See the [pipeline and deploy configuration documentation](https://docs.vllm.ai/projects/vllm-omni/en/latest/configuration/stage_configs/) to configure memory allocation for your hardware.
|
||||
|
||||
## Run examples
|
||||
|
||||
|
||||
@@ -4,7 +4,8 @@ Source <https://github.com/vllm-project/vllm-omni/tree/main/examples/offline_inf
|
||||
|
||||
|
||||
## Setup
|
||||
Please refer to the [stage configuration documentation](https://docs.vllm.ai/projects/vllm-omni/en/latest/configuration/stage_configs/) to configure memory allocation appropriately for your hardware setup.
|
||||
Use `--deploy-config` for deployment overrides such as stage memory allocation.
|
||||
See the [pipeline and deploy configuration documentation](https://docs.vllm.ai/projects/vllm-omni/en/latest/configuration/stage_configs/).
|
||||
|
||||
## Run examples
|
||||
|
||||
@@ -73,8 +74,8 @@ For true stage-level concurrency -- where downstream stages (Talker, Code2Wav)
|
||||
start **before** the upstream stage (Thinker) finishes -- use the async_chunk
|
||||
example. This requires:
|
||||
|
||||
1. A stage config YAML with ``async_chunk: true`` (e.g.
|
||||
``qwen3_omni_moe_async_chunk.yaml``).
|
||||
1. A deploy YAML with ``async_chunk: true`` (for example, an overlay based on
|
||||
``vllm_omni/deploy/qwen3_omni_moe.yaml``).
|
||||
2. Hardware that matches the config (e.g. 2x H100 for the default 3-stage
|
||||
config).
|
||||
|
||||
@@ -100,11 +101,11 @@ bash run_multiple_prompts_async_chunk.sh --max-in-flight 4
|
||||
python end2end_async_chunk.py --query-type text --modalities text
|
||||
```
|
||||
|
||||
#### Custom stage config
|
||||
#### Custom deploy config
|
||||
```bash
|
||||
python end2end_async_chunk.py \
|
||||
--query-type use_audio \
|
||||
--stage-configs-path /path/to/your_async_chunk.yaml
|
||||
--deploy-config /path/to/your_async_chunk.yaml
|
||||
```
|
||||
|
||||
> **Note**: The synchronous ``end2end.py`` (using ``Omni``) is still the
|
||||
|
||||
@@ -69,7 +69,7 @@ python examples/offline_inference/text_to_speech/voxcpm2/end2end.py \
|
||||
|
||||
### Notes
|
||||
- Output: 48 kHz mono WAV.
|
||||
- Stage config: `vllm_omni/model_executor/stage_configs/voxcpm2.yaml` (default).
|
||||
- Deploy config: `vllm_omni/deploy/voxcpm2.yaml` (default).
|
||||
|
||||
---
|
||||
|
||||
@@ -152,7 +152,7 @@ python examples/offline_inference/text_to_speech/fish_speech/end2end.py \
|
||||
--text "Hello, this is a streaming test." \
|
||||
--streaming
|
||||
```
|
||||
Streaming requires `async_chunk: true` in the stage config.
|
||||
Streaming requires `async_chunk: true` in the deploy config.
|
||||
|
||||
### Notes
|
||||
- Output: 44.1 kHz mono WAV.
|
||||
@@ -274,7 +274,7 @@ python examples/offline_inference/text_to_speech/qwen3_tts/end2end.py \
|
||||
--streaming \
|
||||
--output-dir /tmp/out_stream
|
||||
```
|
||||
Streaming requires `async_chunk: true` in the stage config.
|
||||
Streaming requires `async_chunk: true` in the deploy config.
|
||||
|
||||
### Batched decoding
|
||||
The Code2Wav stage supports batched decoding through the SpeechTokenizer. Configure both stages with `max_num_seqs > 1` via `--stage-overrides` and pass multiple prompts via `--txt-prompts`:
|
||||
|
||||
@@ -271,7 +271,7 @@ The script supports the following arguments:
|
||||
vllm serve Qwen/Qwen3-Omni-30B-A3B-Instruct --omni --port 8091
|
||||
```
|
||||
|
||||
If you have custom stage configs file:
|
||||
If you have a custom deploy config file:
|
||||
```bash
|
||||
vllm serve Qwen/Qwen3-Omni-30B-A3B-Instruct --omni --port 8091 --deploy-config /path/to/deploy_config_file
|
||||
```
|
||||
|
||||
@@ -419,7 +419,7 @@ curl -X POST http://localhost:8091/v1/audio/speech \
|
||||
"response_format": "pcm"
|
||||
}' --no-buffer | play -t raw -r 24000 -e signed -b 16 -c 1 -
|
||||
```
|
||||
Raw PCM streaming requires `stream_format="audio"`, `response_format="pcm"`, and `async_chunk: true` on the stage config (default in `qwen3_tts.yaml`). `speed` is not supported when streaming.
|
||||
Raw PCM streaming requires `stream_format="audio"`, `response_format="pcm"`, and `async_chunk: true` in the deploy config (default in `qwen3_tts.yaml`). `speed` is not supported when streaming.
|
||||
|
||||
### Streaming WebSocket
|
||||
The `/v1/audio/speech/stream` endpoint accepts text incrementally and synthesizes the buffered text as one continuous request on `input.done`:
|
||||
|
||||
@@ -92,7 +92,7 @@ warmup_quack_fp8([(14040, 2048, 6144), (14040, 2048, 2048)])
|
||||
| FLUX.1 | `black-forest-labs/FLUX.1-dev`, `black-forest-labs/FLUX.1-schnell` | Yes | Yes | All layers | None | |
|
||||
| FLUX.2-dev | `black-forest-labs/FLUX.2-dev` | Yes | Not validated | All layers | None | ✅︎ |
|
||||
| FLUX.2-klein | `black-forest-labs/FLUX.2-klein-4B` | Yes | Yes | All layers | None | |
|
||||
| HunyuanImage-3.0 | `tencent/HunyuanImage-3.0`, `tencent/HunyuanImage-3.0-Instruct` | Yes | Yes | All layers; use the Hunyuan stage config for multi-stage runs | None | |
|
||||
| HunyuanImage-3.0 | `tencent/HunyuanImage-3.0`, `tencent/HunyuanImage-3.0-Instruct` | Yes | Yes | All layers; use the Hunyuan deploy config for multi-stage runs | None | |
|
||||
| HunyuanVideo-1.5 | `hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-480p_t2v`, `720p_t2v`, `480p_i2v` | Yes | Yes | All layers | None | |
|
||||
| Cosmos3 | `nvidia/Cosmos3-Nano`, `nvidia/Cosmos3-Super` | Yes | Not validated | All layers | None | |
|
||||
| MiniMax-H3 | `MiniMaxAI/MiniMax-H3` (`FL2VA` / `Ref2VA`) | Yes | Not validated | `quantization="fp8"` quantizes eligible DiT and text-encoder linears; mixed-precision input/output heads stay FP32 | None | ✅︎ |
|
||||
|
||||
@@ -92,4 +92,4 @@ For HunyuanImage-3.0, use the Hunyuan-specific adaptation linked above.
|
||||
2. The `ascend` quantization method expects weights produced by the Ascend
|
||||
tooling; it is not a load-time CUDA quantizer.
|
||||
3. Keep the quantized checkpoint aligned with the same model architecture and
|
||||
stage config used for BF16 inference.
|
||||
pipeline and deploy config used for BF16 inference.
|
||||
@@ -83,23 +83,23 @@ Online serving:
|
||||
vllm serve <your-model> --omni --diffusion-kv-cache-dtype fp8
|
||||
```
|
||||
|
||||
Stage config:
|
||||
Deploy config:
|
||||
|
||||
```yaml
|
||||
stage_args:
|
||||
stages:
|
||||
- stage_id: 0
|
||||
stage_type: diffusion
|
||||
engine_args:
|
||||
model_stage: dit
|
||||
diffusion_kv_cache_dtype: "fp8"
|
||||
diffusion_kv_cache_skip_steps: "0,1"
|
||||
diffusion_kv_cache_skip_layers: "0-2"
|
||||
diffusion_kv_cache_dtype: "fp8"
|
||||
diffusion_kv_cache_skip_steps: "0,1"
|
||||
diffusion_kv_cache_skip_layers: "0-2"
|
||||
```
|
||||
|
||||
Legacy YAML keys `kv_cache_dtype`, `kv_cache_skip_steps`, and
|
||||
`kv_cache_skip_layers` are still accepted when constructing
|
||||
`OmniDiffusionConfig` (for example via `from_kwargs`); prefer the `diffusion_*`
|
||||
names for new configs.
|
||||
The `model_stage` and diffusion execution type belong to the registered
|
||||
`PipelineConfig`; the deploy YAML only carries runtime overrides.
|
||||
|
||||
The legacy keyword aliases `kv_cache_dtype`, `kv_cache_skip_steps`, and
|
||||
`kv_cache_skip_layers` remain accepted when constructing
|
||||
`OmniDiffusionConfig` directly. They are not deploy YAML fields; prefer the
|
||||
`diffusion_*` names for new code.
|
||||
|
||||
## Parameters
|
||||
|
||||
|
||||
Reference in new issue
Block a user