docs: restore BitsAndBytes navigation

Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>

docs/.nav.yml

@@ -40,30 +40,12 @@ nav:
   - Models:
     - models/supported_models.md
   - Features:
-    - Sleep Mode: features/sleep_mode.md
+    - Overview: features/README.md
-    - Diffusion Features:
+    - Runtime and Stage Execution:
-      - Overview: user_guide/diffusion_features.md
+      - Execution Modes and Streaming: user_guide/diffusion/execution_modes.md
-      - Feature Compatibility: user_guide/feature_compatibility.md
+      - Sleep Mode: features/sleep_mode.md
-      - Regional Compilation: user_guide/diffusion/regional_compilation.md
-      - Cache Acceleration:
-        - TeaCache: user_guide/diffusion/cache_acceleration/teacache.md
-        - Cache-DiT: user_guide/diffusion/cache_acceleration/cache_dit.md
-      - Attention Backends: user_guide/diffusion/attention_backends.md
-      - Frame Interpolation: user_guide/diffusion/frame_interpolation.md
-      - Parallelism:
-        - Overview: user_guide/diffusion/parallelism/overview.md
-        - CFG Parallel: user_guide/diffusion/parallelism/cfg_parallel.md
-        - Expert Parallel: user_guide/diffusion/parallelism/expert_parallel.md
-        - Hybrid Sharded Data Parallel: user_guide/diffusion/parallelism/hsdp.md
-        - Sequence Parallel: user_guide/diffusion/parallelism/sequence_parallel.md
-        - Tensor Parallel: user_guide/diffusion/parallelism/tensor_parallel.md
-        - VAE Parallelism: user_guide/diffusion/parallelism/vae_parallelism.md
-      - CPU Offloading: user_guide/diffusion/cpu_offload.md
-      - LoRA: user_guide/diffusion/lora.md
-      - Custom Pipeline: features/custom_pipeline.md
-      - Execution Modes: user_guide/diffusion/execution_modes.md
     - Quantization:
-      - Quantization: user_guide/quantization/overview.md
+      - Overview: user_guide/quantization/overview.md
       - Online Quantization: user_guide/quantization/online.md
       - Quantized KV Cache: user_guide/quantization/quantized_kvcache.md
       - FP8 W8A8: user_guide/quantization/fp8.md
@@ -72,9 +54,33 @@ nav:
       - MXFP8 W8A8: user_guide/quantization/mxfp8.md
       - MXFP4 W4A4: user_guide/quantization/mxfp4.md
       - ModelOpt: user_guide/quantization/modelopt.md
-      - GGUF: user_guide/quantization/gguf.md
       - AutoRound: user_guide/quantization/autoround.md
       - msModelSlim: user_guide/quantization/msmodelslim.md
+    - Diffusion Acceleration:
+      - Overview and Compatibility:
+        - Overview: user_guide/diffusion_features.md
+        - Feature Compatibility: user_guide/feature_compatibility.md
+      - CPU Offloading: user_guide/diffusion/cpu_offload.md
+      - Cache Acceleration:
+        - TeaCache: user_guide/diffusion/cache_acceleration/teacache.md
+        - Cache-DiT: user_guide/diffusion/cache_acceleration/cache_dit.md
+      - Parallelism:
+        - Overview: user_guide/diffusion/parallelism/overview.md
+        - CFG Parallel: user_guide/diffusion/parallelism/cfg_parallel.md
+        - Expert Parallel: user_guide/diffusion/parallelism/expert_parallel.md
+        - Hybrid Sharded Data Parallel: user_guide/diffusion/parallelism/hsdp.md
+        - Sequence Parallel: user_guide/diffusion/parallelism/sequence_parallel.md
+        - Tensor Parallel: user_guide/diffusion/parallelism/tensor_parallel.md
+        - VAE Parallelism: user_guide/diffusion/parallelism/vae_parallelism.md
+      - Attention Backends: user_guide/diffusion/attention_backends.md
+      - Regional Compilation: user_guide/diffusion/regional_compilation.md
+      - Frame Interpolation: user_guide/diffusion/frame_interpolation.md
+      - Startup and Loading: user_guide/diffusion/startup_and_loading.md
+      - LoRA: user_guide/diffusion/lora.md
+    - Experimental:
+      - Session State Manager: features/session_state_manager.md
+  - Integrations:
+    - Custom Diffusion Pipeline: features/custom_pipeline.md
     - ComfyUI: features/comfyui.md
   - Fault Tolerance:
     - Failure Modes: user_guide/fault_injection_reliability_matrix.md
@@ -108,6 +114,7 @@ nav:
           - Shared Memory Connector: design/feature/omni_connectors/shared_memory_connector.md
           - Yuanrong Store Connector: design/feature/omni_connectors/yuanrong_connector.md
           - Yuanrong Transfer Engine Connector: design/feature/omni_connectors/yuanrong_transfer_engine_connector.md
+      - Quantization: design/feature/quantization.md
       - Diffusion Acceleration:
         - Parallelism:
           - CFG-Parallel: design/feature/cfg_parallel.md
@@ -117,9 +124,8 @@ nav:
           - Sequence Parallel: design/feature/sequence_parallel.md
           - Tensor Parallel: design/feature/tensor_parallel.md
           - VAE Patch Parallelism: design/feature/vae_parallel.md
-        - Attention Backends:
+        - Attention Optimization:
           - Skip-Softmax: design/feature/skip_softmax.md
-        - Quantization: design/feature/quantization.md
         - Cache-DiT: design/feature/cache_dit.md
         - TeaCache: design/feature/teacache.md
         - Diffusion Continuous Batching: design/feature/diffusion_continuous_batching.md

docs/design/index.md

@@ -11,6 +11,10 @@ in the active navigation.
 
 ## Feature Design Documents
 
+For user-facing configuration and current compatibility, see the
+[Features overview](../features/README.md). A design document defines an
+implementation contract; it is not, by itself, a general support claim.
+
 ### Runtime and stage execution
 
 - [Disaggregated Inference](feature/disaggregated_inference.md)
@@ -30,6 +34,10 @@ in the active navigation.
 - [Yuanrong Store Connector](feature/omni_connectors/yuanrong_connector.md)
 - [Yuanrong Transfer Engine Connector](feature/omni_connectors/yuanrong_transfer_engine_connector.md)
 
+### Quantization
+
+- [Quantization](feature/quantization.md)
+
 ### Diffusion acceleration
 
 #### Parallelism
@@ -42,14 +50,14 @@ in the active navigation.
 - [Tensor Parallel](feature/tensor_parallel.md)
 - [VAE Patch Parallelism](feature/vae_parallel.md)
 
-#### Attention Backends
+#### Attention optimization
+
+The [Diffusion Attention Backends](../user_guide/diffusion/attention_backends.md)
+guide lists all selectable backends and their platform defaults. Skip-Softmax
+is the backend-related optimization with a standalone feature-design contract:
 
 - [Skip-Softmax](feature/skip_softmax.md)
 
-#### Quantization
-
-- [Quantization](feature/quantization.md)
-
 - [Cache-DiT](feature/cache_dit.md)
 - [TeaCache](feature/teacache.md)
 - [Diffusion Continuous Batching](feature/diffusion_continuous_batching.md)

docs/features/README.md

@@ -0,0 +1,67 @@
+# Features
+
+Use this section to choose and configure vLLM-Omni runtime and optimization
+features. The navigation follows the active
+[Feature Design](../design/index.md#feature-design-documents) taxonomy whenever
+an implementation contract has a user-facing workflow.
+
+!!! note "User guides, design documents, and recipes"
+
+    User guides explain how to enable a feature and where it is supported.
+    Design documents define internal contracts and do not, by themselves,
+    imply general or production support. For model-specific launch flags and
+    hardware requirements, use the linked examples and validated recipes from
+    [Supported Models](../models/supported_models.md).
+
+## Runtime and Stage Execution
+
+| Goal | User guide | Related design contract |
+| --- | --- | --- |
+| Choose serial, batched, step-wise, or streaming diffusion execution | [Execution Modes and Streaming](../user_guide/diffusion/execution_modes.md) | [Diffusion Continuous Batching](../design/feature/diffusion_continuous_batching.md), [Async Diffusion Output](../design/feature/async_diffusion_output.md) |
+| Reclaim stage memory without restarting the server | [Sleep Mode](sleep_mode.md) | Runtime lifecycle behavior is documented in the user guide |
+
+Some runtime designs are deliberately not promoted as standalone User Guide
+features yet:
+
+- [Disaggregated Inference](../design/feature/disaggregated_inference.md) and
+  [OmniConnector implementations](../design/index.md#communication) describe
+  topology and transport contracts. Practical deployment configuration remains
+  under [Pipeline and deploy configurations](../configuration/stage_configs.md).
+- [Async Chunk](../design/feature/async_chunk.md) and
+  [Async Omni Output Materialization](../design/feature/omni_async_output_materialization.md)
+  are model- and pipeline-dependent. Use the selected model's deploy
+  configuration and recipe for the supported settings.
+- [Automatic Prefix Caching](../design/feature/prefix_caching.md) remains a
+  design-level contract until its user-facing configuration and compatibility
+  surface is consolidated.
+
+## Quantization
+
+Quantization is a cross-model feature rather than a diffusion-only
+optimization. The unified [`quantization_config` guide](../user_guide/quantization/overview.md)
+covers diffusion-only models, multi-stage omni/TTS models, and multi-stage
+diffusion models. Its [design contract](../design/feature/quantization.md)
+defines the shared configuration and backend extension points.
+
+## Diffusion Acceleration
+
+| Goal | User guide | Related design contract |
+| --- | --- | --- |
+| Compare acceleration methods and supported combinations | [Overview](../user_guide/diffusion_features.md), [Feature Compatibility](../user_guide/feature_compatibility.md) | [Diffusion acceleration designs](../design/index.md#diffusion-acceleration) |
+| Move weights between host and device memory | [CPU Offloading](../user_guide/diffusion/cpu_offload.md) | [Distributed Layerwise Offload](../design/feature/distributed_layerwise_offload.md) |
+| Reuse denoising computation | [Cache Acceleration](../user_guide/diffusion/cache_acceleration/cache_dit.md) | [Cache-DiT](../design/feature/cache_dit.md), [TeaCache](../design/feature/teacache.md) |
+| Distribute diffusion work across devices | [Parallelism](../user_guide/diffusion/parallelism/overview.md) | [Parallelism designs](../design/index.md#parallelism) |
+| Select dense, sparse, or quantized attention paths | [Attention Backends](../user_guide/diffusion/attention_backends.md) | [Skip-Softmax](../design/feature/skip_softmax.md) |
+| Compile repeated diffusion regions | [Regional Compilation](../user_guide/diffusion/regional_compilation.md) | User-facing optimization guide |
+| Add generated video frames | [Frame Interpolation](../user_guide/diffusion/frame_interpolation.md) | User-facing extension guide |
+| Reduce diffusion model startup time | [Startup and Loading](../user_guide/diffusion/startup_and_loading.md) | User-facing loading guide |
+| Apply diffusion adapters | [LoRA](../user_guide/diffusion/lora.md) | User-facing extension guide |
+
+The [Pipeline Parallelism guide](../user_guide/diffusion/parallelism/pipeline_parallel.md)
+remains available by direct link, but it is not promoted in the primary User
+Guide navigation until its user-facing support and placement are settled.
+
+## Experimental
+
+[Session State Manager](session_state_manager.md) is opt-in and experimental.
+Its APIs and compatibility may change without notice.

docs/user_guide/diffusion/startup_and_loading.md

@@ -0,0 +1,60 @@
+# Diffusion Startup and Loading
+
+Large diffusion models can take several minutes to load at startup. vLLM-Omni
+loads safetensors shards in parallel to reduce this initialization time.
+
+Multi-thread weight loading is enabled by default with four threads. No
+configuration is needed for the default behavior.
+
+## Configuration
+
+| Parameter | CLI flag | Default | Description |
+| --- | --- | --- | --- |
+| `enable_multithread_weight_load` | `--disable-multithread-weight-load` | `True` | Pass the flag to disable multi-thread loading |
+| `num_weight_load_threads` | `--num-weight-load-threads` | `4` | Number of parallel weight-loading threads |
+
+!!! tip
+
+    The default balances startup speed and disk I/O contention. Fast NVMe
+    storage may benefit from more threads, while network storage or hard disks
+    may not.
+
+## Online Serving
+
+```bash
+# Default: multi-thread loading with four threads
+vllm serve Qwen/Qwen-Image --omni --port 8091
+
+# Increase the thread count
+vllm serve Wan-AI/Wan2.2-I2V-A14B-Diffusers --omni \
+  --num-weight-load-threads 8
+
+# Disable multi-thread loading
+vllm serve Qwen/Qwen-Image --omni --disable-multithread-weight-load
+```
+
+## Offline Inference
+
+```python
+from vllm_omni import Omni
+
+# Default: multi-thread loading with four threads
+omni = Omni(model="Qwen/Qwen-Image")
+
+# Increase the thread count
+omni = Omni(
+    model="Wan-AI/Wan2.2-I2V-A14B-Diffusers",
+    num_weight_load_threads=8,
+)
+```
+
+## Reference Benchmarks
+
+The following measurements were collected on NVIDIA H800 hardware. Treat them
+as reference results rather than a guarantee for other storage or hardware
+configurations.
+
+| Model | Sequential loading | Multi-thread loading | Speedup |
+| --- | ---: | ---: | ---: |
+| **Qwen/Qwen-Image** (53.7 GiB) | 168 s | 27 s | **6.2x** |
+| **Wan-AI/Wan2.2-I2V-A14B-Diffusers** (64.5 GiB) | 283 s | 56 s | **5.1x** |

docs/user_guide/diffusion_features.md

@@ -49,7 +49,7 @@ Parallelism methods distribute computation across GPUs without quality loss (mat
 
 | Method | Description | Best For |
 |--------|-------------|----------|
-| **[Multi-Thread Weight Loading](#multi-thread-weight-loading)** | Loads safetensors shards in parallel using a thread pool | All diffusion models; reduces startup from minutes to seconds |
+| **[Multi-Thread Weight Loading](diffusion/startup_and_loading.md)** | Loads safetensors shards in parallel using a thread pool | All diffusion models; reduces startup from minutes to seconds |
 
 **Note:** Some acceleration methods can be combined together for optimized performance. See [Feature Compatibility Table](#feature-compatibility) and [Feature Compatibility Tutorial](feature_compatibility.md) for detailed configuration examples.
 
@@ -100,7 +100,9 @@ The following tables show which models support each feature:
 
 - **🔀SP (Ulysses & Ring)**: Includes both Ulysses-SP and Ring-Attention methods
 - ✅ = Fully supported
+- ✅* = Supported with the constraint listed below the table
 - ❌ = Not supported
+- ❓ = Not verified; not recommended
 
 > Notes:
 
@@ -119,7 +121,7 @@ The following tables show which models support each feature:
 | **FLUX.2-dev**           |     ✅     |     ✅      |           ✅           |       ✅        |         ✅         |          ❌          |   ✅    |             ✅             |          ✅           |       ❌        |        ❌         |
 | **GLM-Image**            |     ❌     |     ❌      |           ❌           |       ✅        |         ✅         |          ❌          |   ✅    |             ❌             |          ❌           |       ❌        |        ❌         |
 | **Hidream-I1-Full**        |     ❌     |     ❌      |           ❌           |       ❌        |         ✅         |          ❌          |   ❌    |             ❌             |          ❌           |       ❌        |        ❌         |
-| **HunyuanImage3**        |     ❌     |     ✅      |           ❌           |       ❌        |         ✅         |          ❌          |   ❌    |             ❌             |          ❌           |       ✅        |        ❌         |
+| **HunyuanImage3**        |     ❌     |     ✅      |           ❌           |       ❌        |         ✅         |          ❌          |   ❌    |             ❌             |          ❌           |       ✅        |        ✅*        |
 | **Krea 2**               |     ❌     |     ✅      |           ❌           |       ❌        |         ❌         |          ❌          |   ✅    |             ✅             |      ✅ (decode)      |       ❌        |        ❌         |
 | **LongCat-Image**        |     ✅     |     ✅      |           ✅           |       ✅        |         ✅         |          ❌          |   ❌    |             ✅             |          ❌           |       ❌        |        ❌         |
 | **LongCat-Image-Edit**   |     ✅     |     ✅      |           ✅           |       ✅        |         ✅         |          ❌          |   ❌    |             ✅             |          ❌           |       ❌        |        ❌         |
@@ -145,6 +147,7 @@ The following tables show which models support each feature:
 > 2. `Tongyi-MAI/Z-Image-Turbo` and `SII-GAIR/daVinci-MagiHuman-Base-1080p` are distilled models with minimal NFEs; CFG-Parallel is not necessary.
 > 3. Cosmos3 T2I uses `Cosmos3OmniDiffusersPipeline` with `modalities=["image"]`. Model-level CPU offload swaps the nested UND reasoner and GEN generator pathways; layerwise offload remains available for blockwise GEN/UND offload.
 > 4. Krea 2 currently supports single-GPU inference plus LoRA, Cache-DiT, HSDP, CPU/layerwise offload, and VAE-patch-parallel (decode). TP/SP/CFG-Parallel are not yet wired. The few-step distilled (Turbo) checkpoint uses `is_distilled=true` (fixed timestep shift `mu=1.15`); generate at 2048x2048 by default with `num_inference_steps≈8` and `guidance_scale=0`. The Raw checkpoint uses 1024x1024, `num_inference_steps=28`, and `guidance_scale=4.5`.
+> 5. HunyuanImage3 supports step execution. Multi-request step execution requires `TORCH_SDPA`; see [Diffusion Execution Modes](diffusion/execution_modes.md#step-execution).
 
 ### VideoGen
 
@@ -155,13 +158,16 @@ The following tables show which models support each feature:
 | **Wan2.1-VACE**              |     ❌     |     ✅      |           ✅           |       ✅        |         ✅         |         ❌         |   ✅    |             ✅             |      ✅ (decode)      |       ❌        |        ❌         |
 | **LTX-2**                    |     ❌     |     ✅      |           ✅           |       ✅        |         ✅         |         ❌         |   ✅    |             ✅             |      ✅ (decode)      |       ❌        |        ❌         |
 | **LTX-2.3**                  |     ❌     |     ✅      |           ✅           |       ✅        |         ✅         |         ❌         |   ✅    |             ✅             |      ✅ (decode)      |       ❌        |        ❌         |
-| **Helios**                   |     ❌     |     ✅      |           ✅           |       ✅        |         ✅         |         ❌         |   ✅    |             ✅             |          ❌           |       ❌        |        ❌         |
+| **Helios**                   |     ❌     |     ✅      |           ✅           |       ✅        |         ✅         |         ❌         |   ✅    |             ✅             |          ❌           |       ❌        |        ✅*        |
 | **HunyuanVideo-1.5 T2V I2V** |     ❌     |     ✅      |           ✅           |       ✅        |         ✅         |         ❌         |   ✅    |             ✅             |  ✅ (encode/decode)   |       ✅        |        ❌         |
 | **DreamID-Omni**             |     ❌     |     ❌      |           ❌           |       ✅        |         ❌         |         ❌         |   ✅    |             ✅             |          ❌           |       ❌        |        ❌         |
 | **Cosmos3**                  |     ❌     |     ✅      |           ✅           |       ✅        |         ✅         |         ❌         |   ✅    |             ✅             |  ✅ (encode/decode)   |       ✅        |        ❌         |
 | **LongCat-Video-Avatar-1.5** |     ❌     |     ❌      |           ❌           |       ❌        |         ❌         |         ❌         |   ❌    |             ❌             |          ❌           |       ❌        |        ❌         |
 | **MiniMax-H3**               | ✅ (FL2VA) |     ✅      |           ✅           |       ❌        |       ✅ (DiT/TE)  |         ❌         |   ✅    |             ✅             |       ✅ (tile)       |      ✅ (DiT)      |        ❌         |
 
+> **Step execution note:** Helios supports single-request step execution only;
+> use `max_num_seqs=1`.
+
 **Frame Interpolation Support**
 
 - **Supported**: Wan2.2 text-to-video, image-to-video, and TI2V pipelines
@@ -190,7 +196,7 @@ The following tables show which models support each feature:
 | **🔀Ring-Attn** | ✅ | ✅ | ✅ | | | | | | | | | | | |
 | **🔀CFG-Parallel** | ✅ | ✅ | ✅ | ✅ | | | | | | | | | | |
 | **🔀Tensor Parallel** | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | | | | | |
-| **🔀HSDP** | ❓ | ❓ | ❓ | ❓ | ❓ | ❌ | | | | | | | | |
+| **🔀HSDP** | ❓ | ❓ | ✅ | ❓ | ✅ | ❌ | | | | | | | | |
 | **🔀Expert Parallel** | ❓ | ❓ | ❓ | ❓ | ❓ | ❓ | ❓ | | | | | | | |
 | **💾CPU Offloading (Layerwise)** | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | | | | | | |
 | **💾CPU Offloading (Module-wise)** | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❓ | ❓ | ❌ | | | | | |
@@ -201,97 +207,43 @@ The following tables show which models support each feature:
 
 !!! info
 
-    1. Tensor Parallel and HSDP are not compatible.
+    1. HSDP can be combined with Ulysses-SP or CFG-Parallel. Tensor Parallel
+       and HSDP are not compatible; other HSDP combinations in the table
+       remain unverified.
     2. TeaCache and Cache-DiT are not compatible.
     3. CPU Offloading (Layerwise) and CPU Offloading (Module-wise) are not compatible.
-    4. CPU Offloading (Layerwise) supports single-card for now.
+    4. The CPU Offloading (Layerwise) row describes local layerwise offload.
-    5. Using FP8-Quant as an example of qunatization methods.
+       Multi-device Distributed Layerwise Offload has a separate topology and
+       compatibility matrix in the [CPU Offloading guide](diffusion/cpu_offload.md#distributed-layerwise-offloading).
+    5. The compatibility matrix uses FP8 as the representative quantization method.
     6. Step Execution is not compatible with any diffusion cache backend. LoRA is supported, but each scheduled batch must use a single adapter (requests with different `lora_request` or `lora_scale` are kept in separate batches).
 
 
 ## Multi-Thread Weight Loading
 
-Large diffusion models can take several minutes to load weights at startup (e.g., ~3 min for Qwen-Image, ~5 min for Wan2.2 I2V 14B). Multi-thread weight loading speeds up this process by loading safetensors shards in parallel using a thread pool instead of sequentially.
+The loading guide now lives at [Diffusion Startup and
-
+Loading](diffusion/startup_and_loading.md). This heading remains so existing
-This optimization is **enabled by default** with 4 threads. No configuration is needed for the default behavior.
+links to this section continue to work.
-
-### Configuration
-
-| Parameter | CLI Flag | Default | Description |
-|-----------|----------|---------|-------------|
-| `enable_multithread_weight_load` | `--disable-multithread-weight-load` | `True` (enabled) | Pass the flag to disable multi-thread loading |
-| `num_weight_load_threads` | `--num-weight-load-threads` | `4` | Number of threads for parallel weight loading |
-
-!!! tip
-    The default of 4 threads balances speed and disk I/O contention. On fast NVMe storage you may benefit from more threads (e.g., 8). On HDD or network storage, the default of 4 avoids saturating I/O bandwidth.
-
-### Online Serving
-
-```bash
-# Default (multi-thread enabled, 4 threads)
-vllm serve Qwen/Qwen-Image --omni --port 8091
-
-# Custom thread count
-vllm serve Wan-AI/Wan2.2-I2V-A14B-Diffusers --omni --num-weight-load-threads 8
-
-# Disable multi-thread loading
-vllm serve Qwen/Qwen-Image --omni --disable-multithread-weight-load
-```
-
-### Offline Inference
-
-```python
-from vllm_omni import Omni
-
-# Default (multi-thread enabled, 4 threads)
-omni = Omni(model="Qwen/Qwen-Image")
-
-# Custom thread count
-omni = Omni(
-    model="Wan-AI/Wan2.2-I2V-A14B-Diffusers",
-    num_weight_load_threads=8,
-)
-```
-
-### Benchmarks
-
-Measured on NVIDIA H800:
-
-| Model | Before | After | Speedup |
-|-------|--------|-------|---------|
-| **Qwen/Qwen-Image** (53.7 GiB) | 168s | 27s | **6.2x** |
-| **Wan-AI/Wan2.2-I2V-A14B-Diffusers** (64.5 GiB) | 283s | 56s | **5.1x** |
 
 ## Learn More
 
-**Cache Acceleration:**
+The Diffusion Acceleration navigation groups the remaining guides as follows:
 
-- **[TeaCache Configuration Guide](diffusion/cache_acceleration/teacache.md)** - Parameter tuning, performance tips, troubleshooting
+| Area | Guide |
-- **[Cache-DiT Advanced Guide](diffusion/cache_acceleration/cache_dit.md)** - DBCache, TaylorSeer, SCM techniques and optimization
+| --- | --- |
+| Compatibility | [Feature Compatibility](feature_compatibility.md) |
+| CPU offloading | [CPU Offloading](diffusion/cpu_offload.md) |
+| Cache acceleration | [TeaCache](diffusion/cache_acceleration/teacache.md), [Cache-DiT](diffusion/cache_acceleration/cache_dit.md) |
+| Parallelism | [Parallelism Overview](diffusion/parallelism/overview.md) |
+| Attention | [Attention Backends](diffusion/attention_backends.md) |
+| Compilation | [Regional Compilation](diffusion/regional_compilation.md) |
+| Video extension | [Frame Interpolation](diffusion/frame_interpolation.md) |
+| Startup | [Startup and Loading](diffusion/startup_and_loading.md) |
+| Adapters | [LoRA](diffusion/lora.md) |
 
-**Parallelism Methods:**
+Related cross-model and runtime features are documented separately:
 
-- **[Parallelism Overview](diffusion/parallelism/overview.md)** - Tensor Parallelism, Sequence Parallelism, CFG Parallelism, Pipeline Parallelism, HSDP, and Expert Parallelism
+- [Quantization](quantization/overview.md) covers diffusion-only models,
-
+  multi-stage omni/TTS models, and multi-stage diffusion models.
-**Memory Optimization:**
+- [Execution Modes and Streaming](diffusion/execution_modes.md) covers the
-
+  diffusion runtime, including batching, step execution, and streaming output.
-- **[CPU Offload Guide](diffusion/cpu_offload.md)** - Offload model components to CPU, reduce GPU memory usage
-- **[VAE Parallelism Guide](diffusion/parallelism/vae_parallelism.md)** - Distribute VAE decode work across GPUs for high-resolution images and videos
-- **[Quantization](quantization/overview.md)** - Quantization methods for diffusion, multi-stage omni/TTS, and multi-stage diffusion models
-
-**Extensions:**
-
-- **[LoRA Inference Guide](diffusion/lora.md)** - Low-Rank Adaptation for style customization and fine-tuning
-- **[Frame Interpolation Guide](diffusion/frame_interpolation.md)** - Worker-side post-generation video frame interpolation for smoother motion
-
-**Execution Modes:**
-
-- **[Diffusion Execution Modes](diffusion/execution_modes.md)** - Configure request batching, step execution, continuous batching, and streaming output
-
-**Startup Optimization:**
-
-- **[Multi-Thread Weight Loading](#multi-thread-weight-loading)** - Speed up model startup by loading safetensors shards in parallel
-
-**Advanced Topics:**
-
-- **[Feature Compatibility](feature_compatibility.md)** - How to combine multiple features for maximum performance

examples/offline_inference/image_to_video/image_to_video.py

@@ -61,9 +61,7 @@ from vllm_omni.diffusion.utils.param_utils import apply_declared_extra_args
 from vllm_omni.entrypoints.omni import Omni
 from vllm_omni.inputs.data import OmniDiffusionSamplingParams
 from vllm_omni.model_extras import (
-    build_image_to_video_prompt as build_model_image_to_video_prompt,
+    build_image_to_video_prompt,
-)
-from vllm_omni.model_extras import (
     get_extra_body_params,
     get_model_class_name,
 )
@@ -85,22 +83,6 @@ def parse_json_object(value: str, flag_name: str = "argument") -> dict[str, Any]
 parse_profiler_config = functools.partial(parse_json_object, flag_name="--profiler-config")
 
 
-def build_image_to_video_prompt(
-    prompt: str,
-    negative_prompt: str | None,
-    media_inputs: dict[str, Any],
-) -> dict[str, Any]:
-    """Build the canonical request envelope for the shared I2V example."""
-    result: dict[str, Any] = {
-        "prompt": prompt,
-        "modalities": ["video"],
-        "multi_modal_data": media_inputs,
-    }
-    if negative_prompt is not None:
-        result["negative_prompt"] = negative_prompt
-    return result
-
-
 def parse_args() -> argparse.Namespace:
     parser = argparse.ArgumentParser(
         description=(
@@ -540,13 +522,10 @@ def main():
         # Preserve the historical empty-prompt behavior for non-LTX examples.
         negative_prompt = ""
     prompt_dict = build_image_to_video_prompt(
+        model_class_name=model_class_name,
         prompt=args.prompt,
         negative_prompt=negative_prompt,
         media_inputs=media_inputs,
-    )
-    prompt_dict = build_model_image_to_video_prompt(
-        model_class_name=model_class_name,
-        prompt=prompt_dict,
         height=height,
         width=width,
         num_frames=num_frames,

examples/offline_inference/text_to_image/README.md

@@ -57,9 +57,8 @@ python examples/offline_inference/text_to_image/text_to_image.py \
   --output lingbot_t2i.png
 ```
 
-The shared T2I entry point selects the `image` output modality. The LingBot
+The LingBot prompt builder selects the `image` output modality and requests one
-pipeline validates that T2I requests produce one frame and do not include an
+frame. Passing an input image is not supported in LingBot T2I mode.
-input image.
 
 ## Quick Start
 

examples/offline_inference/text_to_image/text_to_image.py

@@ -21,9 +21,7 @@ from vllm_omni.inputs.data import OmniDiffusionSamplingParams
 from vllm_omni.lora.request import LoRARequest
 from vllm_omni.lora.utils import stable_lora_int_id
 from vllm_omni.model_extras import (
-    build_text_to_image_prompt as build_model_text_to_image_prompt,
+    build_text_to_image_prompt,
-)
-from vllm_omni.model_extras import (
     get_extra_body_params,
     get_model_class_name,
     should_init_extra_args_for_non_diffusion_stages,
@@ -58,17 +56,6 @@ def parse_json_object(value: str, flag_name: str = "argument") -> dict[str, Any]
 parse_profiler_config = functools.partial(parse_json_object, flag_name="--profiler-config")
 
 
-def build_text_to_image_prompt(prompt: str, negative_prompt: str | None) -> dict[str, Any]:
-    """Build the canonical request envelope for the shared T2I example."""
-    result: dict[str, Any] = {
-        "prompt": prompt,
-        "modalities": ["image"],
-    }
-    if negative_prompt is not None:
-        result["negative_prompt"] = negative_prompt
-    return result
-
-
 def _normalize_images_for_save(images: list[Any]) -> list[Any]:
     """Convert NumPy diffusion outputs to PIL images before saving."""
     normalized = []
@@ -518,12 +505,9 @@ def main():
     generation_start = time.perf_counter()
 
     prompt_dict = build_text_to_image_prompt(
+        model_class_name=model_class_name,
         prompt=args.prompt,
         negative_prompt=args.negative_prompt,
-    )
-    prompt_dict = build_model_text_to_image_prompt(
-        model_class_name=model_class_name,
-        prompt=prompt_dict,
         height=args.height,
         width=args.width,
     )

examples/online_serving/image_to_video/README.md

@@ -1,9 +1,6 @@
 # Image-To-Video
 
-This example demonstrates online image-to-video generation with vLLM-Omni.
+This example demonstrates how to deploy the Wan2.2 image-to-video model for online video generation using vLLM-Omni.
-The startup script defaults to Wan2.2, but `MODEL` can select any supported
-image-to-video model. The existing curl helper remains a concrete Wan2.2
-request through the standard Videos API.
 
 ## Start Server
 
@@ -23,7 +20,9 @@ bash run_server.sh
 
 The script allows overriding:
 - `MODEL` (default: `Wan-AI/Wan2.2-I2V-A14B-Diffusers`)
-- `PORT` (default: `8099`)
+- `PORT` (default: `8091`)
+- `BOUNDARY_RATIO` (default: `0.875`)
+- `FLOW_SHIFT` (default: `12.0`)
 - `CACHE_BACKEND` (default: `none`)
 - `ENABLE_CACHE_DIT_SUMMARY` (default: `0`)
 

examples/online_serving/image_to_video/run_server.sh

@@ -1,12 +1,12 @@
 #!/bin/bash
-# Shared image-to-video server start script
+# Wan2.2 image-to-video server start script
 
 MODEL="${MODEL:-Wan-AI/Wan2.2-I2V-A14B-Diffusers}"
 PORT="${PORT:-8099}"
 CACHE_BACKEND="${CACHE_BACKEND:-none}"
 ENABLE_CACHE_DIT_SUMMARY="${ENABLE_CACHE_DIT_SUMMARY:-0}"
 
-echo "Starting image-to-video server..."
+echo "Starting Wan2.2 I2V server..."
 echo "Model: $MODEL"
 echo "Port: $PORT"
 echo "Cache backend: $CACHE_BACKEND"

examples/online_serving/text_to_image/README.md

@@ -1,9 +1,6 @@
 # Text-To-Image
 
-This example demonstrates online text-to-image generation with vLLM-Omni. The
+This example demonstrates how to deploy Qwen-Image model for online image generation service using vLLM-Omni.
-startup script and Python clients default to Qwen-Image, but `MODEL` can select
-any supported text-to-image model. The existing curl helper remains a concrete
-Qwen-Image request through the standard Images API.
 
 ## Start Server
 
@@ -229,8 +226,8 @@ count, use `size` and `n` rather than `height`, `width`, or
 | `height`                 | int   | None    | Image height in pixels         |
 | `width`                  | int   | None    | Image width in pixels          |
 | `size`                   | str   | None    | Image size (e.g., "1024x1024") |
-| `num_inference_steps`    | int   | model   | Number of denoising steps      |
+| `num_inference_steps`    | int   | 50      | Number of denoising steps      |
-| `true_cfg_scale`         | float | model   | True CFG scale when supported  |
+| `true_cfg_scale`         | float | 4.0     | Qwen-Image CFG scale           |
 | `seed`                   | int   | None    | Random seed (reproducible)     |
 | `negative_prompt`        | str   | None    | Negative prompt                |
 | `num_outputs_per_prompt` | int   | 1       | Number of images to generate   |

examples/online_serving/text_to_image/gradio_demo.py

@@ -1,6 +1,6 @@
 #!/usr/bin/env python3
 """
-Shared text-to-image Gradio demo for online serving.
+Qwen-Image Gradio Demo for online serving.
 
 Usage:
     python gradio_demo.py [--server http://localhost:8091] [--port 7860]
@@ -77,9 +77,9 @@ def generate_image(
 def create_demo(server_url: str):
     """Create Gradio demo interface."""
 
-    with gr.Blocks(title="Text-to-Image Demo") as demo:
+    with gr.Blocks(title="Qwen-Image Demo") as demo:
-        gr.Markdown("# Text-to-Image Online Generation")
+        gr.Markdown("# Qwen-Image Online Generation")
-        gr.Markdown("Generate images using the model configured on the server")
+        gr.Markdown("Generate images using Qwen-Image model")
 
         with gr.Row():
             with gr.Column(scale=1):
@@ -115,6 +115,7 @@ def create_demo(server_url: str):
                         label="Inference Steps",
                         minimum=10,
                         maximum=100,
+                        # Default steps aligned with run_curl_text_to_image.sh to 100
                         value=100,
                         step=5,
                     )
@@ -172,7 +173,7 @@ def create_demo(server_url: str):
 
 
 def main():
-    parser = argparse.ArgumentParser(description="Shared text-to-image Gradio demo")
+    parser = argparse.ArgumentParser(description="Qwen-Image Gradio Demo")
     parser.add_argument("--server", default="http://localhost:8091", help="Server URL")
     parser.add_argument("--port", type=int, default=7860, help="Gradio port")
     parser.add_argument("--share", action="store_true", help="Create public link")

examples/online_serving/text_to_image/openai_chat_client.py

@@ -1,6 +1,6 @@
 #!/usr/bin/env python3
 """
-Shared OpenAI-compatible text-to-image generation client.
+Qwen-Image OpenAI-compatible image generation client.
 
 Usage:
     python openai_chat_client.py --prompt "A beautiful landscape" --output output.png
@@ -39,7 +39,7 @@ def generate_image(
         height: Image height in pixels
         width: Image width in pixels
         steps: Number of diffusion steps
-        true_cfg_scale: True CFG scale for models that support it
+        true_cfg_scale: Qwen-Image CFG scale
         seed: Random seed
         negative_prompt: Negative prompt
         num_outputs_per_prompt: Number of images to generate
@@ -70,7 +70,7 @@ def generate_image(
         payload["num_inference_steps"] = steps
     if true_cfg_scale is not None:
         payload["true_cfg_scale"] = true_cfg_scale
-    if negative_prompt is not None:
+    if negative_prompt:
         payload["negative_prompt"] = negative_prompt
     if seed is not None:
         payload["seed"] = seed
@@ -115,15 +115,15 @@ def generate_image(
 
 
 def main():
-    parser = argparse.ArgumentParser(description="Shared text-to-image client")
+    parser = argparse.ArgumentParser(description="Qwen-Image chat client")
     parser.add_argument("--prompt", "-p", default="a cup of coffee on the table", help="Text prompt")
-    parser.add_argument("--output", "-o", default="text_to_image_output.png", help="Output file")
+    parser.add_argument("--output", "-o", default="qwen_image_output.png", help="Output file")
     parser.add_argument("--server", "-s", default="http://localhost:8091", help="Server URL")
-    parser.add_argument("--height", type=int, help="Image height; defaults to the model setting")
+    parser.add_argument("--height", type=int, default=1024, help="Image height")
-    parser.add_argument("--width", type=int, help="Image width; defaults to the model setting")
+    parser.add_argument("--width", type=int, default=1024, help="Image width")
-    parser.add_argument("--steps", type=int, help="Inference steps; defaults to the model setting")
+    parser.add_argument("--steps", type=int, default=50, help="Inference steps")
-    parser.add_argument("--cfg-scale", type=float, help="True CFG scale when supported")
+    parser.add_argument("--cfg-scale", type=float, default=4.0, help="True CFG scale")
-    parser.add_argument("--seed", type=int, help="Random seed")
+    parser.add_argument("--seed", type=int, default=0, help="Random seed")
     parser.add_argument("--negative", help="Negative prompt")
 
     parser.add_argument("--lora-path", default=None, help="Server-local LoRA adapter folder (PEFT format)")

examples/online_serving/text_to_image/run_server.sh

@@ -1,10 +1,10 @@
 #!/bin/bash
-# Shared text-to-image online serving startup script
+# Qwen-Image online serving startup script
 
 MODEL="${MODEL:-Qwen/Qwen-Image}"
 PORT="${PORT:-8091}"
 
-echo "Starting text-to-image server..."
+echo "Starting Qwen-Image server..."
 echo "Model: $MODEL"
 echo "Port: $PORT"
 

tests/config/test_config_factory.py

@@ -2206,24 +2206,6 @@ class TestBaseConfigInheritance:
 class TestPlatformOverrides:
     """Test platform-specific deploy config overrides."""
 
-    def test_qwen3_omni_musa_clears_parent_quantization_for_audio_stages(self):
-        pipeline = resolve_pipeline_config(
-            "qwen3_omni_moe",
-            Q3_OMNI_ALL_STAGES_HF_CONFIG,
-        )
-        assert isinstance(pipeline, PipelineConfig)
-        deploy_path = Path(get_deploy_config_path("qwen3_omni_moe.yaml"))
-
-        base_stages = merge_pipeline_deploy(pipeline, load_deploy_config(deploy_path))
-        assert "hf_overrides" not in base_stages[1].yaml_engine_args
-        assert "hf_overrides" not in base_stages[2].yaml_engine_args
-
-        musa = _apply_platform_overrides(load_deploy_config(deploy_path), platform="musa")
-        musa_stages = merge_pipeline_deploy(pipeline, musa)
-        expected = {"quantization_config": None}
-        assert musa_stages[1].yaml_engine_args["hf_overrides"] == expected
-        assert musa_stages[2].yaml_engine_args["hf_overrides"] == expected
-
     def test_qwen3_tts_rocm_disables_code2wav_outer_cudagraph(self):
         deploy_path = Path(get_deploy_config_path("qwen3_tts.yaml"))
 

tests/examples/offline_inference/test_image_task_prompts.py

@@ -1,47 +0,0 @@
-# SPDX-License-Identifier: Apache-2.0
-# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
-
-from __future__ import annotations
-
-import pytest
-from PIL import Image
-
-from examples.offline_inference.image_to_video.image_to_video import build_image_to_video_prompt
-from examples.offline_inference.text_to_image.text_to_image import build_text_to_image_prompt
-
-pytestmark = [pytest.mark.core_model, pytest.mark.cpu]
-
-
-@pytest.mark.parametrize(
-    ("negative_prompt", "expected_negative_prompt"),
-    [(None, None), ("", ""), ("blurry", "blurry")],
-)
-def test_text_to_image_builds_canonical_prompt(
-    negative_prompt: str | None,
-    expected_negative_prompt: str | None,
-) -> None:
-    result = build_text_to_image_prompt("a red fox", negative_prompt)
-
-    assert result["prompt"] == "a red fox"
-    assert result["modalities"] == ["image"]
-    if expected_negative_prompt is None:
-        assert "negative_prompt" not in result
-    else:
-        assert result["negative_prompt"] == expected_negative_prompt
-
-
-def test_image_to_video_builds_canonical_prompt() -> None:
-    image = Image.new("RGB", (32, 16), "red")
-
-    result = build_image_to_video_prompt(
-        prompt="the fox turns toward the camera",
-        negative_prompt="flicker",
-        media_inputs={"image": image},
-    )
-
-    assert result == {
-        "prompt": "the fox turns toward the camera",
-        "modalities": ["video"],
-        "multi_modal_data": {"image": image},
-        "negative_prompt": "flicker",
-    }

tests/model_extras/test_model_extras.py

@@ -3,8 +3,6 @@
 
 from __future__ import annotations
 
-from typing import Any
-
 import pytest
 from PIL import Image
 
@@ -137,6 +135,67 @@ def test_lingbot_extra_registry_declares_request_params() -> None:
     assert should_init_extra_args_for_non_diffusion_stages("LingBotVideoPipeline") is False
 
 
+@pytest.mark.core_model
+@pytest.mark.cpu
+def test_lingbot_text_to_image_prompt_builder_preserves_empty_negative_prompt() -> None:
+    assert build_text_to_image_prompt(
+        "LingBotVideoPipeline",
+        prompt="a red fox in fresh snow",
+        negative_prompt="",
+        height=192,
+        width=320,
+    ) == {
+        "prompt": "a red fox in fresh snow",
+        "modalities": ["image"],
+        "negative_prompt": "",
+    }
+
+
+@pytest.mark.core_model
+@pytest.mark.cpu
+def test_lingbot_image_to_video_prompt_builder() -> None:
+    image = Image.new("RGB", (320, 192), "red")
+    result = build_image_to_video_prompt(
+        "LingBotVideoPipeline",
+        prompt="the fox looks toward the camera",
+        negative_prompt=None,
+        media_inputs={"image": image},
+        height=192,
+        width=320,
+        num_frames=9,
+    )
+    assert result == {
+        "prompt": "the fox looks toward the camera",
+        "modalities": ["video"],
+        "multi_modal_data": {"image": image},
+        "num_frames": 9,
+    }
+
+
+@pytest.mark.core_model
+@pytest.mark.cpu
+@pytest.mark.parametrize(
+    "media_inputs",
+    [
+        {},
+        {"image": "input.png"},
+        {"image": [Image.new("RGB", (16, 16))]},
+        {"image": Image.new("RGB", (16, 16)), "video": b"video"},
+    ],
+    ids=["missing", "path", "multiple", "extra-modality"],
+)
+def test_lingbot_image_to_video_prompt_builder_rejects_invalid_media(
+    media_inputs: dict[str, object],
+) -> None:
+    with pytest.raises(ValueError, match="exactly one PIL image"):
+        build_image_to_video_prompt(
+            "LingBotVideoPipeline",
+            prompt="move",
+            negative_prompt=None,
+            media_inputs=media_inputs,
+        )
+
+
 @pytest.mark.core_model
 @pytest.mark.cpu
 @pytest.mark.parametrize("pipeline_name", ["Cosmos3OmniDiffusersPipeline", "Cosmos3OmniPipeline"])
@@ -283,11 +342,8 @@ def test_ming_flash_omni_declared_extra_args_route_into_sampling_params() -> Non
 def test_ming_flash_omni_text_to_image_prompt_builder() -> None:
     assert build_text_to_image_prompt(
         "MingImagePipeline",
-        {
+        prompt="Please draw a cute cat.",
-            "prompt": "Please draw a cute cat.",
+        negative_prompt="ugly, blurry",
-            "modalities": ["image"],
-            "negative_prompt": "ugly, blurry",
-        },
         height=512,
         width=768,
     ) == {
@@ -304,7 +360,8 @@ def test_ming_flash_omni_text_to_image_prompt_builder() -> None:
     # pipeline's 1024x1024 default still applies.
     assert build_text_to_image_prompt(
         "MingImagePipeline",
-        {"prompt": "Draw a poster.", "modalities": ["image"]},
+        prompt="Draw a poster.",
+        negative_prompt=None,
     ) == {
         "prompt": "Draw a poster.",
         "modalities": ["image"],
@@ -334,6 +391,28 @@ def test_ming_flash_omni_image_to_image_prompt_builder() -> None:
     assert "negative_prompt" not in result
 
 
+@pytest.mark.core_model
+@pytest.mark.cpu
+@pytest.mark.parametrize("pipeline_name", ["Cosmos3OmniDiffusersPipeline", "Cosmos3OmniPipeline"])
+def test_cosmos3_text_to_image_prompt_builder_selects_image_modality(pipeline_name: str) -> None:
+    assert build_text_to_image_prompt(
+        pipeline_name,
+        prompt="a red sports car at golden hour",
+        negative_prompt="blurry, distorted",
+        height=1024,
+        width=1024,
+    ) == {
+        "prompt": "a red sports car at golden hour",
+        "modalities": ["image"],
+        "negative_prompt": "blurry, distorted",
+    }
+    assert build_text_to_image_prompt(
+        pipeline_name,
+        prompt="a red sports car",
+        negative_prompt=None,
+    ) == {"prompt": "a red sports car", "modalities": ["image"]}
+
+
 @pytest.mark.core_model
 @pytest.mark.cpu
 def test_audiox_extra_registry_declares_request_and_response_params() -> None:
@@ -426,11 +505,8 @@ def test_unknown_pipeline_has_empty_extra_registry() -> None:
 def test_bagel_text_to_image_prompt_builder() -> None:
     assert build_text_to_image_prompt(
         "BagelPipeline",
-        {
+        prompt="a cat",
-            "prompt": "a cat",
+        negative_prompt="blurry",
-            "modalities": ["image"],
-            "negative_prompt": "blurry",
-        },
         height=512,
         width=768,
     ) == {
@@ -467,17 +543,14 @@ def test_bagel_image_to_image_prompt_builder() -> None:
 
 @pytest.mark.core_model
 @pytest.mark.cpu
-def test_unknown_pipeline_preserves_canonical_text_to_image_prompt() -> None:
+def test_unknown_pipeline_uses_default_text_to_image_prompt() -> None:
-    canonical_prompt = {"prompt": "a cat", "modalities": ["image"]}
+    assert build_text_to_image_prompt(
-    assert (
+        "UnknownPipeline",
-        build_text_to_image_prompt(
+        prompt="a cat",
-            "UnknownPipeline",
+        negative_prompt=None,
-            canonical_prompt,
+        height=512,
-            height=512,
+        width=512,
-            width=512,
+    ) == {"prompt": "a cat"}
-        )
-        is canonical_prompt
-    )
 
 
 @pytest.mark.core_model
@@ -496,14 +569,12 @@ def test_unknown_pipeline_uses_default_image_to_image_prompt() -> None:
     }
 
 
-def _build_vace_prompt(media_inputs: dict[str, Any], *, num_frames: int = 5) -> dict:
+def _build_vace_prompt(media_inputs: dict[str, object], *, num_frames: int = 5) -> dict:
     return build_image_to_video_prompt(
         "WanVACEPipeline",
-        {
+        prompt="a bird flying",
-            "prompt": "a bird flying",
+        negative_prompt=None,
-            "modalities": ["video"],
+        media_inputs=media_inputs,
-            "multi_modal_data": media_inputs,
-        },
         height=16,
         width=320,
         num_frames=num_frames,
@@ -526,7 +597,7 @@ def _build_vace_prompt(media_inputs: dict[str, Any], *, num_frames: int = 5) ->
     ],
     ids=["i2v", "v2lf", "flf2v", "inpaint", "r2v"],
 )
-def test_vace_image_to_video_prompt_builder(media_inputs: dict[str, Any]) -> None:
+def test_vace_image_to_video_prompt_builder(media_inputs: dict[str, object]) -> None:
     result = _build_vace_prompt(media_inputs)
     mmd = result["multi_modal_data"]
     if "reference_images" in mmd:
@@ -545,7 +616,7 @@ def test_vace_image_to_video_prompt_builder(media_inputs: dict[str, Any]) -> Non
         ({"control_image": Image.new("RGB", (320, 16))}, "Unsupported VACE media input"),
     ],
 )
-def test_vace_rejects_invalid_media_combinations(media_inputs: dict[str, Any], message: str) -> None:
+def test_vace_rejects_invalid_media_combinations(media_inputs: dict[str, object], message: str) -> None:
     with pytest.raises(ValueError, match=message):
         _build_vace_prompt(media_inputs)
 
@@ -593,7 +664,8 @@ def test_mammothmoda2_extra_registry_declares_request_and_response_params() -> N
 
     wrapper_prompt = build_text_to_image_prompt(
         "MammothModa2ForConditionalGeneration",
-        {"prompt": "a cat", "modalities": ["image"]},
+        prompt="a cat",
+        negative_prompt=None,
         height=256,
         width=256,
     )
@@ -607,11 +679,8 @@ def test_mammothmoda2_text_to_image_prompt_builder() -> None:
     # prompt is ignored (MammothModa2 t2i uses CFG, not an explicit negative path).
     assert build_text_to_image_prompt(
         "MammothModa2DiTPipeline",
-        {
+        prompt="a cat",
-            "prompt": "a cat",
+        negative_prompt="blurry",
-            "modalities": ["image"],
-            "negative_prompt": "blurry",
-        },
         height=512,
         width=768,
     ) == {

tests/worker/test_omni_gpu_model_runner.py

@@ -3,88 +3,13 @@ from types import SimpleNamespace
 
 import pytest
 import torch
-from vllm.v1.cudagraph_dispatcher import CUDAGraphMode
 
-from vllm_omni.worker.gpu_model_runner import (
+from vllm_omni.worker.gpu_model_runner import OmniGPUModelRunner, _filter_mrope_kwargs_for_model
-    OmniGPUModelRunner,
-    _filter_mrope_kwargs_for_model,
-)
 from vllm_omni.worker.omni_connector_model_runner_mixin import OmniConnectorModelRunnerMixin
 
 pytestmark = [pytest.mark.core_model, pytest.mark.cpu]
 
 
-def _runner_for_talker_graph_init(
-    *,
-    talker_mtp_graph_safe: bool | None,
-    has_separate_talker: bool = True,
-    model_stage: str = "talker",
-) -> OmniGPUModelRunner:
-    runner = object.__new__(OmniGPUModelRunner)
-    runner.model = SimpleNamespace(
-        talker=object() if has_separate_talker else None,
-        talker_mtp=object(),
-        model_stage=model_stage,
-    )
-    if talker_mtp_graph_safe is not None:
-        runner.model.talker_mtp_graph_safe = talker_mtp_graph_safe
-    runner.model_config = SimpleNamespace(hf_text_config=SimpleNamespace(hidden_size=4))
-    runner.compilation_config = SimpleNamespace(
-        cudagraph_mode=CUDAGraphMode.FULL,
-        max_cudagraph_capture_size=1,
-    )
-    runner.vllm_config = object()
-    runner.max_num_reqs = 1
-    runner.dtype = torch.float32
-    runner._make_buffer = lambda *args, **kwargs: SimpleNamespace(args=args, kwargs=kwargs)
-    return runner
-
-
-def test_talker_mtp_skips_graph_when_model_declares_unsafe(monkeypatch):
-    runner = _runner_for_talker_graph_init(talker_mtp_graph_safe=False)
-    talker_mtp = runner.model.talker_mtp
-    monkeypatch.setattr(
-        "vllm_omni.worker.gpu_model_runner.current_omni_platform.get_graph_wrapper_cls",
-        lambda: pytest.fail("graph wrapper must not be selected"),
-    )
-
-    OmniGPUModelRunner._init_talker_mtp(runner)
-
-    assert runner.talker_mtp is talker_mtp
-
-
-@pytest.mark.parametrize("model_stage", ["thinker", "code2wav"])
-def test_non_talker_stage_does_not_use_talker_mtp_graph(monkeypatch, model_stage: str):
-    runner = _runner_for_talker_graph_init(
-        talker_mtp_graph_safe=None,
-        has_separate_talker=False,
-        model_stage=model_stage,
-    )
-    talker_mtp = runner.model.talker_mtp
-    monkeypatch.setattr(
-        "vllm_omni.worker.gpu_model_runner.current_omni_platform.get_graph_wrapper_cls",
-        lambda: pytest.fail("graph wrapper must not be selected"),
-    )
-
-    OmniGPUModelRunner._init_talker_mtp(runner)
-
-    assert runner.talker_mtp is talker_mtp
-
-
-@pytest.mark.parametrize("talker_mtp_graph_safe", [None, True])
-def test_talker_mtp_uses_graph_for_legacy_or_explicit_safe_model(monkeypatch, talker_mtp_graph_safe):
-    runner = _runner_for_talker_graph_init(talker_mtp_graph_safe=talker_mtp_graph_safe)
-    wrapped = object()
-    monkeypatch.setattr(
-        "vllm_omni.worker.gpu_model_runner.current_omni_platform.get_graph_wrapper_cls",
-        lambda: lambda *args, **kwargs: wrapped,
-    )
-
-    OmniGPUModelRunner._init_talker_mtp(runner)
-
-    assert runner.talker_mtp is wrapped
-
-
 class DummyBuffer:
     """A minimal buffer wrapper that exposes the `.gpu` attribute."""
 

vllm_omni/deploy/qwen3_omni_moe.yaml

@@ -69,18 +69,6 @@ stages:
       repetition_penalty: 1.1
 
 platforms:
-  # The supported Qwen3-Omni ModelOpt checkpoint quantizes the thinker only. Clear the
-  # checkpoint-level metadata for the BF16 audio stages on the verified MUSA
-  # profile before vLLM performs its normal quantization auto-detection.
-  musa:
-    stages:
-      - stage_id: 1
-        hf_overrides:
-          quantization_config: null
-      - stage_id: 2
-        hf_overrides:
-          quantization_config: null
-
   npu:
     stages:
       - stage_id: 0

vllm_omni/model_executor/models/qwen3_omni/qwen3_omni.py

@@ -152,9 +152,6 @@ class Qwen3OmniMoeForConditionalGeneration(
                 dtype=torch.long,
             )
         elif self.model_stage == "talker":
-            # The outer wrapper exposes talker_mtp for every stage, but only
-            # the talker stage owns the module that the method invokes.
-            self.talker_mtp_graph_safe = current_omni_platform.supports_talker_mtp_graph_capture()
             multimodal_config.skip_mm_profiling = True
             self.has_preprocess = True
             self.has_postprocess = True

vllm_omni/model_extras/cosmos3.py

@@ -3,6 +3,32 @@
 
 from __future__ import annotations
 
+from typing import Any
+
+
+def build_text_to_image_prompt(
+    prompt: str,
+    negative_prompt: str | None,
+    height: int | None = None,
+    width: int | None = None,
+) -> dict[str, Any]:
+    """Build a Cosmos3 T2I prompt by selecting image output modality.
+
+    Cosmos3 uses the same pipeline class for T2I and video modes. The OpenAI
+    image endpoint selects T2I by adding ``modalities=["image"]``. ``height``
+    and ``width`` are accepted for registry compatibility; sizing travels
+    through sampling params instead of the prompt payload.
+    """
+    del height, width
+    text_prompt: dict[str, Any] = {
+        "prompt": prompt,
+        "modalities": ["image"],
+    }
+    if negative_prompt is not None:
+        text_prompt["negative_prompt"] = negative_prompt
+    return text_prompt
+
+
 COSMOS3_EXTRA_BODY_PARAMS = frozenset(
     {
         "flow_shift",

vllm_omni/model_extras/lingbot_video.py

@@ -3,6 +3,11 @@
 
 from __future__ import annotations
 
+from collections.abc import Mapping
+from typing import Any
+
+from PIL import Image
+
 LINGBOT_VIDEO_EXTRA_BODY_PARAMS = frozenset(
     {
         "batch_cfg",
@@ -19,3 +24,43 @@ LINGBOT_VIDEO_EXTRA_BODY_PARAMS = frozenset(
         "t_thresh",
     }
 )
+
+
+def build_text_to_image_prompt(
+    prompt: str,
+    negative_prompt: str | None,
+    height: int | None = None,
+    width: int | None = None,
+) -> dict[str, Any]:
+    del height, width
+    result: dict[str, Any] = {
+        "prompt": prompt,
+        "modalities": ["image"],
+    }
+    if negative_prompt is not None:
+        result["negative_prompt"] = negative_prompt
+    return result
+
+
+def build_image_to_video_prompt(
+    prompt: str,
+    negative_prompt: str | None,
+    media_inputs: Mapping[str, Any],
+    height: int | None = None,
+    width: int | None = None,
+    num_frames: int | None = None,
+) -> dict[str, Any]:
+    del height, width
+    if set(media_inputs) != {"image"} or not isinstance(media_inputs["image"], Image.Image):
+        raise ValueError("LingBot text-image-to-video requires exactly one PIL image input.")
+
+    result: dict[str, Any] = {
+        "prompt": prompt,
+        "modalities": ["video"],
+        "multi_modal_data": {"image": media_inputs["image"]},
+    }
+    if num_frames is not None:
+        result["num_frames"] = num_frames
+    if negative_prompt is not None:
+        result["negative_prompt"] = negative_prompt
+    return result

vllm_omni/model_extras/registry.py

@@ -28,12 +28,21 @@ from vllm_omni.model_extras.cosmos3 import (
     COSMOS3_EXTRA_BODY_PARAMS,
     COSMOS3_EXTRA_OUTPUT_PARAMS,
 )
+from vllm_omni.model_extras.cosmos3 import (
+    build_text_to_image_prompt as build_cosmos3_text_to_image_prompt,
+)
 from vllm_omni.model_extras.helios import (
     HELIOS_EXTRA_BODY_PARAMS,
     HELIOS_EXTRA_OUTPUT_PARAMS,
 )
 from vllm_omni.model_extras.hunyuan_image3 import build_x_to_text_prompt as build_hunyuan_x_to_text_prompt
 from vllm_omni.model_extras.lingbot_video import LINGBOT_VIDEO_EXTRA_BODY_PARAMS
+from vllm_omni.model_extras.lingbot_video import (
+    build_image_to_video_prompt as build_lingbot_image_to_video_prompt,
+)
+from vllm_omni.model_extras.lingbot_video import (
+    build_text_to_image_prompt as build_lingbot_text_to_image_prompt,
+)
 from vllm_omni.model_extras.ltx2 import LTX_EXTRA_BODY_PARAMS, LTX_EXTRA_OUTPUT_PARAMS
 from vllm_omni.model_extras.magi_human import (
     MAGI_HUMAN_EXTRA_BODY_PARAMS,
@@ -138,6 +147,18 @@ def build_x_to_text_prompt(
     return builder(model, prompt, has_image)
 
 
+def default_text_to_image_prompt(
+    prompt: str,
+    negative_prompt: str | None,
+    height: int | None = None,
+    width: int | None = None,
+) -> dict[str, Any]:
+    result: dict[str, Any] = {"prompt": prompt}
+    if negative_prompt is not None:
+        result["negative_prompt"] = negative_prompt
+    return result
+
+
 def default_image_to_image_prompt(
     prompt: str,
     negative_prompt: str | None,
@@ -154,6 +175,20 @@ def default_image_to_image_prompt(
     return result
 
 
+def default_image_to_video_prompt(
+    prompt: str,
+    negative_prompt: str | None,
+    media_inputs: Mapping[str, Any],
+    height: int | None = None,
+    width: int | None = None,
+    num_frames: int | None = None,
+) -> dict[str, Any]:
+    del height, width, num_frames
+    if set(media_inputs) != {"image"} or not isinstance(media_inputs["image"], Image.Image):
+        raise ValueError("This model only supports a single --image input in the shared image-to-video example.")
+    return default_image_to_image_prompt(prompt, negative_prompt, media_inputs["image"])
+
+
 _EXTRA_SPECS: dict[str, dict[str, Any]] = {
     "AudioXPipeline": {
         "extra_body_params": AUDIOX_EXTRA_BODY_PARAMS,
@@ -173,11 +208,12 @@ _EXTRA_SPECS: dict[str, dict[str, Any]] = {
     "Cosmos3OmniDiffusersPipeline": {
         "extra_body_params": COSMOS3_EXTRA_BODY_PARAMS,
         "extra_output_params": COSMOS3_EXTRA_OUTPUT_PARAMS,
-        # The shared T2I example already supplies modalities=["image"].
+        "text_to_image_prompt_builder": build_cosmos3_text_to_image_prompt,
     },
     "Cosmos3OmniPipeline": {
         "extra_body_params": COSMOS3_EXTRA_BODY_PARAMS,
         "extra_output_params": COSMOS3_EXTRA_OUTPUT_PARAMS,
+        "text_to_image_prompt_builder": build_cosmos3_text_to_image_prompt,
     },
     "MagiHumanPipeline": {
         "extra_body_params": MAGI_HUMAN_EXTRA_BODY_PARAMS,
@@ -193,8 +229,8 @@ _EXTRA_SPECS: dict[str, dict[str, Any]] = {
     },
     "LingBotVideoPipeline": {
         "extra_body_params": LINGBOT_VIDEO_EXTRA_BODY_PARAMS,
-        # Shared T2I/I2V envelopes select the output modality. LingBot's
+        "text_to_image_prompt_builder": build_lingbot_text_to_image_prompt,
-        # pipeline owns model-specific validation and normalization.
+        "image_to_video_prompt_builder": build_lingbot_image_to_video_prompt,
     },
     **{
         model_class_name: {
@@ -227,7 +263,7 @@ _EXTRA_SPECS: dict[str, dict[str, Any]] = {
 }
 
 # Multi-stage discovery reports the top-level wrapper rather than its DiT
-# submodule, so both names must resolve to the same request builders.
+# submodule, so both names must resolve to the same request adapters.
 _EXTRA_SPECS["MammothModa2ForConditionalGeneration"] = _EXTRA_SPECS["MammothModa2DiTPipeline"]
 _EXTRA_SPECS["Mammothmoda2Model"] = _EXTRA_SPECS["MammothModa2DiTPipeline"]
 
@@ -269,21 +305,18 @@ def should_init_extra_args_for_non_diffusion_stages(model_class_name: str | None
 
 def build_text_to_image_prompt(
     model_class_name: str | None,
-    prompt: dict[str, Any],
+    prompt: str,
+    negative_prompt: str | None,
     height: int | None = None,
     width: int | None = None,
 ) -> dict[str, Any]:
-    """Build a model-specific T2I prompt from an example-owned envelope."""
     spec = _get_spec(model_class_name)
-    builder: TextToImagePromptBuilder | None = spec.get("text_to_image_prompt_builder") if spec else None
+    builder: TextToImagePromptBuilder = (
-    if builder is None:
+        spec.get("text_to_image_prompt_builder", default_text_to_image_prompt)
-        return prompt
+        if spec is not None
-    return builder(
+        else default_text_to_image_prompt
-        prompt=str(prompt["prompt"]),
-        negative_prompt=prompt.get("negative_prompt"),
-        height=height,
-        width=width,
     )
+    return builder(prompt, negative_prompt, height, width)
 
 
 def build_image_to_image_prompt(
@@ -305,24 +338,17 @@ def build_image_to_image_prompt(
 
 def build_image_to_video_prompt(
     model_class_name: str | None,
-    prompt: dict[str, Any],
+    prompt: str,
+    negative_prompt: str | None,
+    media_inputs: Mapping[str, Any],
     height: int | None = None,
     width: int | None = None,
     num_frames: int | None = None,
 ) -> dict[str, Any]:
-    """Build a model-specific I2V prompt from an example-owned envelope."""
     spec = _get_spec(model_class_name)
-    builder: ImageToVideoPromptBuilder | None = spec.get("image_to_video_prompt_builder") if spec else None
+    builder: ImageToVideoPromptBuilder = (
-    if builder is None:
+        spec.get("image_to_video_prompt_builder", default_image_to_video_prompt)
-        return prompt
+        if spec is not None
-    media_inputs = prompt.get("multi_modal_data") or {}
+        else default_image_to_video_prompt
-    if not isinstance(media_inputs, Mapping):
-        raise TypeError("Canonical I2V prompt multi_modal_data must be a mapping.")
-    return builder(
-        prompt=str(prompt["prompt"]),
-        negative_prompt=prompt.get("negative_prompt"),
-        media_inputs=media_inputs,
-        height=height,
-        width=width,
-        num_frames=num_frames,
     )
+    return builder(prompt, negative_prompt, media_inputs, height, width, num_frames)

vllm_omni/platforms/interface.py

@@ -155,11 +155,6 @@ class OmniPlatform(Platform):
         """Check if the platform supports torch.compile with inductor backend."""
         raise NotImplementedError
 
-    @classmethod
-    def supports_talker_mtp_graph_capture(cls) -> bool:
-        """Whether a model may capture its dedicated talker MTP graph."""
-        return True
-
     @classmethod
     def has_flash_attn_package(cls) -> bool:
         """Check if a Flash Attention package is available and usable on this platform."""

vllm_omni/platforms/musa/platform.py

@@ -115,11 +115,6 @@ class MUSAOmniPlatform(OmniPlatform, MUSAPlatformBase):
         """MUSA supports torch.compile with inductor backend."""
         return True
 
-    @classmethod
-    def supports_talker_mtp_graph_capture(cls) -> bool:
-        """MUSA keeps Qwen3 talker MTP outside its dedicated FULL graph."""
-        return False
-
     @classmethod
     def supports_float64(cls) -> bool:
         """MUSA does not support float64 yet."""

vllm_omni/worker/gpu_model_runner.py

@@ -206,12 +206,8 @@ class OmniGPUModelRunner(GPUModelRunner):
         cudagraph_mode = self.compilation_config.cudagraph_mode
         assert cudagraph_mode is not None
         has_separate_talker = getattr(self.model, "talker", None) is not None
-        talker_mtp_graph_safe = getattr(self.model, "talker_mtp_graph_safe", None)
+        talker_mtp_graph_safe = getattr(self.model, "talker_mtp_graph_safe", False)
-        # Preserve the legacy separate-talker default only when the model has
+        if cudagraph_mode.has_full_cudagraphs() and (has_separate_talker or talker_mtp_graph_safe):
-        # not declared graph safety. An explicit False must disable wrapping.
-        if talker_mtp_graph_safe is None:
-            talker_mtp_graph_safe = has_separate_talker
-        if cudagraph_mode.has_full_cudagraphs() and talker_mtp_graph_safe:
             graph_wrapper_cls = current_omni_platform.get_graph_wrapper_cls()
             self.talker_mtp = graph_wrapper_cls(talker_mtp, self.vllm_config, runtime_mode=CUDAGraphMode.FULL)
         # TTS exposes mtp_hidden_size; Omni uses hf_text_config.hidden_size.