[Docs] Redirect the former PD guide URL

Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
[Docs] Document current PD design
2026-08-13 12:48:43 +08:00 · 2026-08-13 12:44:42 +08:00 · 2026-08-13 12:23:56 +08:00 · 2026-08-13 06:54:34 +08:00 · 2026-08-13 06:50:41 +08:00
27 changed files with 375 additions and 864 deletions
@@ -43,6 +43,7 @@ nav:
    - Overview: features/README.md
    - Runtime and Stage Execution:
      - Execution Modes and Streaming: user_guide/diffusion/execution_modes.md
+      - Prefill-Decode Disaggregation (experimental): features/pd_disaggregation.md
      - Sleep Mode: features/sleep_mode.md
    - Quantization:
      - Overview: user_guide/quantization/overview.md
@@ -14,10 +14,6 @@ For introduction, please check [Introduction for stage config](./stage_configs.m

 - **[GPU Memory Calculation and Configuration](./gpu_memory_utilization.md)** - Guide on how to calculate memory requirements and set up `gpu_memory_utilization` for optimal performance

-## Multi-Stage Recipes
-
- **[Prefill-Decode Disaggregation](./pd_disaggregation.md)** - How to derive a PD-aware Qwen3-Omni stage config from the default config without introducing another bundled YAML
-
 ## Optimization Features

 - **[Diffusion Features Overview](../user_guide/diffusion_features.md)** - Complete overview of all diffusion model features and supported models
@@ -1,171 +0,0 @@
-# Prefill-Decode (PD) Disaggregation
-
-PD disaggregation splits the Qwen3-Omni thinker into separate prefill and decode
-stages so prompt processing and token generation can run on different workers.
-
-This is documented as a stage-config recipe instead of a bundled YAML because the
-deployment-specific values usually change per environment:
-
- GPU placement
- `tensor_parallel_size`
- connector backend and connector ports
- connector IPs or bootstrap addresses
-
-Start from the [default Qwen3-Omni stage config](gh-file:vllm_omni/deploy/qwen3_omni_moe.yaml)
-and copy it to your own file, for example `qwen3_omni_pd.yaml`. Then apply the
-changes below.
-
-## Requirements
-
- 3+ GPUs for a basic layout: prefill, decode, and talker+code2wav
- A KV connector supported by vLLM, such as `MooncakeConnector`
- Matching `tensor_parallel_size` on the prefill and decode thinker stages
-
-## 1. Split the thinker into prefill and decode stages
-
-Replace the original thinker stage with two stages:
-
-```yaml
-stage_args:
-  - stage_id: 0
-    stage_type: llm
-    is_prefill_only: true
-    runtime:
-      devices: "0"
-    engine_args:
-      max_num_seqs: 16
-      model_stage: thinker
-      model_arch: Qwen3OmniMoeForConditionalGeneration
-      worker_type: ar
-      scheduler_cls: vllm_omni.core.sched.omni_ar_scheduler.OmniARScheduler
-      gpu_memory_utilization: 0.9
-      enforce_eager: true
-      trust_remote_code: true
-      engine_output_type: latent
-      distributed_executor_backend: "mp"
-      enable_prefix_caching: false
-      max_num_batched_tokens: 32768
-      hf_config_name: thinker_config
-      tensor_parallel_size: 1
-      kv_transfer_config:
-        kv_connector: "MooncakeConnector"
-        kv_role: "kv_producer"
-        kv_rank: 0
-        kv_parallel_size: 2
-        kv_connector_extra_config:
-          mooncake_bootstrap_port: 25201
-    final_output: false
-    is_comprehension: true
-    default_sampling_params:
-      temperature: 0.4
-      top_p: 0.9
-      top_k: 1
-      max_tokens: 2048
-      seed: 42
-      detokenize: True
-      repetition_penalty: 1.05
-
-  - stage_id: 1
-    stage_type: llm
-    is_decode_only: true
-    runtime:
-      devices: "1"
-    engine_args:
-      max_num_seqs: 64
-      model_stage: thinker
-      model_arch: Qwen3OmniMoeForConditionalGeneration
-      worker_type: ar
-      scheduler_cls: vllm_omni.core.sched.omni_ar_scheduler.OmniARScheduler
-      gpu_memory_utilization: 0.9
-      enforce_eager: true
-      trust_remote_code: true
-      engine_output_type: latent
-      distributed_executor_backend: "mp"
-      enable_prefix_caching: false
-      max_num_batched_tokens: 32768
-      hf_config_name: thinker_config
-      tensor_parallel_size: 1
-      kv_transfer_config:
-        kv_connector: "MooncakeConnector"
-        kv_role: "kv_consumer"
-        kv_rank: 1
-        kv_parallel_size: 2
-        kv_connector_extra_config:
-          mooncake_bootstrap_port: 25202
-    engine_input_source: [0]
-    final_output: true
-    final_output_type: text
-    is_comprehension: true
-    default_sampling_params:
-      temperature: 0.4
-      top_p: 0.9
-      top_k: 1
-      max_tokens: 2048
-      seed: 42
-      detokenize: True
-      repetition_penalty: 1.05
-```
-
-Notes:
-
- `is_prefill_only: true` marks the thinker stage that only saves KV.
- `is_decode_only: true` marks the thinker stage that resumes from remote KV.
- `kv_transfer_config` is required on both stages.
- The orchestrator forces the prefill stage to run with `max_tokens=1`, so the
-  prefill side only processes the prompt and exports KV.
-
-## 2. Shift the downstream stages by one index
-
-After inserting the extra thinker stage, renumber the remaining stages:
-
-```yaml
-  - stage_id: 2
-    runtime:
-      devices: "2"
-    engine_input_source: [1]
-    sync_process_input_func: vllm_omni.model_executor.stage_input_processors.qwen3_omni.thinker2talker_token_only
-
-  - stage_id: 3
-    runtime:
-      devices: "2"
-    engine_args:
-      max_num_seqs: 1
-    engine_input_source: [2]
-    sync_process_input_func: vllm_omni.model_executor.stage_input_processors.qwen3_omni.talker2code2wav_token_only
-```
-
-Compared with the default Qwen3-Omni config:
-
- the talker becomes stage `2` instead of stage `1`
- the code2wav stage becomes stage `3` instead of stage `2`
- the talker now reads from decode stage `1`
-
-## 3. Add runtime edges for the four-stage pipeline
-
-```yaml
-runtime:
-  enabled: true
-  edges:
-    - from: 0
-      to: 1
-    - from: 1
-      to: 2
-    - from: 2
-      to: 3
-```
-
-## 4. Launch with your custom config
-
-```bash
-vllm serve Qwen/Qwen3-Omni-30B-A3B-Instruct --omni --port 8091 \
-    --deploy-config /path/to/qwen3_omni_pd.yaml
-```
-
-## Operational Notes
-
- `MooncakeConnector` does not support heterogeneous TP sizes across the PD
-  pair. Keep prefill and decode at the same `tensor_parallel_size`.
- If the thinker requires TP=2, both thinker stages must use TP=2 and be given
-  separate GPU sets, for example `"0,1"` for prefill and `"2,3"` for decode.
- Choose connector ports and addresses that match your deployment. The values
-  shown above are examples only.
@@ -1,6 +1,6 @@
 # Prometheus Metrics Design

-This document describes how vLLM-Omni exposes Prometheus metrics for multi-stage pipelines, the constraints that shaped the design, and how the pipeline-level metrics coexist with upstream vLLM per-engine metrics.
+This document describes how vLLM-Omni exposes Prometheus metrics for multi-stage pipelines, the constraints that shaped the design, and how the pipeline-level metrics coexist with upstream vLLM per-engine metrics. This iteration covers the text and audio surface; diffusion / image / video families land in a follow-up.

 ## Objectives

@@ -20,13 +20,19 @@ an implementation contract has a user-facing workflow.
 | Choose serial, batched, step-wise, or streaming diffusion execution | [Execution Modes and Streaming](../user_guide/diffusion/execution_modes.md) | [Diffusion Continuous Batching](../design/feature/diffusion_continuous_batching.md), [Async Diffusion Output](../design/feature/async_diffusion_output.md) |
 | Reclaim stage memory without restarting the server | [Sleep Mode](sleep_mode.md) | Runtime lifecycle behavior is documented in the user guide |

+The [Prefill-Decode Disaggregation design reference](pd_disaggregation.md)
+describes the experimental Qwen3-Omni runtime topology and its current
+integration boundaries. It is not yet a supported launch recipe.
+
 Some runtime designs are deliberately not promoted as standalone User Guide
 features yet:

- [Disaggregated Inference](../design/feature/disaggregated_inference.md) and
+- Generic, OmniConnector-based
+  [Disaggregated Inference](../design/feature/disaggregated_inference.md) is not
+  yet a standalone User Guide feature. Its
  [OmniConnector implementations](../design/index.md#communication) describe
-  topology and transport contracts. Practical deployment configuration remains
-  under [Pipeline and deploy configurations](../configuration/stage_configs.md).
+  transport contracts, while practical connector configuration remains under
+  [Pipeline and deploy configurations](../configuration/stage_configs.md).
 - [Async Chunk](../design/feature/async_chunk.md) and
  [Async Omni Output Materialization](../design/feature/omni_async_output_materialization.md)
  are model- and pipeline-dependent. Use the selected model's deploy
@@ -0,0 +1,140 @@
+# Prefill-Decode (PD) Disaggregation (experimental)
+
+!!! warning "Experimental design reference"
+    PD disaggregation is only partially integrated on the current `main` branch.
+    The runtime contains PD detection and routing scaffolding, but there is no
+    supported deploy overlay or end-to-end validated launch recipe yet. Treat this
+    page as a description of the current design, not as production guidance.
+
+Prefill-decode disaggregation splits the Qwen3-Omni Thinker into two logical
+stages: a prefill worker that processes the prompt and produces KV cache, and a
+decode worker that imports that cache and generates tokens. The Talker and
+Code2Wav stages remain downstream; PD does not disaggregate the entire
+Qwen3-Omni pipeline.
+
+## Current design
+
+### Runtime topology
+
+```mermaid
+flowchart LR
+    client["Client request"] --> orchestrator["Omni Orchestrator"]
+    orchestrator -->|"original prompt<br/>max_tokens = 1"| prefill["Thinker prefill<br/>KV producer"]
+    prefill -->|"KV blocks"| connector[("vLLM KV connector")]
+    prefill -->|"KV routing metadata"| orchestrator
+    orchestrator -->|"original prompt<br/>remote KV parameters"| decode["Thinker decode<br/>KV consumer"]
+    connector -->|"remote KV load"| decode
+    decode -->|"generated text"| client
+```
+
+The orchestrator owns the request lifecycle, while the vLLM KV connector owns
+the bulk KV transfer. The original prompt is submitted to both Thinker stages:
+the decode worker still needs the request tokens and multimodal features to
+construct its request, while the imported KV cache prevents it from repeating
+the completed prefill computation.
+
+For the full Qwen3-Omni pipeline, splitting the Thinker adds one logical stage:
+
+| Stage | Role | Output or handoff |
+| --- | --- | --- |
+| 0 | Thinker prefill | Saves prompt KV and prompt-side multimodal output |
+| 1 | Thinker decode | Loads remote KV, generates text, and produces decode-side conditioning |
+| 2 | Talker | Converts combined Thinker conditioning into codec codes |
+| 3 | Code2Wav | Converts codec codes into audio |
+
+Text remains a final output of the decode stage, and audio remains a final
+output of Code2Wav.
+
+### Request lifecycle
+
+1. At startup, PD detection looks for one stage marked `is_prefill_only` and
+   one downstream stage marked `is_decode_only`. The decode stage must list the
+   prefill stage as an input source.
+2. The orchestrator clones the Thinker sampling parameters for prefill, forces
+   `max_tokens=1`, clears stop conditions, and sets producer-side
+   `kv_transfer_params`. This makes prefill finish by length after exporting KV.
+3. When prefill finishes, the orchestrator captures its KV routing metadata and
+   Qwen multimodal output under the original request ID.
+4. The orchestrator submits the original prompt to decode with consumer-side
+   `kv_transfer_params`. Those parameters identify the transfer, prefill engine,
+   bootstrap endpoint, and, when required by the connector, remote request.
+5. The decode engine imports the KV cache and continues generation. The normal
+   Omni stage path then carries Thinker conditioning through Talker and
+   Code2Wav.
+
+### Control and data paths
+
+PD uses two different connector layers. They solve different handoffs and are
+not interchangeable.
+
+| Path | Payload | Owner |
+| --- | --- | --- |
+| PD control path | Request ID, original prompt, `transfer_id`, bootstrap address, engine ID, and connector-specific request metadata | Omni Orchestrator |
+| PD data path | Thinker attention KV blocks | vLLM KV connector |
+| Downstream model path | Thinker hidden states and embeddings, Talker codec codes, and audio tensors | vLLM-Omni stage connectors and processors |
+
+### Detection and validation
+
+The current runtime accepts one logical prefill/decode pair per pipeline. Each
+stage may use tensor parallelism, but both sides of the pair must satisfy these
+invariants:
+
+- both stages define `kv_transfer_config`
+- prefill uses `kv_producer` or `kv_both`
+- decode uses `kv_consumer` or `kv_both`
+- both stages use the same KV connector
+- `kv_buffer_device` and `kv_buffer_size` agree when set on both stages
+- both stages use the same `tensor_parallel_size`
+
+These checks happen before request processing so an inconsistent PD topology
+fails during engine initialization.
+
+## What is implemented today
+
+| Area | Current state |
+| --- | --- |
+| Pair detection and topology validation | Implemented in the shared PD helper |
+| Prefill sampling preparation | Implemented for sync and async entrypoints |
+| Prefill-to-decode routing | Implemented in the current Orchestrator |
+| Deploy-based configuration | Not yet migrated; the supported `stages:` deploy schema cannot currently expand Qwen3-Omni into the four-stage PD topology |
+| Qwen3 Thinker-to-Talker handoff | Bridge state and embedding-merge helpers exist, but the merge is not wired into the live stage processor |
+| Connector compatibility | Incomplete; the Orchestrator currently requires `remote_request_id`, while the pinned Mooncake connector completes prefill without returning that metadata |
+| Validation coverage | PD entrypoint tests and the PD deploy overlay in online end-to-end tests are temporarily disabled during migration |
+| Performance | No non-regression or throughput/latency result is established for the current path |
+
+Because these pieces are still converging, the previous legacy `stage_args` YAML
+and `vllm serve` command have been removed. The serve CLI now accepts the
+deploy-based `stages:` schema, so retaining the legacy recipe would imply a
+launch path that current `main` rejects.
+
+## Configuration contract under development
+
+The intended deploy integration starts from the
+[default Qwen3-Omni deploy config](gh-file:vllm_omni/deploy/qwen3_omni_moe.yaml)
+and expands its frozen three-stage pipeline into the four stages shown above.
+A complete integration must:
+
+- split the Thinker definition without duplicating the Talker or Code2Wav model
+  contracts
+- remap downstream stage IDs, input sources, and stage connectors
+- inject producer and consumer `kv_transfer_config` values into the two Thinker
+  stages
+- allocate separate device sets for prefill and decode
+- preserve both final outputs: decode text and Code2Wav audio
+- provide connector-specific routing metadata without hard-coding one connector
+  contract
+
+The required number of GPUs depends on tensor-parallel sizes and whether Talker
+and Code2Wav share a device. A basic TP=1 layout uses one device for prefill, one
+for decode, and one for the two downstream audio stages.
+
+## Implementation map
+
+- PD detection, validation, and sampling-parameter preparation:
+  [`PDDisaggregationMixin`](gh-file:vllm_omni/entrypoints/pd_utils.py)
+- startup detection and connector metadata:
+  [`AsyncOmniEngine`](gh-file:vllm_omni/engine/async_omni_engine.py)
+- request-scoped prefill-to-decode routing:
+  [`Orchestrator`](gh-file:vllm_omni/engine/orchestrator.py)
+- Qwen3 downstream conditioning helpers:
+  [Qwen3-Omni stage input processor](gh-file:vllm_omni/model_executor/stage_input_processors/qwen3_omni.py)
@@ -14,10 +14,7 @@ A: At first, you can check current [issues](https://github.com/vllm-project/vllm

 > Q: Does vLLM-Omni support AWQ or any other quantization?

-A: AWQ is not currently listed as a validated vLLM-Omni path. Quantization
-support depends on the model, pipeline stage, checkpoint format, and hardware
-backend. See the current [quantization support matrix](../user_guide/quantization/overview.md)
-for online methods, pre-quantized checkpoints, and model-level validation.
+A: We plan to introduce GGUF FP8 prequantized models and online FP8 quantization in version 0.16.0. Support for other quantization types will follow in future releases. For details, please see our [Q1 quantization roadmap](https://github.com/vllm-project/vllm-omni/issues/1057).

 > Q: Does vLLM-Omni support multimodal streaming input and output?

@@ -1,23 +1,23 @@
 # Production Metrics

-vLLM-Omni exposes Prometheus metrics via the `/metrics` endpoint on the OpenAI-compatible API server.
+vLLM-Omni exposes Prometheus metrics via the `/metrics` endpoint on the OpenAI-compatible API server. This page covers the text and audio surface; diffusion / image / video metrics are tracked in a follow-up PR.

 ```bash
 vllm-omni serve Qwen/Qwen3-Omni-30B-A3B-Instruct --port 8000 --log-stats
 curl http://localhost:8000/metrics
 ```

-**`--log-stats` is required to populate metric data.** Without the flag, the endpoint still returns `200 OK` but the upstream `vllm:*` wrap is not registered at all, and the 15 `vllm_omni:*` families are registered as placeholders with no data written to them. This keeps the runtime cost essentially zero for deployments that don't need monitoring. With the flag, all ~80 families populate.
+**`--log-stats` is required to populate metric data.** Without the flag, the endpoint still returns `200 OK` but the upstream `vllm:*` wrap is not registered at all, and the 15 `vllm:omni_*` families are registered as placeholders with no data written to them. This keeps the runtime cost essentially zero for deployments that don't need monitoring. With the flag, all ~80 families populate.

 ## Metric Namespaces

 | Prefix | Source | Present when |
 |--------|--------|--------------|
-| `vllm_omni:` | vLLM-Omni orchestrator / audio modality / cross-stage transfer | Pipeline-dependent |
+| `vllm:omni_` | vLLM-Omni orchestrator / audio modality / cross-stage transfer | Pipeline-dependent |
 | `vllm:` | Upstream vLLM engine, wrapped by `OmniPrometheusStatLogger` to expose `{stage, replica}` | Pipeline includes an LLM (AR) stage |
 | `http_` / `process_` | Uvicorn / Python runtime | Always |

-## Pipeline-Level Metrics (`vllm_omni:`)
+## Pipeline-Level Metrics (`vllm:omni_`)

 Defined in `vllm_omni/metrics/prometheus.py`. Track request lifecycle across the full multi-stage pipeline.

@@ -25,66 +25,42 @@ Defined in `vllm_omni/metrics/prometheus.py`. Track request lifecycle across the

 | Metric | Type | Labels | Description |
 |--------|------|--------|-------------|
-| `vllm_omni:num_requests_running` | Gauge | `model_name` | Pipeline-global in-flight requests (dispatched to engine, not yet finalized) |
-| `vllm_omni:num_requests_waiting` | Gauge | `model_name` | Requests waiting in the Orchestrator queue |
-| `vllm_omni:requests_success_total` | Counter | `model_name`, `finished_reason` | Total requests by completion reason. `finished_reason` ∈ {`stop`, `length`, `abort`, ...} mirroring upstream `vllm:request_success_total`; aborts cover client disconnect / cancellation paths in addition to upstream `FinishReason.ABORT` |
+| `vllm:omni_num_requests_running` | Gauge | `model_name` | Pipeline-global in-flight requests (dispatched to engine, not yet finalized) |
+| `vllm:omni_num_requests_waiting` | Gauge | `model_name` | Requests waiting in the Orchestrator queue |
+| `vllm:omni_requests_success_total` | Counter | `model_name`, `finished_reason` | Total requests by completion reason. `finished_reason` ∈ {`stop`, `length`, `abort`, ...} mirroring upstream `vllm:request_success_total`; aborts cover client disconnect / cancellation paths in addition to upstream `FinishReason.ABORT` |

 ### Latency

 | Metric | Type | Labels | Description |
 |--------|------|--------|-------------|
-| `vllm_omni:e2e_request_latency_s` | Histogram | `model_name` | Pipeline-global end-to-end request latency in seconds |
+| `vllm:omni_e2e_request_latency_s` | Histogram | `model_name` | Pipeline-global end-to-end request latency in seconds |

-## Audio Modality Metrics (`vllm_omni:`)
+## Audio Modality Metrics (`vllm:omni_`)

 Emitted at request finalize, except for `audio_ttfp_s` (streaming-hook at the first audio packet) and `audio_underrun_s` / `audio_continuity_ok_total` (streaming finalize, after the chunk stream is exhausted). All carry `{model_name, stage, replica}` plus the listed extra label.

 | Metric | Type | Extra label | Description |
 |--------|------|-------------|-------------|
-| `vllm_omni:audio_ttfp_s` | Histogram | — | Time from request arrival to first audio packet/frame |
-| `vllm_omni:audio_duration_s` | Histogram | — | Audio content duration (`audio_frames / sample_rate`) |
-| `vllm_omni:audio_rtf` | Histogram | — | Real-time factor (`stage_gen_time_s / audio_duration_s`); streaming TTS SLO red line `< 1`; uses `RTF_BUCKETS` |
-| `vllm_omni:audio_frames_total` | Counter | — | Cumulative audio frame count; throughput via `rate()` |
-| `vllm_omni:audio_underrun_s` | Histogram | — | Per-request worst-case player deficit; `> 0` indicates listener heard silent gaps |
-| `vllm_omni:audio_continuity_ok_total` | Counter | `threshold_ms` | Incremented when the request's worst underrun stayed below `threshold_ms` |
-| `vllm_omni:audio_skipped_requests_total` | Counter | `reason` | Silent-loss counter — code2wav rejected malformed codec input and returned `200 OK` with empty audio |
+| `vllm:omni_audio_ttfp_s` | Histogram | — | Time from request arrival to first audio packet/frame |
+| `vllm:omni_audio_duration_s` | Histogram | — | Audio content duration (`audio_frames / sample_rate`) |
+| `vllm:omni_audio_rtf` | Histogram | — | Real-time factor (`stage_gen_time_s / audio_duration_s`); streaming TTS SLO red line `< 1`; uses `RTF_BUCKETS` |
+| `vllm:omni_audio_frames_total` | Counter | — | Cumulative audio frame count; throughput via `rate()` |
+| `vllm:omni_audio_underrun_s` | Histogram | — | Per-request worst-case player deficit; `> 0` indicates listener heard silent gaps |
+| `vllm:omni_audio_continuity_ok_total` | Counter | `threshold_ms` | Incremented when the request's worst underrun stayed below `threshold_ms` |
+| `vllm:omni_audio_skipped_requests_total` | Counter | `reason` | Silent-loss counter — code2wav rejected malformed codec input and returned `200 OK` with empty audio |

 The continuity math comes from `vllm_omni/benchmarks/audio_continuity.py::compute_continuity_stats` so the server-side observation aligns with the bench-side definition.

-## Diffusion Metrics (`vllm_omni:`)
-
-Per-request timing breakdowns for diffusion (image/video) stages. Emitted at request finalize when `stage_metrics.diffusion_metrics` is present. All carry `{model_name, stage, replica}`.
-
-| Metric | Type | Description |
-|--------|------|-------------|
-| `vllm_omni:diffusion_exec_s` | Histogram | DiT forward pass execution time per request in seconds |
-| `vllm_omni:diffusion_exec_per_step_s` | Histogram | DiT forward pass execution time per denoising step in seconds (`exec_s / num_inference_steps`) |
-| `vllm_omni:diffusion_preprocess_s` | Histogram | Diffusion input preprocessing time per request in seconds |
-| `vllm_omni:diffusion_postprocess_s` | Histogram | Diffusion output postprocessing (VAE decode) time per request in seconds |
-
-`diffusion_exec_s` uses `SECONDS_BUCKETS` (50 ms → 300 s); the other three use `SECONDS_FAST_BUCKETS` (1 ms → 60 s). `diffusion_exec_per_step_s` is skipped when `num_inference_steps` is not set.
-
-### Example PromQL
-
-```promql
-# P99 DiT exec time by stage
-histogram_quantile(0.99, rate(vllm_omni:diffusion_exec_s_bucket[5m]))
-
-# Ratio of preprocess to total exec wall time
-rate(vllm_omni:diffusion_preprocess_s_sum[5m])
-  / rate(vllm_omni:diffusion_exec_s_sum[5m])
-```
-
-## Cross-Stage Transfer Metrics (`vllm_omni:`)
+## Cross-Stage Transfer Metrics (`vllm:omni_`)

 Per-physical-transfer histograms tracking the data hop between adjacent stages. Labels `{model_name, from_stage, from_replica, to_stage, to_replica}` let dashboards attribute latency to specific replica edges. `from_replica` / `to_replica` are resolved from the orchestrator's sticky-routing binding (`stage_pool.get_bound_replica_id(request_id)`), so no extra plumbing through `TransferEdgeStats` is needed.

 | Metric | Type | Description |
 |--------|------|-------------|
-| `vllm_omni:transfer_size_bytes` | Histogram | Per-transfer payload size in bytes |
-| `vllm_omni:transfer_tx_s` | Histogram | Sender-side time (serialize + submit to connector) |
-| `vllm_omni:transfer_rx_s` | Histogram | Receiver-side time (recv + deserialize) |
-| `vllm_omni:transfer_in_flight_s` | Histogram | Network in-flight time (TX done → RX recv start) |
+| `vllm:omni_transfer_size_bytes` | Histogram | Per-transfer payload size in bytes |
+| `vllm:omni_transfer_tx_s` | Histogram | Sender-side time (serialize + submit to connector) |
+| `vllm:omni_transfer_rx_s` | Histogram | Receiver-side time (recv + deserialize) |
+| `vllm:omni_transfer_in_flight_s` | Histogram | Network in-flight time (TX done → RX recv start) |

 ## vLLM Engine Metrics (`vllm:`)

@@ -107,9 +83,9 @@ For the full list of upstream metrics, see [the vLLM docs](https://github.com/vl

 | Metric group | Multi-stage LLM (Qwen3-Omni) |
 |---|---|
-| `vllm_omni:` request tracking + latency | With `--log-stats` |
-| `vllm_omni:` audio modality | With `--log-stats`, if pipeline has a talker stage |
-| `vllm_omni:` transfer | With `--log-stats`, if pipeline has ≥ 2 stages |
+| `vllm:omni_` request tracking + latency | With `--log-stats` |
+| `vllm:omni_` audio modality | With `--log-stats`, if pipeline has a talker stage |
+| `vllm:omni_` transfer | With `--log-stats`, if pipeline has ≥ 2 stages |
 | `vllm:` engine metrics (per `(stage, replica)`) | With `--log-stats` |
 | `vllm:` MFU metrics | With `--log-stats --enable-mfu-metrics` |

@@ -118,5 +94,5 @@ For the full list of upstream metrics, see [the vLLM docs](https://github.com/vl
 - All time-bearing metrics use the `_s` suffix (values in seconds). Buckets are `SECONDS_BUCKETS` for e2e / generation-style values and `SECONDS_FAST_BUCKETS` (1 ms → 60 s) for the fine-grained transfer and audio-underrun values.
 - Counters use the `_total` suffix (auto-appended by `prometheus_client`).
 - Sizes use the `_bytes` suffix.
- All omni-specific families are prefixed `vllm_omni:`. The upstream `unregister_vllm_metrics()` function is monkey-patched (see `vllm_omni/patch.py`) to a scoped version that still strips upstream `vllm:*` collectors so multi-engine init within one process does not crash on duplicate registration, but preserves anything prefixed `vllm_omni:`.
- Text and audio first-output use distinct families (`vllm:time_to_first_token_seconds` reused from upstream for text; `vllm_omni:audio_ttfp_s` for audio) rather than a single metric with a `modality` label.
+- All omni-specific families are prefixed `vllm:omni_`. The upstream `unregister_vllm_metrics()` function is monkey-patched (see `vllm_omni/patch.py`) to a scoped version that still strips upstream `vllm:*` collectors so multi-engine init within one process does not crash on duplicate registration, but preserves anything prefixed `vllm:omni_` / `vllm_omni`.
+- Text and audio first-output use distinct families (`vllm:time_to_first_token_seconds` reused from upstream for text; `vllm:omni_audio_ttfp_s` for audio) rather than a single metric with a `modality` label.
@@ -92,7 +92,7 @@ current support.
 |--------|--------------|-------------|----------|
 | **[FP8](quantization/fp8.md)** | `quantization="fp8"` | FP8 W8A8 on validated transformer stages | Memory reduction, inference speedup |
 | **[INT8](quantization/int8.md)** | `quantization="int8"` | INT8 W8A8 on validated transformer stages | Memory reduction, broad GPU compatibility |
-| **[BitsAndBytes W4](quantization/bitsandbytes.md)** | `quantization="bitsandbytes"` | Online NF4/FP4 weight-only quantization on validated CUDA transformer stages | Reducing transformer weight memory without preparing a separate checkpoint |
+| **[GGUF](quantization/gguf.md)** | `quantization="gguf"` | Native GGUF transformer-only weights (Q4, Q8, etc.) | Memory reduction on consumer GPUs |

 ## Supported Models

@@ -143,8 +143,7 @@ python text_to_video.py \
 - `--enable-layerwise-offload`: enable layerwise offloading on DiT modules.
 - `--frame-rate`: generation FPS for pipelines that require it (e.g., LTX2).
 - `--audio-sample-rate`: fallback audio sample rate when the pipeline returns audio.
- `--quantization`: online quantization method, for example `fp8`; supported
-  pre-quantized checkpoints are normally auto-detected from checkpoint metadata.
+- `--quantization`: quantization method (`fp8` for FP8, `gguf` for GGUF).
 - `--flow-shift`: scheduler flow_shift parameter.
 - `--lora-path`: path to PEFT LoRA adapter folder or checkpoint file.
 - `--lora-scale`: scale factor for LoRA weights.
@@ -90,8 +90,8 @@ vllm serve "Qwen/Qwen-Image" \
    --quantization-config '{"method": "fp8"}'
 ```

-Other vLLM-Omni quantization methods, such as `modelopt`, `mxfp4`, `mxfp8`,
-serialized checkpoints, static FP8 configs, and layer-name skip lists
+Other vLLM-Omni quantization methods, such as `gguf`, `modelopt`, `mxfp4`,
+`mxfp8`, serialized checkpoints, static FP8 configs, and layer-name skip lists
 such as `ignored_layers`, are intentionally not translated by this compatibility
 shortcut. Use Diffusers-native configuration through `diffusers_load_kwargs` or
 a native vLLM-Omni pipeline for those cases.
@@ -10,10 +10,6 @@ checkpoint's `config.json` and auto-detects
 AutoRound is static quantization: no `--quantization` flag is needed at
 inference time when the checkpoint already contains the quantization config.

-The validation terms on this page follow the
-[support levels](overview.md#support-levels) defined in the overview. A listed
-checkpoint is not enough by itself to establish end-to-end model support.
-
 ## Hardware Support

 | Device | Support |
@@ -25,57 +21,46 @@ checkpoint is not enough by itself to establish end-to-end model support.
 | Intel XPU | ✅ |
 | Ascend NPU | ❌ |

-Legend: `✅` backend available, `❌` unsupported, `⭕` not verified in this
-guide. This table is method-level backend availability. AutoRound is an Intel
-project; vLLM selects a compatible CUDA or XPU compute backend at load time.
+Legend: `✅` supported, `❌` unsupported, `⭕` not verified in this
+guide. AutoRound is Intel-supported.

 ## Model Type Support

 ### Diffusion Model (Qwen-Image, Wan2.2)

-| Model | Checkpoint | Scope | Scheme | Validation |
-|-------|------------|-------|--------|------------|
-| FLUX.1-dev | `vllm-project-org/FLUX.1-dev-AutoRound-w4a16` | Diffusion transformer | W4A16 | Validated |
-| Qwen-Image | `INC4AI/Qwen-Image-AutoRound-W4A16` | Diffusion transformer | W4A16 | Validated with end-to-end generation and quality comparison |
-| Wan2.2-I2V | `Intel/Wan2.2-I2V-A14B-Diffusers-int4-AutoRound` | Both diffusion transformers | W4A16 | CI-backed |
-| Wan2.2-T2V | `Intel/Wan2.2-T2V-A14B-Diffusers-int4-AutoRound` | Both diffusion transformers | W4A16 | CI-backed |
-| Wan2.2-TI2V | `Intel/Wan2.2-TI2V-5B-Diffusers-int4-AutoRound` | Diffusion transformer | W4A16 | Validated; not in the scheduled Wan AutoRound job |
-
-CUDA execution uses GPTQ-Marlin or another compatible vLLM AutoRound backend;
-Intel XPU uses the Intel-supported backend. The text encoder and VAE stay in
-BF16 for all entries above.
+| Model | Checkpoint | Scope | Scheme | Backend |
+|-------|------------|-------|--------|---------|
+| FLUX.1-dev | `vllm-project-org/FLUX.1-dev-AutoRound-w4a16` | Diffusion transformer | W4A16 | GPTQ-Marlin or Intel-supported AutoRound backend |
+| Qwen-Image | Not listed | Diffusion transformer | W4A16 | Not validated |
+| Wan2.2-I2V | `Intel/Wan2.2-I2V-A14B-Diffusers-int4-AutoRound` | Diffusion transformer | W4A16 | GPTQ-Marlin or Intel-supported AutoRound backend |
+| Wan2.2-T2V | `Intel/Wan2.2-T2V-A14B-Diffusers-int4-AutoRound` | Diffusion transformer | W4A16 | GPTQ-Marlin or Intel-supported AutoRound backend |
+| Wan2.2-TI2V | `Intel/Wan2.2-TI2V-5B-Diffusers-int4-AutoRound` | Diffusion transformer | W4A16 | GPTQ-Marlin or Intel-supported AutoRound backend |

 ### World Model (Cosmos3)

-| Model | Checkpoint | Scope | Scheme | Validation |
-|-------|------------|-------|--------|------------|
-| Cosmos3-Nano | `Intel/Cosmos3-Nano-int4-AutoRound` | World-model transformer | W4A16 | Integrated; no model-specific end-to-end test is maintained in-tree |
-| Cosmos3-Super | `Intel/Cosmos3-Super-int4-AutoRound` | World-model transformer | W4A16 | Validated with a manual serving run |
-
-The Cosmos VAE and guardrail components stay in BF16.
+| Model | Checkpoint | Scope | Scheme | Backend |
+|-------|------------|-------|--------|---------|
+| Cosmos3-Nano | `Intel/Cosmos3-Nano-int4-AutoRound` | World-model transformer | W4A16 | GPTQ-Marlin or Intel-supported AutoRound backend |
+| Cosmos3-Super | `Intel/Cosmos3-Super-int4-AutoRound` | World-model transformer | W4A16 | GPTQ-Marlin or Intel-supported AutoRound backend |

 ### Multi-Stage Omni/TTS Model (Qwen3-Omni, Qwen3-TTS)

-| Model | Checkpoint | Scope | Scheme | Validation |
-|-------|------------|-------|--------|------------|
-| Qwen2.5-Omni-7B | `Intel/Qwen2.5-Omni-7B-int4-AutoRound` | Thinker language-model stage | W4A16 | Validated outside scheduled CI |
-| Qwen3-Omni-30B-A3B-Instruct | `Intel/Qwen3-Omni-30B-A3B-Instruct-int4-AutoRound` | Thinker language-model stage | W4A16 | Validated outside scheduled CI |
+| Model | Checkpoint | Scope | Scheme | Backend |
+|-------|------------|-------|--------|---------|
+| Qwen2.5-Omni-7B | `Intel/Qwen2.5-Omni-7B-int4-AutoRound` | Language-model stage | W4A16 | AutoRound |
+| Qwen3-Omni-30B-A3B-Instruct | `Intel/Qwen3-Omni-30B-A3B-Instruct-int4-AutoRound` | Thinker language-model stage | W4A16 | AutoRound |
 | Qwen3-TTS | Not listed | TTS language-model stage | W4A16 | Not validated |

 AutoRound support is checkpoint-driven. A model is supported when its
 checkpoint uses a compatible INC/AutoRound config and the target stage maps to
 vLLM-Omni's runtime module names.

-For Qwen-Omni, only the thinker language model uses the AutoRound checkpoint.
-Audio and vision encoders, the talker, and waveform-decoder stages stay in
-BF16.
-
 ### Multi-Stage Diffusion Model (BAGEL, GLM-Image)

-| Model | Checkpoint | Scope | Validation |
-|-------|------------|-------|------------|
-| GLM-Image | `Intel/GLM-Image-int4-AutoRound` | Diffusion transformer | CI-backed |
-| BAGEL | Not listed | Checkpoint-defined diffusion or transformer stage | Not validated |
+| Model | Scope | Status | Notes |
+|-------|-------|--------|-------|
+| GLM-Image | Diffusion transformer  | ✅ | `Intel/GLM-Image-int4-AutoRound` |
+| BAGEL | Checkpoint-defined diffusion or transformer stage | Not validated | Requires a compatible AutoRound checkpoint |

 ## Configuration

@@ -134,14 +119,6 @@ The checkpoint should contain a config like:
 At load time, vLLM-Omni builds an `OmniINCConfig`, remaps checkpoint block names
 to runtime module names, and selects the matching vLLM compute backend.

-Checkpoint auto-detection and successful layer construction establish the
-**Integrated** level. Promote a model to **Validated** only after the named
-checkpoint completes an end-to-end generation or multimodal request and its
-output passes a model-appropriate sanity or quality check. Compare against the
-BF16 baseline for quality-sensitive image and video paths. Promote it to
-**CI-backed** only when that named checkpoint is selected by scheduled hardware
-CI.
-
 Example checkpoint creation:

 ```bash
@@ -9,10 +9,6 @@ stay in BF16/FP16. No pre-quantized checkpoint is required.
 This is an online (dynamic) path: load the normal HuggingFace checkpoint and
 quantize during model loading via the `bitsandbytes` CUDA kernels.

-The validation terms on this page follow the
-[support levels](overview.md#support-levels). BitsAndBytes kernel support does
-not imply that every diffusion transformer has been quality-validated.
-
 ## Hardware Support

 | Device | Support |
@@ -25,7 +21,7 @@ not imply that every diffusion transformer has been quality-validated.
 | Intel XPU | ❌ |
 | Ascend NPU | ❌ |

-Legend: `✅` backend available, `❌` unsupported. Non-CUDA platforms raise in
+Legend: `✅` supported, `❌` unsupported. Non-CUDA platforms raise in
 `get_quant_method()`; CUDA requires compute capability 7.5+ (`SM 75+`).

 Requires the optional `bitsandbytes` package (`pip install bitsandbytes`).
@@ -34,11 +30,11 @@ Requires the optional `bitsandbytes` package (`pip install bitsandbytes`).

 ### Diffusion Model (Qwen-Image, Wan2.2)

-| Model | HF models | Scope | Level | Recommendation |
-|-------|-----------|-------|-------|----------------|
-| Z-Image | `Tongyi-MAI/Z-Image-Turbo` | Diffusion transformer | Validated | All heavy linear layers; sensitive embedders stay BF16 |
-| Qwen-Image | `Qwen/Qwen-Image`, `Qwen/Qwen-Image-2512` | Diffusion transformer | Not validated | Compare with BF16 before enabling |
-| Wan2.2 | Wan2.2 diffusion pipelines | One or both diffusion transformers | Not validated | Validate quality and cascade behavior before enabling |
+| Model | HF models | CUDA | Mode | Recommendation |
+|-------|-----------|:----:|------|----------------|
+| Z-Image | `Tongyi-MAI/Z-Image-Turbo` | Yes | Online W4 weight-only | All heavy linear layers; sensitive embedders stay BF16 |
+| Qwen-Image | `Qwen/Qwen-Image`, `Qwen/Qwen-Image-2512` | Not validated | Online W4 weight-only | Compare vs BF16 before enabling |
+| Wan2.2 | Wan2.2 diffusion pipelines | Not validated | Online W4 weight-only | Validate before enabling in docs |

 Other diffusion models may work if their transformer uses supported linear
 layers, but they are not validated in this guide.
@@ -108,10 +104,6 @@ Multi-GPU tensor parallelism (`tensor_parallel_size` > 1) is not validated for
 BitsAndBytes in diffusion models; each rank quantizes its own weight shard
 independently.

-Z-Image-Turbo is validated outside scheduled full-model BitsAndBytes CI. The
-real CUDA kernel smoke tests cover quantization and matrix multiplication, but
-do not establish model-level quality for unlisted architectures.
-
 If quality regresses, use `ignored_layers` to keep sensitive projections in BF16
 (for example `to_out` or `w2`).

@@ -13,11 +13,6 @@ components, such as the tokenizer, scheduler, text encoder, vision/audio
 encoder, and VAE, on the base checkpoint unless a model-specific guide says
 otherwise.

-The validation terms on this page follow the
-[support levels](overview.md#support-levels) defined in the overview. ModelOpt
-format detection and checkpoint adaptation are shared runtime integrations;
-each model and stage still requires its own end-to-end validation.
-
 !!! note
    ModelOpt checkpoints are pre-quantized checkpoints. Do not pass
    `--quantization fp8` for these checkpoints. The checkpoint
@@ -63,11 +58,11 @@ stay unquantized unless the model-specific recipe validates otherwise.
 | Intel XPU | ⭕ |
 | Ascend NPU | ❌ |

-Legend: `✅` backend available, `❌` unsupported, `⭕` not verified in this
-guide. The optional CUTLASS FP8 runtime override requires CUDA SM89+. ModelOpt
-NVFP4 and mixed FP8/NVFP4 diffusion checkpoints are currently validated on
-Blackwell CUDA systems in the recipes below; other CUDA generations require
-separate backend and quality validation.
+Legend: `✅` supported, `❌` unsupported, `⭕` not verified in this guide.
+The optional CUTLASS FP8 runtime override requires CUDA SM89+. ModelOpt NVFP4
+and mixed FP8/NVFP4 diffusion checkpoints are currently validated on Blackwell
+CUDA systems in the recipes below; other CUDA generations require separate
+backend and quality validation.

 ## Model Type Support

@@ -84,9 +79,6 @@ separate backend and quality validation.
 | HunyuanImage-3.0 | `feizhai123/hunyuan-image3-modelopt-mixed-experts-nvfp4-dense-fp8` | MoE diffusion transformer | Validated for ModelOpt mixed FP8/NVFP4 checkpoints |
 | Wan2.2 | Not available | Diffusion transformer | Not validated |

-The validated diffusion entries above have named checkpoint and recipe
-evidence. They are not all exercised by scheduled full-model CI.
-
 For full serving commands and benchmark context, see
 [`recipes/Qwen/Qwen-Image.md`](https://github.com/vllm-project/vllm-omni/blob/main/recipes/Qwen/Qwen-Image.md)
 and
@@ -96,8 +88,8 @@ and

 | Model | Scope | Status |
 |-------|-------|--------|
-| Qwen3-Omni | Thinker language-model stage (FP8) | Validated outside scheduled CI |
-| Qwen3-Omni | Thinker language-model stage (W4A4 NVFP4) | CI-backed for the H100 Marlin fallback; native Blackwell path is recipe-validated |
+| Qwen3-Omni | Thinker language-model stage | ModelOpt FP8 checkpoint path |
+| Qwen3-Omni | Thinker language-model stage (W4A4 NVFP4) | Validated; see [Qwen3-Omni NVFP4 W4A4](#qwen3-omni-nvfp4-w4a4-thinker) below |
 | Qwen3-TTS | TTS language-model stage | Not validated |

 Audio encoder, vision encoder, talker, and code2wav stages stay in BF16 unless
@@ -110,9 +102,9 @@ Qwen3-Omni-30B-A3B-Instruct thinker language model. The thinker text body
 (attention + MoE experts) is quantized to NVFP4 with FP8 per-tensor input
 scales; the audio encoder, vision encoder, talker, and code2wav stay in BF16.

-| Variant | HF checkpoint | Hardware and validation |
-|---------|---------------|-------------------------|
-| W4A4 NVFP4 (full thinker) | `YihongJin/Qwen3-Omni-30B-A3B-Instruct-NVFP4-W4A4-full-thinker-awqclip` | sm_100+ native FlashInfer FP4 GEMM; H100 Marlin fallback is CI-backed |
+| Variant | HF checkpoint | Hardware |
+|---------|---------------|----------|
+| W4A4 NVFP4 (full thinker) | `YihongJin/Qwen3-Omni-30B-A3B-Instruct-NVFP4-W4A4-full-thinker-awqclip` | sm_100+ (Blackwell, FlashInfer FP4 GEMM) |

 Calibration uses ModelOpt `mtq.NVFP4_DEFAULT_CFG` with the `awq_clip` algorithm
 on 1024 ultrachat samples chat-templated through the Qwen3-Omni tokenizer.
@@ -240,6 +232,3 @@ omni = Omni(
 5. Keep `--quantization fp8` for online FP8 from BF16 checkpoints; use this
   ModelOpt path only when the checkpoint already contains ModelOpt quantized
   weights and scales.
-6. The Qwen3-Omni full-model CI smoke exercises the H100 Marlin fallback. A
-   scheduled Blackwell test is still required before treating the native
-   FlashInfer FP4 kernel path as CI-backed.
@@ -3,17 +3,12 @@
 ## Overview

 [msModelSlim](https://github.com/Ascend/msmodelslim) is an Ascend compression
-toolkit for producing pre-quantized model checkpoints. vLLM-Omni consumes its
-output either through a compatible generic Ascend checkpoint path or through
-the native MXFP8 and MXFP4 merge workflows documented in this guide.
+toolkit for producing pre-quantized model checkpoints. In vLLM-Omni, these
+checkpoints run through the Ascend/NPU path with `--quantization ascend`.

 msModelSlim is static quantization: quantized weights are generated offline
 before vLLM-Omni inference starts.

-The validation terms on this page follow the
-[support levels](overview.md#support-levels). A producer recipe or hardware
-backend alone is an integration point, not model-level validation.
-
 ## Hardware Support

 | Device | Support |
@@ -25,28 +20,23 @@ backend alone is an integration point, not model-level validation.
 | Intel XPU | ❌ |
 | Ascend NPU | ✅ |

-Legend: `✅` backend available, `❌` unsupported, `⭕` not verified in this
+Legend: `✅` supported, `❌` unsupported, `⭕` not verified in this
 guide.

 ## Model Type Support

 ### Diffusion Model (Qwen-Image, Wan2.2)

-| Model | Base model | Scope | Level | Notes |
-|-------|------------|-------|-------|-------|
-| Wan2.2 | Wan2.2 diffusion weights | DiT or diffusion stage | Integrated | Upstream msModelSlim provides a producer recipe; use the [MXFP8](mxfp8.md) or [MXFP4](mxfp4.md) workflow for vLLM-Omni's native merged formats |
-| Qwen-Image | `Qwen/Qwen-Image`, `Qwen/Qwen-Image-2512` | DiT or diffusion stage | Not validated | No supported checkpoint workflow is documented |
-| HunyuanImage-3.0 | `tencent/HunyuanImage-3.0`, `tencent/HunyuanImage-3.0-Instruct` | DiT or diffusion stage | Integrated | Generate quantized weights with the HunyuanImage-3.0 msModelSlim adaptation and validate against BF16 |
+| Model | Base model | Scope | Hardware | Notes |
+|-------|------------|-------|----------|-------|
+| Wan2.2 | Wan2.2 diffusion weights | DiT or diffusion stage | Ascend NPU | Upstream msModelSlim provides a Wan2.2 quantization recipe; vLLM-Omni inference validation is not listed |
+| Qwen-Image | `Qwen/Qwen-Image`, `Qwen/Qwen-Image-2512` | DiT or diffusion stage | Ascend NPU | Not validated in this guide |
+| HunyuanImage-3.0 | `tencent/HunyuanImage-3.0`, `tencent/HunyuanImage-3.0-Instruct` | DiT or diffusion stage | Ascend A2/A3 NPU | Generate quantized weights with the HunyuanImage-3.0 msModelSlim adaptation |

 Public Hugging Face quantized weights are not available yet. Use the
 [HunyuanImage-3.0 msModelSlim adaptation](https://gitcode.com/betta18/msmodelslim/tree/hyimage3_mxfp8)
 to generate the checkpoint manually.

-msModelSlim is also the producer used by the native offline MXFP8 and MXFP4
-DualScale workflows. Those workflows preprocess and annotate each transformer
-checkpoint and are auto-detected at inference; follow their method pages rather
-than passing `--quantization ascend`.
-
 ### Multi-Stage Omni/TTS Model (Qwen3-Omni, Qwen3-TTS)

 | Model | Scope | Status | Notes |
@@ -63,32 +53,25 @@ than passing `--quantization ascend`.

 ## Configuration

-For a compatible generic Ascend checkpoint, select the Ascend quantization
-method explicitly:
-
 Offline inference:

 ```bash
-python text_to_image.py --model <compatible-ascend-checkpoint> --quantization ascend
+python text_to_image.py --model <quantized-model-path> --quantization ascend
 ```

 Online serving:

 ```bash
-vllm serve <compatible-ascend-checkpoint> --omni --quantization ascend
+vllm serve <quantized-model-path> --omni --quantization ascend
 ```

 ## Parameters

 | Parameter | Type | Default | Description |
 |-----------|------|---------|-------------|
-| `--quantization` | str | - | Use `ascend` only for a compatible generic Ascend-format checkpoint; omit it for native merged MXFP8/MXFP4 checkpoints |
+| `--quantization` | str | - | Use `ascend` for msModelSlim-produced checkpoints |
 | `model` | str | - | Path to the quantized checkpoint generated by Ascend tooling |

-Do not pass `--quantization ascend` for a Wan checkpoint processed by the native
-MXFP8 or MXFP4 merge tools. Their transformer metadata selects `mxfp8` or
-`mxfp4_dualscale` automatically.
-
 Example msModelSlim command for a Wan2.2 W8A8 checkpoint:

 ```bash
@@ -110,6 +93,3 @@ For HunyuanImage-3.0, use the Hunyuan-specific adaptation linked above.
   tooling; it is not a load-time CUDA quantizer.
 3. Keep the quantized checkpoint aligned with the same model architecture and
   stage config used for BF16 inference.
-4. No named full-model msModelSlim checkpoint is currently scheduled in NPU
-   CI. Treat new producer outputs as **Integrated** until an end-to-end BF16
-   comparison establishes **Validated** support.
@@ -13,10 +13,6 @@ vLLM-Omni provides two quantization methods with different scale structures:
 | `mxfp4` | Single-scale (per-32 fine only) | Online only | Quick accuracy baseline; no checkpoint prep needed |
 | `mxfp4_dualscale` | Dual-scale (fine per-32 + coarse per-512 + per-channel `mul_scale`) | Online + Offline | Production; better accuracy; offline recommended |

-The validation terms on this page follow the
-[support levels](overview.md#support-levels). Current model guidance covers
-Wan diffusion transformers only; the text encoder and VAE stay in BF16.
-
 !!! tip "Recommended: `mxfp4_dualscale` offline"
    For production deployments, use the `mxfp4_dualscale` offline mode with a
    pre-quantized checkpoint produced by msModelSlim. Offline checkpoints load
@@ -38,45 +34,26 @@ Wan diffusion transformers only; the text encoder and VAE stay in BF16.

 ## Hardware Support

-| Device | Online `mxfp4` | Online `mxfp4_dualscale` | Offline `mxfp4_dualscale` |
-|--------|-----------------|---------------------------|----------------------------|
-| NVIDIA Blackwell GPU (SM 100+) | ❌ | ❌ | ❌ |
-| NVIDIA Ada/Hopper GPU (SM 89+) | ❌ | ❌ | ❌ |
-| NVIDIA Ampere GPU (SM 80+) | ❌ | ❌ | ❌ |
-| AMD ROCm (gfx950 / MI355X) | ✅ | ❌ | ❌ |
-| Intel XPU | ❌ | ❌ | ❌ |
-| Ascend NPU (Atlas 950 A5) | ✅ | ✅ | ✅ |
+| Device | Support |
+|--------|---------|
+| NVIDIA Blackwell GPU (SM 100+) | ⭕ |
+| NVIDIA Ada/Hopper GPU (SM 89+) | ⭕ |
+| NVIDIA Ampere GPU (SM 80+) | ⭕ |
+| AMD ROCm (gfx950 / MI355X) | ✅ |
+| Intel XPU | ⭕ |
+| Ascend NPU (Atlas 950 A5) | ✅ |

-Legend: `✅` backend available, `❌` unsupported, `⭕` not verified in this
-guide. Backend availability does not imply full-model checkpoint validation.
-
-!!! note "Why NVIDIA is marked unsupported"
-    This table describes the methods currently wired into vLLM-Omni, not every
-    low-precision format NVIDIA hardware could execute. `DiffusionMXFP4Config`
-    has runtime implementations only for Ascend and ROCm gfx950; on CUDA it
-    raises `NotImplementedError` because no CUDA MXFP4 linear method or
-    checkpoint loader is registered.
-
-    On NVIDIA, use [ModelOpt](modelopt.md) NVFP4 for the documented 4-bit
-    checkpoint path. NVFP4 and OCP MXFP4 have different checkpoint metadata,
-    scale layouts, and kernel contracts, so they are not interchangeable. A
-    native CUDA MXFP4 path would require a CUDA linear implementation, loader
-    contract, and model-level output validation.
-
-The ROCm path dispatches online single-scale `mxfp4` to AITER on gfx950. It has
-kernel and dispatch coverage, but this guide does not yet claim end-to-end
-quality validation for a named Wan checkpoint on ROCm. DualScale and
-pre-quantized MXFP4 checkpoints are not supported on ROCm.
+Legend: `✅` supported, `❌` unsupported, `⭕` not verified in this guide.

 ## Model Type Support

 ### Diffusion Model (Wan2.2)

-| Model | Online (Ascend) | Offline (Ascend) | Validation and notes |
-|-------|-----------------|------------------|----------------------|
-| Wan2.2-T2V-A14B | `mxfp4` / `mxfp4_dualscale` | `mxfp4_dualscale` | Validated outside scheduled full-model offline NPU CI; both cascade transformers are quantized |
-| Wan2.2-I2V-A14B | `mxfp4` / `mxfp4_dualscale` | `mxfp4_dualscale` | Validated outside scheduled full-model offline NPU CI; same cascade scheme as T2V-A14B |
-| Wan2.2-TI2V-5B | ❌ | ❌ | Unsupported because W4A4 causes unacceptable accuracy loss |
+| Model | Online | Offline | Notes |
+|-------|--------|---------|-------|
+| Wan2.2-T2V-A14B | `mxfp4` / `mxfp4_dualscale` | `mxfp4_dualscale` | MoE cascade (`transformer` + `transformer_2`); both transformers quantized with the same config |
+| Wan2.2-I2V-A14B | `mxfp4` / `mxfp4_dualscale` | `mxfp4_dualscale` | MoE cascade; same scheme as T2V-A14B |
+| Wan2.2-TI2V-5B | ❌ | ❌ | Parameter count too small; W4A4 causes unacceptable accuracy loss |

 The choice between `mxfp4` and `mxfp4_dualscale` in **online mode** is about
 quantization quality, not model compatibility — both work on cascade (A14B) and
@@ -106,13 +83,12 @@ by the merge script); there is no offline `mxfp4` single-scale format.

 ### `mxfp4` — Single-Scale Online Mode

-Online mode requires no pre-processing. On Ascend, vLLM-Omni quantizes BF16
-weights at load time using `npu_dynamic_mx_quant`. On ROCm gfx950, the same
-`mxfp4` config dispatches to the AITER online kernel. A single block scale
+Online mode requires no pre-processing. vLLM-Omni quantizes BF16 weights to
+MXFP4 at load time using `npu_dynamic_mx_quant`. A single block scale
 (`float8_e8m0fnu`, one per 32 K elements) is computed on the fly; no
-calibration `mul_scale` is available. The config applies equally to
-single-transformer and cascade (A14B) models, but named Wan model validation is
-currently limited to the documented Ascend path.
+calibration `mul_scale` is available. Applies equally to single-transformer
+and cascade (A14B) models — both transformers in a cascade receive the same
+quantization config automatically.

 ```python
 from vllm_omni import Omni
@@ -135,10 +111,10 @@ vllm serve <your-model> --omni --quantization mxfp4

 ### `mxfp4_dualscale` — DualScale Online Mode

-Ascend-only online DualScale mode computes both fine and coarse scales on the
-fly from BF16 weights using `npu_dynamic_dual_level_mx_quant`. It applies
-equally to single-transformer and cascade (A14B) models. Compared to `mxfp4`
-online, DualScale provides better quantization accuracy at higher compute cost.
+Online DualScale mode computes both fine and coarse scales on the fly from BF16
+weights using `npu_dynamic_dual_level_mx_quant`. Applies equally to
+single-transformer and cascade (A14B) models. Compared to `mxfp4` online,
+DualScale provides better quantization accuracy at higher compute cost.

 The default configuration keeps the leading 5 transformer blocks in BF16
 (`num_bf16_fallback_layers=5`). Accuracy evaluation on Wan2.2-A14B shows this
@@ -365,11 +341,6 @@ unless they appear in `ignored_layers`.
   quantization. Use `ignored_layers` and `num_bf16_fallback_layers` to trade
   off compression vs. accuracy for precision-sensitive layers.

-8. The native methods, merge mappings, and per-transformer cascade metadata
-   have unit coverage. The current Buildkite configuration does not schedule a
-   named full-model offline MXFP4 checkpoint on NPU, so this path is
-   **Validated**, not **CI-backed**.
-
 ## Adapting MXFP4 for a New Model

 This section is aimed at developers who want to add MXFP4 support to a model
@@ -15,45 +15,30 @@ This method supports three modes:
 | **Offline (Native)** | msModelSlim-exported MXFP8 weights converted to diffusers format via `merge_mxfp8_checkpoint.py` — weights and scales are loaded directly from the preprocessed checkpoint |
 | **Offline (AutoRound)** | AutoRound MXFP8 checkpoints with `data_type="mx_fp"` — auto-detected from `config.json` |

-The validation terms on this page follow the
-[support levels](overview.md#support-levels). Current model guidance covers
-Wan diffusion transformers only; the text encoder and VAE stay in BF16.
-
 ## Hardware Support

 | Device | Online | Offline (Native) | Offline (AutoRound) |
 |--------|--------|------------------|---------------------|
-| NVIDIA Blackwell GPU (SM 100+) | ❌ | ❌ | ⭕ |
-| NVIDIA Ada/Hopper GPU (SM 89+) | ❌ | ❌ | ⭕ |
-| NVIDIA Ampere GPU (SM 80+) | ❌ | ❌ | ⭕ |
-| AMD ROCm | ❌ | ❌ | ⭕ |
+| NVIDIA Blackwell GPU (SM 100+) | ⭕ | ⭕ | ⭕ |
+| NVIDIA Ada/Hopper GPU (SM 89+) | ⭕ | ⭕ | ⭕ |
+| NVIDIA Ampere GPU (SM 80+) | ⭕ | ⭕ | ⭕ |
+| AMD ROCm | ⭕ | ⭕ | ⭕ |
 | Intel XPU | ✅ | ❌ | ✅ |
 | Ascend NPU (Atlas 950 A5) | ✅ | ✅ | ⭕ |

-Legend: `✅` backend available, `❌` unsupported, `⭕` not verified in this
-guide. Backend availability does not imply full-model checkpoint validation.
+Legend: `✅` supported, `❌` unsupported, `⭕` not verified in this guide.

-The native `DiffusionMXFP8Config` currently dispatches only to Intel XPU and
-Ascend NPU implementations. CUDA reaches `NotImplementedError`, so the NVIDIA
-rows are marked unsupported even though this is an implementation gap rather
-than a general claim about NVIDIA hardware capabilities. On NVIDIA, use
-[FP8](fp8.md) for load-time quantization or [ModelOpt](modelopt.md) FP8 for
-pre-quantized checkpoints. Adding native CUDA MXFP8 requires a registered CUDA
-linear method, a compatible checkpoint contract, and model-level validation.
-
-Native `mxfp8` is available online on Intel XPU and online or offline on
-Ascend. Intel XPU only supports AutoRound MXFP8 for offline mode. The AutoRound
-column is a separate checkpoint format and compute path.
+**Note**: Intel XPU only supports AutoRound MXFP8 for offline mode. Use AutoRound quantized checkpoints or online mode for XPU.

 ## Model Type Support

 ### Diffusion Model (Wan2.2)

-| Model | Mode | Scope | Validation |
-|-------|------|-------|------------|
-| Wan2.2-T2V-A14B | Online + Offline | MoE cascade: `transformer` + `transformer_2` | Validated outside scheduled full-model offline NPU CI |
-| Wan2.2-I2V-A14B | Online + Offline | MoE cascade: `transformer` + `transformer_2` | Validated outside scheduled full-model offline NPU CI |
-| Wan2.2-TI2V-5B | Online + Offline | Single diffusion transformer | Validated outside scheduled full-model offline NPU CI |
+| Model | Mode | Notes |
+|-------|------|-------|
+| Wan2.2-T2V-A14B | Online + Offline | MoE cascade; quantizes two transformers (`transformer` + `transformer_2`) |
+| Wan2.2-I2V-A14B | Online + Offline | MoE cascade; quantizes two transformers (`transformer` + `transformer_2`) |
+| Wan2.2-TI2V-5B | Online + Offline | Single transformer |

 ### Multi-Stage Omni/TTS Model (Qwen3-Omni, Qwen3-TTS)

@@ -233,7 +218,3 @@ omni = Omni(model="<autoround-mxfp8-model>")
   `--output-dir`), regenerate it with the current script. The old script
   wrote a separate `quantization_config.json` that is not read by vLLM-Omni;
   the current script injects the config directly into `transformer/config.json`.
-4. The native loader, merge mappings, and Wan cascade config propagation have
-   unit coverage. The current Buildkite configuration does not schedule a
-   named full-model offline MXFP8 checkpoint on NPU, so this path is
-   **Validated**, not **CI-backed**.
@@ -6,7 +6,7 @@ Online quantization means vLLM-Omni computes quantized weights and scales while
 loading the model. Use it when you want memory savings without preparing a
 separate quantized checkpoint.

-This mode is different from pre-quantized checkpoint formats such as ModelOpt,
+This mode is different from pre-quantized checkpoint formats such as GGUF,
 AutoRound, msModelSlim, or serialized Int8 checkpoints. Those formats are
 prepared before serving and are documented in their method-specific guides.
 For MXFP8 and MXFP4, use this page for load-time quantization from BF16
@@ -15,22 +15,18 @@ by msModelSlim and the merge tools.

 ## Hardware Support

-| Device | FP8 W8A8 | Int8 W8A8 | BitsAndBytes W4 | MXFP8 W8A8 | MXFP4 W4A4 |
-|--------|----------|-----------|------------------|------------|------------|
-| NVIDIA Blackwell GPU (SM 100+) | ✅ | ✅ | ✅ | ❌ | ❌ |
-| NVIDIA Ada/Hopper GPU (SM 89+) | ✅ | ✅ | ✅ | ❌ | ❌ |
-| NVIDIA Ampere GPU (SM 80+) | ✅ | ✅ | ✅ | ❌ | ❌ |
-| AMD ROCm (gfx950) | ⭕ | ⭕ | ❌ | ❌ | ✅ |
-| Intel XPU | ⭕ | ⭕ | ❌ | ✅ | ❌ |
-| Ascend NPU | ❌ | ✅ | ❌ | ✅ | ✅ |
+| Device | FP8 W8A8 | Int8 W8A8 | MXFP8 W8A8 | MXFP4 W4A4 |
+|--------|----------|-----------|------------|------------|
+| NVIDIA Blackwell GPU (SM 100+) | ✅ | ✅ | ⭕ | ⭕ |
+| NVIDIA Ada/Hopper GPU (SM 89+) | ✅ | ✅ | ⭕ | ⭕ |
+| NVIDIA Ampere GPU (SM 80+) | ✅ | ✅ | ⭕ | ⭕ |
+| AMD ROCm | ⭕ | ⭕ | ⭕ | ⭕ |
+| Intel XPU | ⭕ | ⭕ | ✅ | ⭕ |
+| Ascend NPU | ❌ | ✅ | ✅ | ✅ |

-Legend: `✅` backend available, `❌` unsupported, `⭕` not verified in this
-guide. These entries describe method-level backend availability, not validation
-for every model. FP8 on Ampere may use a weight-only path where available.
-On ROCm gfx950, MXFP4 means online single-scale `mxfp4`; online and offline
-`mxfp4_dualscale` are Ascend-only. The native MXFP8/MXFP4 configs do not
-currently register CUDA implementations; use FP8 or ModelOpt FP8/NVFP4 on
-NVIDIA instead.
+Legend: `✅` supported, `❌` unsupported, `⭕` not verified in this
+guide. FP8 on Ampere may use a weight-only path where available. MXFP8 and
+MXFP4 are documented for the Ascend NPU path.

 ## Model Type Support

@@ -40,9 +36,8 @@ NVIDIA instead.
 |--------|-------|----------------|--------|
 | FP8 W8A8 | [FP8](fp8.md) | Qwen-Image; Wan2.2 is not validated | Validated for Qwen-Image family and other DiT models |
 | Int8 W8A8 | [Int8](int8.md) | Qwen-Image; Wan2.2 is not validated | Validated for Qwen-Image and Z-Image |
-| BitsAndBytes W4 | [BitsAndBytes](bitsandbytes.md) | Z-Image-Turbo; Qwen-Image and Wan2.2 are not validated | Validated for Z-Image-Turbo |
 | MXFP8 W8A8 | [MXFP8](mxfp8.md) | Wan2.2-T2V-A14B, Wan2.2-I2V-A14B, Wan2.2-TI2V-5B | Validated on Ascend NPU and Intel XPU |
-| MXFP4 W4A4 | [MXFP4](mxfp4.md) | Wan2.2-T2V-A14B, Wan2.2-I2V-A14B | Validated on Ascend NPU; the ROCm gfx950 backend is integrated but not model-validated; TI2V-5B is unsupported |
+| MXFP4 W4A4 | [MXFP4](mxfp4.md) | Wan2.2-T2V-A14B, Wan2.2-I2V-A14B | Ascend NPU only; TI2V-5B is not supported |

 ### Multi-Stage Omni/TTS Model (Qwen3-Omni, Qwen3-TTS)

@@ -64,7 +59,6 @@ from vllm_omni import Omni

 omni_fp8 = Omni(model="<your-model>", quantization="fp8")
 omni_int8 = Omni(model="<your-model>", quantization="int8")
-omni_bnb = Omni(model="Tongyi-MAI/Z-Image-Turbo", quantization="bitsandbytes")
 omni_mxfp8 = Omni(model="<your-model>", quantization="mxfp8")
 omni_mxfp4 = Omni(model="<your-model>", quantization="mxfp4")
 omni_mxfp4_dualscale = Omni(model="<your-model>", quantization="mxfp4_dualscale")
@@ -75,7 +69,6 @@ CLI:
 ```bash
 vllm serve <your-model> --omni --quantization fp8
 vllm serve <your-model> --omni --quantization int8
-vllm serve Tongyi-MAI/Z-Image-Turbo --omni --quantization bitsandbytes
 vllm serve <your-model> --omni --quantization mxfp8
 vllm serve <your-model> --omni --quantization mxfp4
 vllm serve <your-model> --omni --quantization mxfp4_dualscale
@@ -92,19 +85,14 @@ config = build_quant_config({
 })
 ```

-Component routing is an extension mechanism, not automatic support for every
-pipeline component. See [Quantization Scope](overview.md#quantization-scope)
-before applying it to an encoder, VAE, or decoder.
-
 ## Parameters

 | Parameter | Methods | Description |
 |-----------|---------|-------------|
-| `method` | FP8, Int8, BitsAndBytes, MXFP8, MXFP4 | Quantization method: `"fp8"`, `"int8"`, `"bitsandbytes"`, `"mxfp8"`, `"mxfp4"`, or `"mxfp4_dualscale"` |
-| `ignored_layers` | FP8, Int8, BitsAndBytes, MXFP8, MXFP4 | Layer name patterns to keep in BF16/FP16 |
+| `method` | FP8, Int8, MXFP8, MXFP4 | Quantization method: `"fp8"`, `"int8"`, `"mxfp8"`, `"mxfp4"`, or `"mxfp4_dualscale"` |
+| `ignored_layers` | FP8, Int8, MXFP8, MXFP4 | Layer name patterns to keep in BF16/FP16 |
 | `activation_scheme` | FP8, Int8 | The runtime value `"dynamic"` selects online activation scaling |
 | `weight_block_size` | FP8 | Optional block-wise FP8 weight quantization size |
-| `quant_type` | BitsAndBytes | Weight format: `"nf4"` (default) or `"fp4"` |
 | `num_bf16_fallback_layers` | MXFP4 DualScale | Leading transformer blocks to keep in BF16 for online `mxfp4_dualscale`; defaults to `5` |

 ## Validation and Notes
@@ -16,123 +16,72 @@ For the internal architecture and backend extension points, see the
 | Runtime attention quantization | [Quantized KV Cache](quantized_kvcache.md) | vLLM-Omni dynamically quantizes eligible diffusion Flash Attention tensors during inference. | FP8 FA |
 | Pre-quantized checkpoints | Method-specific guides | The checkpoint or an offline quantizer provides quantized weights and scales before serving. | ModelOpt, AutoRound, msModelSlim, serialized Int8, offline MXFP8, offline MXFP4 DualScale |

-Online quantization starts from a normal BF16/FP16 checkpoint and repeats the
-conversion on each fresh model load. It is the simplest path for experiments.
-Pre-quantized checkpoints store the packed weights and scales produced by a
-separate tool; they are easier to reuse across deployments and can carry
-calibrated or mixed-precision policies that load-time conversion cannot
-reproduce.
+## Hardware Support

-## Support Levels
+| Device | FP8 W8A8 | Int8 W8A8 | BitsAndBytes W4 | ModelOpt | MXFP8 W8A8 | MXFP4 W4A4 | AutoRound | msModelSlim |
+|--------|----------|-----------|------------------|----------|------------|------------|-----------|-------------|
+| NVIDIA Blackwell GPU (SM 100+) | ✅ | ✅ | ✅ | ✅ | ⭕ | ⭕ | ✅ | ❌ |
+| NVIDIA Ada/Hopper GPU (SM 89+) | ✅ | ✅ | ✅ | ✅ | ⭕ | ⭕ | ✅ | ❌ |
+| NVIDIA Ampere GPU (SM 80+) | ✅ | ✅ | ✅ | ⭕ | ⭕ | ⭕ | ✅ | ❌ |
+| AMD ROCm | ⭕ | ⭕ | ❌ | ⭕ | ⭕ | ✅ | ⭕ | ❌ |
+| Intel XPU | ⭕ | ⭕ | ❌ | ⭕ | ⭕ | ⭕ | ✅ | ❌ |
+| Ascend NPU | ❌ | ✅ | ❌ | ❌ | ✅ | ✅ | ❌ | ✅ |

-Quantization support is specific to a model, component, checkpoint format, and
-hardware backend. The tables in this guide use the following levels:
+Legend: `✅` supported, `❌` unsupported, `⭕` not verified in this
+guide. FP8 on Ampere may use a weight-only path where available.

-| Level | Meaning |
-|-------|---------|
-| **CI-backed** | A named model and quantized checkpoint are exercised by scheduled full-model hardware CI. |
-| **Validated** | A named model and checkpoint have end-to-end or quality-validation evidence, but the check is not necessarily scheduled in CI. |
-| **Integrated** | The config, loader, quantized layers, or model wiring exists, but this guide has no completed end-to-end validation for the named model and checkpoint. |
-| **Not validated** | There is no current support recommendation for this model/component combination. |
-| **Unsupported** | The combination is intentionally rejected or known not to work. |
+## Model Type Support

-A method being available on a device does not make every model on that device
-supported. Likewise, successfully detecting a checkpoint's
-`quantization_config` establishes loader compatibility, not output quality.
+### Diffusion Model (Qwen-Image, Wan2.2)

-## Hardware × Quantization Method
+These models run a diffusion transformer as the primary inference module. The
+default quantization target is the transformer; tokenizer, scheduler, text
+encoder, and VAE stay on the base checkpoint unless a method guide says
+otherwise.

-This matrix uses concrete inference paths as columns. Producer toolkits are not
-separate methods: NVIDIA ModelOpt produces the ModelOpt FP8/NVFP4 formats,
-Intel AutoRound produces AutoRound checkpoints, and Ascend msModelSlim produces
-checkpoints consumed by the native MXFP or compatible Ascend paths.
+| Method | Guide | Mode | Example models | Status |
+|--------|-------|------|----------------|--------|
+| FP8 W8A8 | [FP8](fp8.md) | Online W8A8 or checkpoint FP8 | Qwen-Image; Wan2.2 is not validated | Validated for Qwen-Image family and other DiT models |
+| Int8 W8A8 | [Int8](int8.md) | Online or serialized W8A8 | Qwen-Image; Wan2.2 is not validated | Validated for Qwen-Image and Z-Image |
+| BitsAndBytes W4 | [BitsAndBytes](bitsandbytes.md) | Online W4 weight-only | Z-Image-Turbo; Qwen-Image/Wan2.2 not validated | Validated for Z-Image-Turbo |
+| ModelOpt | [ModelOpt](modelopt.md) | Pre-quantized FP8 checkpoints | Qwen-Image, Z-Image, FLUX.2, HunyuanImage-3.0 | Validated for ModelOpt FP8 diffusion checkpoints |
+| MXFP8 W8A8 | [MXFP8](mxfp8.md) | Online W8A8 or offline pre-quantized | Wan2.2-T2V-A14B, I2V-A14B, TI2V-5B | Ascend NPU only; validated for Wan2.2 |
+| MXFP4 W4A4 | [MXFP4](mxfp4.md) | `mxfp4`: online single-scale only; `mxfp4_dualscale`: online or offline dual-scale (offline recommended) | Wan2.2-T2V-A14B, I2V-A14B | Ascend NPU only; validated for Wan2.2 A14B cascade models; TI2V-5B not supported; offline `mxfp4_dualscale` uses calibrated `mul_scale` for best accuracy |
+| AutoRound | [AutoRound](autoround.md) | Pre-quantized W4A16 checkpoints | FLUX.1-dev; Qwen-Image/Wan2.2 not validated | Checkpoint-driven |
+| msModelSlim | [msModelSlim](msmodelslim.md) | Pre-quantized Ascend checkpoints | Wan2.2 recipe; HunyuanImage-3.0 inference target | Ascend/NPU path |

-| Hardware | [Online FP8](fp8.md) | [Int8](int8.md) | [BitsAndBytes W4](bitsandbytes.md) | [ModelOpt FP8](modelopt.md) | [ModelOpt NVFP4](modelopt.md#supported-modelopt-checkpoint-formats) | [AutoRound W4A16](autoround.md) | [OCP MXFP8](mxfp8.md) | [OCP MXFP4](mxfp4.md) |
-|----------|-----------------------|------------------|------------------------------------------|--------------------------------|-------------------------------------------------------------------------|--------------------------------------|--------------------------|--------------------------|
-| NVIDIA Blackwell (SM 100+) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ |
-| NVIDIA Hopper (SM 90) | ✅ | ✅ | ✅ | ✅ | ✅ (Marlin fallback) | ✅ | ❌ | ❌ |
-| NVIDIA Ada (SM 89) | ✅ | ✅ | ✅ | ✅ | ⭕ | ✅ | ❌ | ❌ |
-| NVIDIA Ampere (SM 80+) | ✅ | ✅ | ✅ | ⭕ | ⭕ | ✅ | ❌ | ❌ |
-| AMD ROCm (gfx950 for MXFP4) | ⭕ | ⭕ | ❌ | ⭕ | ⭕ | ⭕ | ❌ | ✅ (online only) |
-| Intel XPU | ⭕ | ⭕ | ❌ | ⭕ | ⭕ | ✅ | ✅ | ❌ |
-| Ascend NPU | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ |
+### Multi-Stage Omni/TTS Model (Qwen3-Omni, Qwen3-TTS)

-Legend: `✅` backend available, `❌` rejected or no implementation, `⭕` not
-verified in this guide. This matrix does not establish model-level support; use
-the model-and-stage tables below for that.
+These models combine an AR language model with audio, vision, talker, or TTS
+stages. Quantization is scoped to the AR language-model stage when the
+checkpoint contains a supported `quantization_config`; the non-AR stages stay
+in BF16 unless the model guide explicitly adds support.

-The Int8 column covers online and serialized Int8 paths. OCP MXFP8 is online on
-Intel XPU and online/offline on Ascend; Intel's separate AutoRound MXFP8
-checkpoint path is described in the [MXFP8 guide](mxfp8.md). OCP MXFP4 on AMD
-means online single-scale `mxfp4` on gfx950, while MXFP4 DualScale is
-Ascend-only. NVIDIA's NVFP4 and OCP MXFP4 have different checkpoint metadata,
-scale layouts, and kernel contracts. Mixed ModelOpt FP8/NVFP4 checkpoints also
-require a model-specific mixed-precision policy and validation. FP8 on Ampere
-may use a weight-only path where available.
+| Method | Guide | Scope | Example models | Status |
+|--------|-------|-------|----------------|--------|
+| ModelOpt | [ModelOpt](modelopt.md) | Thinker or language-model checkpoint config | Qwen3-Omni thinker | ModelOpt checkpoint path |
+| Int8 | [Int8](int8.md) | Not currently validated for omni/TTS stages | Qwen3-Omni, Qwen3-TTS | Not validated |
+| BitsAndBytes | [BitsAndBytes](bitsandbytes.md) | Not currently validated for omni/TTS stages | Qwen3-Omni, Qwen3-TTS | Not validated |
+| MXFP8 | [MXFP8](mxfp8.md) | Not currently validated for omni/TTS stages | Qwen3-Omni, Qwen3-TTS | Not validated |
+| MXFP4 | [MXFP4](mxfp4.md) | Not currently validated for omni/TTS stages | Qwen3-Omni, Qwen3-TTS | Not validated |
+| AutoRound | [AutoRound](autoround.md) | Thinker or language-model checkpoint config | Qwen2.5-Omni, Qwen3-Omni | Supported through AutoRound checkpoints |
+| msModelSlim | [msModelSlim](msmodelslim.md) | Not currently validated for omni/TTS stages | Qwen3-Omni, Qwen3-TTS | Not validated |

-## Current Model and Stage Support
+### Multi-Stage Diffusion Model (BAGEL, GLM-Image)

-The current offline checkpoint guidance is transformer-first. A listed
-checkpoint quantizes the stage named in the **Scope** column; it does not imply
-that the model's encoders, VAE, decoder, or other stages are also quantized.
+These models split generation across multiple stages. Quantization must be
+attached to the intended stage rather than applied globally.

-### Vendor and Runtime Ecosystems
-
-The producer and runtime backend are related, but they are not the same support
-claim. A producer can emit several checkpoint formats, and each format still
-needs a compatible vLLM-Omni loader, model-stage wiring, and validation. A
-runtime may also provide online quantization without an offline producer path.
-
-| Ecosystem | Current paths | Current vLLM-Omni scope | Model evidence |
-|-----------|---------------|-------------------------|----------------|
-| NVIDIA | ModelOpt FP8, NVFP4, and mixed FP8/NVFP4 | Diffusion transformer or Qwen-Omni thinker | Qwen-Image and other named diffusion checkpoints are validated; Qwen3-Omni NVFP4 has H100 CI for the Marlin fallback |
-| Intel | AutoRound W4A16 and AutoRound MXFP8 | Diffusion/world-model transformer or Qwen-Omni thinker | Wan A14B and GLM-Image W4A16 are CI-backed; Qwen-Image, Qwen-Omni, and Cosmos3-Super have validation outside scheduled CI |
-| AMD ROCm | Online OCP MXFP4 through AITER on gfx950; no pre-quantized MXFP4 or AMD Quark workflow is integrated | Diffusion transformer linear layers | Integrated backend with dispatch and weight-allocation unit coverage; no named-model quality validation or scheduled gfx950 full-model CI |
-| Ascend | msModelSlim output through native MXFP8/MXFP4 merge workflows or a compatible generic Ascend format | Wan diffusion transformers | Native Wan MXFP8/MXFP4 paths are validated outside scheduled full-model NPU CI; other model families remain integrated or unvalidated |
-
-### Pre-Quantized Model Matrix
-
-| Model family | Method | Scope | Level | Notes |
-|--------------|--------|-------|-------|-------|
-| Qwen-Image | [ModelOpt](modelopt.md) FP8 or mixed FP8/NVFP4 | Diffusion transformer | Validated | Named Qwen-Image 2512 checkpoints and recipes are available. Text encoder and VAE stay BF16. |
-| Qwen-Image | [AutoRound](autoround.md) W4A16 | Diffusion transformer | Validated | Uses `INC4AI/Qwen-Image-AutoRound-W4A16`; text encoder and VAE stay BF16. |
-| Wan2.2 I2V/T2V A14B | [AutoRound](autoround.md) W4A16 | Both diffusion transformers | CI-backed | The high-noise and low-noise DiTs are quantized; text encoder and VAE stay BF16. |
-| Wan2.2 TI2V 5B | [AutoRound](autoround.md) W4A16 | Diffusion transformer | Validated | Named checkpoint available; not covered by the scheduled Wan AutoRound job. |
-| Wan2.2 | [MXFP8](mxfp8.md) | One or both diffusion transformers | Validated | Native Ascend and AutoRound checkpoint formats are integrated; no scheduled full-model offline NPU job. |
-| Wan2.2 I2V/T2V A14B | [MXFP4 DualScale](mxfp4.md) | Both diffusion transformers | Validated | Offline calibrated checkpoints are recommended; no scheduled full-model offline NPU job. |
-| Qwen2.5-Omni | [AutoRound](autoround.md) W4A16 | Thinker language model | Validated | Encoders and output/audio stages stay BF16. |
-| Qwen3-Omni | [ModelOpt](modelopt.md) FP8 | Thinker language model | Validated | Tested outside scheduled CI. |
-| Qwen3-Omni | [ModelOpt](modelopt.md) NVFP4 | Thinker language model | CI-backed | H100 CI exercises the Marlin fallback; the native Blackwell FP4 path has separate recipe validation. |
-| Qwen3-Omni | [AutoRound](autoround.md) W4A16 | Thinker language model | Validated | Audio encoder, vision encoder, talker, and Code2Wav stay BF16. |
-| GLM-Image | [AutoRound](autoround.md) W4A16 | Diffusion transformer stage | CI-backed | Text-to-image and image-to-image full-model smoke coverage. |
-| Cosmos3-Super | [AutoRound](autoround.md) W4A16 | World-model transformer | Validated | Manually validated serving path; VAE and other components stay BF16. |
-| Cosmos3-Nano | [AutoRound](autoround.md) W4A16 | World-model transformer | Integrated | Named checkpoint is listed, but no model-specific end-to-end test is maintained in-tree. |
-| MiniMax H3 | None | None | Not validated | No offline vendor checkpoint path is documented; online FP8 targets the DiT only. |
-
-The method pages contain the complete checkpoint lists, including FLUX,
-Z-Image, and HunyuanImage-3.0 variants.
-
-### Online Quantization
-
-| Model family | Methods | Scope | Level |
-|--------------|---------|-------|-------|
-| Qwen-Image | FP8, Int8 | Diffusion transformer | Validated |
-| Z-Image | Int8, BitsAndBytes W4 | Diffusion transformer | Validated |
-| Wan2.2 | MXFP8 | One or both diffusion transformers | Validated on the documented XPU/NPU paths |
-| Wan2.2 A14B | MXFP4 | Both diffusion transformers | Validated on Ascend; the ROCm gfx950 online backend is integrated but not model-validated |
-| MiniMax H3 | FP8 | DiT and optional reference DiT | Validated |
-| Qwen3-Omni | Generic online methods, such as FP8 | Thinker language model only | Integrated; not model-validated as an online recommendation |
-
-See [Online Quantization](online.md) for configuration and hardware details.
-
-### Unlisted and Long-Tail Models
-
-For models not listed above, generic quantized linear layers or compatible
-checkpoint metadata are only an integration starting point. The default support
-target is the model's primary transformer, DiT, or AR language-model stage.
-Text/audio/vision encoders, VAEs, vocoders, decoders, and output stages remain
-BF16 until a model-specific integration supplies quant-aware layers, checkpoint
-mapping, end-to-end output validation, and backend coverage.
+| Method | Guide | Scope | Example models | Status |
+|--------|-------|-------|----------------|--------|
+| FP8 | [FP8](fp8.md) | Stage-specific DiT or transformer module | BAGEL, GLM-Image | Requires model-specific validation |
+| Int8 | [Int8](int8.md) | Stage-specific DiT or transformer module | BAGEL, GLM-Image | Requires model-specific validation |
+| BitsAndBytes | [BitsAndBytes](bitsandbytes.md) | Stage-specific transformer or DiT module | BAGEL, GLM-Image | Not validated |
+| ModelOpt | [ModelOpt](modelopt.md) | Checkpoint-defined diffusion stage | BAGEL, GLM-Image | Requires model-specific validation |
+| MXFP8 | [MXFP8](mxfp8.md) | Stage-specific DiT or transformer module | BAGEL, GLM-Image | Not validated |
+| MXFP4 | [MXFP4](mxfp4.md) | Stage-specific DiT or transformer module | BAGEL, GLM-Image | Not validated |
+| AutoRound | [AutoRound](autoround.md) | Checkpoint-defined stage | BAGEL, GLM-Image | No validated checkpoint listed |
+| msModelSlim | [msModelSlim](msmodelslim.md) | Ascend-generated stage weights | GLM-Image | Requires model-specific adaptation |

 !!! note
    "Online quantization" means vLLM-Omni computes the quantization data while
@@ -143,8 +92,8 @@ mapping, end-to-end output validation, and backend coverage.

 ### Diffusion Model (Qwen-Image, Wan2.2)

-The default target is the diffusion transformer. `build_quant_config()` can
-construct a component router:
+The default target is the diffusion transformer. Component routing is available
+through `build_quant_config()`:

 ```python
 from vllm_omni.quantization import build_quant_config
@@ -155,19 +104,11 @@ config = build_quant_config({
 })
 ```

-!!! warning "Routing syntax is not a support claim"
-    A component dictionary only selects a quantization config by module prefix.
-    It does not make an encoder or VAE quantizable. The model must construct
-    that component from quantization-aware layers, and an offline checkpoint
-    must contain matching packed weights and scales. Current user guidance
-    keeps diffusion text encoders and VAEs in BF16.
-
 | Component | Default quantized? | Notes |
 |-----------|--------------------|-------|
-| Diffusion transformer | Yes, for listed model/method pairs | Primary target for FP8, Int8, BitsAndBytes, ModelOpt, MXFP8, MXFP4, AutoRound, and msModelSlim |
-| Second diffusion transformer | Yes, for listed Wan2.2 A14B methods | `transformer` and `transformer_2` read their own offline checkpoint metadata; arbitrary mixed methods are not currently validated |
+| Diffusion transformer | Yes | Primary target for FP8, Int8, BitsAndBytes, ModelOpt, MXFP8, MXFP4, AutoRound, and msModelSlim |
 | Text encoder | No | Keep BF16 unless a method-specific guide documents support |
-| VAE | No | Keep BF16 |
+| VAE | No | Keep BF16; storage-only paths are method-specific |
 | Scheduler/tokenizer | No | Loaded from the base model repository |

 ### Multi-Stage Omni/TTS Model (Qwen3-Omni, Qwen3-TTS)
@@ -180,18 +121,6 @@ config = build_quant_config({
 | Talker or TTS stage | No | BF16 unless model-specific support is documented |
 | Code2Wav | No | BF16 |

-An explicit `ComponentQuantizationConfig` can be used by model developers to
-route different configs to `audio_tower`, `visual`, and `language_model`, but
-the supported Qwen-Omni checkpoints currently quantize only the thinker
-language model.
-
-### World Model (Cosmos3)
-
-| Component | Default quantized? | Notes |
-|-----------|--------------------|-------|
-| World-model transformer | AutoRound checkpoints only | Cosmos3-Super is validated; Cosmos3-Nano is integrated but not end-to-end tested in-tree |
-| VAE and guardrail components | No | BF16 |
-
 ### Multi-Stage Diffusion Model (BAGEL, GLM-Image)

 | Component | Default quantized? | Notes |
@@ -212,7 +141,6 @@ build_quant_config("fp8")
 build_quant_config({"method": "fp8", "activation_scheme": "static"})
 build_quant_config("bitsandbytes")
 build_quant_config("auto-round", bits=4, group_size=128)
-# Component routing syntax; model-specific support is still required.
 build_quant_config({"transformer": {"method": "fp8"}, "vae": None})
 build_quant_config(None)
 ```
@@ -77,6 +77,7 @@ plugins:
        user_guide/diffusion/step_execution.md: user_guide/diffusion/execution_modes.md
        user_guide/diffusion/cpu_offload_diffusion.md: user_guide/diffusion/cpu_offload.md
        user_guide/quantization/index.md: user_guide/quantization/overview.md
+        configuration/pd_disaggregation.md: features/pd_disaggregation.md
        design/feature/diffusion_request_level_batching.md: design/feature/diffusion_continuous_batching.md
        design/feature/diffusion_step_execution.md: design/feature/diffusion_continuous_batching.md
  - glightbox
@@ -95,7 +95,6 @@ def test_formatter_preserves_single_video_audio_actions_and_metadata(
        "resolution": 512,
        "width": None,
        "height": None,
-        "num_inference_steps": 1,
    }


@@ -42,10 +42,6 @@ _EXPECTED_FAMILIES = [
    defs.AUDIO_UNDERRUN_S,
    defs.AUDIO_CONTINUITY_OK_METRIC,
    defs.AUDIO_SKIPPED_REQUESTS_METRIC,
-    defs.DIFFUSION_EXEC_S,
-    defs.DIFFUSION_EXEC_PER_STEP_S,
-    defs.DIFFUSION_PREPROCESS_S,
-    defs.DIFFUSION_POSTPROCESS_S,
 ]


@@ -59,10 +55,6 @@ class TestRegistration:
        mod.observe_audio_underrun("s", "r", 0.01)
        mod.inc_audio_continuity_ok("s", "r", 100)
        mod.inc_audio_skipped("s", "r", "malformed_codec")
-        mod.observe_diffusion_exec("s", "r", 0.5)
-        mod.observe_diffusion_exec_per_step("s", "r", 0.01)
-        mod.observe_diffusion_preprocess("s", "r", 0.01)
-        mod.observe_diffusion_postprocess("s", "r", 0.2)

        out = generate_latest(REGISTRY).decode()
        for name in _EXPECTED_FAMILIES:
@@ -164,50 +156,6 @@ class TestAudio:
        assert unknown == 1.0


-# ---------------------------------------------------------------------------
-# Diffusion observe API
-# ---------------------------------------------------------------------------
-
-
-class TestDiffusion:
-    def test_diffusion_exec_observed(self, mod: OmniModalityMetrics) -> None:
-        stage, replica = "dit_exec", "0"
-        mod.observe_diffusion_exec(stage, replica, 1.25)
-        out = generate_latest(REGISTRY).decode()
-        prefix = f'{defs.DIFFUSION_EXEC_S}_sum{{model_name="{_MODEL}",replica="{replica}",stage="{stage}"}}'
-        assert _sample_value(out, prefix) == 1.25
-
-    def test_diffusion_preprocess_observed(self, mod: OmniModalityMetrics) -> None:
-        stage, replica = "dit_pre", "0"
-        mod.observe_diffusion_preprocess(stage, replica, 0.03)
-        out = generate_latest(REGISTRY).decode()
-        prefix = f'{defs.DIFFUSION_PREPROCESS_S}_sum{{model_name="{_MODEL}",replica="{replica}",stage="{stage}"}}'
-        assert _sample_value(out, prefix) == pytest.approx(0.03)
-
-    def test_diffusion_postprocess_observed(self, mod: OmniModalityMetrics) -> None:
-        stage, replica = "dit_post", "0"
-        mod.observe_diffusion_postprocess(stage, replica, 0.8)
-        out = generate_latest(REGISTRY).decode()
-        prefix = f'{defs.DIFFUSION_POSTPROCESS_S}_sum{{model_name="{_MODEL}",replica="{replica}",stage="{stage}"}}'
-        assert _sample_value(out, prefix) == 0.8
-
-    def test_diffusion_exec_per_step_observed(self, mod: OmniModalityMetrics) -> None:
-        stage, replica = "dit_per_step", "0"
-        mod.observe_diffusion_exec_per_step(stage, replica, 0.078)
-        out = generate_latest(REGISTRY).decode()
-        prefix = f'{defs.DIFFUSION_EXEC_PER_STEP_S}_sum{{model_name="{_MODEL}",replica="{replica}",stage="{stage}"}}'
-        assert _sample_value(out, prefix) == pytest.approx(0.078)
-
-    def test_log_stats_false_skips(self) -> None:
-        silent = OmniModalityMetrics(model_name="silent-model", log_stats=False)
-        silent.observe_diffusion_exec("s", "r", 1.0)
-        silent.observe_diffusion_exec_per_step("s", "r", 0.02)
-        silent.observe_diffusion_preprocess("s", "r", 0.01)
-        silent.observe_diffusion_postprocess("s", "r", 0.5)
-        out = generate_latest(REGISTRY).decode()
-        assert 'model_name="silent-model"' not in out
-
-
 # ---------------------------------------------------------------------------
 # Stub-driven routing tests for the finalize / streaming helpers.
 # ---------------------------------------------------------------------------
@@ -235,18 +183,6 @@ class _StubModMetrics:
    def inc_audio_skipped(self, s, r, reason):
        self.calls.append(("inc_audio_skipped", s, r, reason))

-    def observe_diffusion_exec(self, s, r, v):
-        self.calls.append(("observe_diffusion_exec", s, r, v))
-
-    def observe_diffusion_exec_per_step(self, s, r, v):
-        self.calls.append(("observe_diffusion_exec_per_step", s, r, v))
-
-    def observe_diffusion_preprocess(self, s, r, v):
-        self.calls.append(("observe_diffusion_preprocess", s, r, v))
-
-    def observe_diffusion_postprocess(self, s, r, v):
-        self.calls.append(("observe_diffusion_postprocess", s, r, v))
-

 class _Bag:
    def __init__(self, **kw):
@@ -303,88 +239,8 @@ class TestObserveModalityAtFinalize:
        )
        assert ("observe_audio_duration", "1", "0", 1.0) in stub.calls

-    def test_diffusion_path_full(self):
-        stub = _StubModMetrics()
-        stage_metrics = _Bag(
-            stage_gen_time_ms=1000.0,
-            diffusion_metrics={
-                "preprocess_time_s": 0.05,
-                "diffusion_engine_exec_time_s": 1.2,
-                "postprocess_time_s": 0.3,
-                "image_num": 1,
-                "resolution": 1024,
-                "num_inference_steps": 50,
-            },
-        )
-        observe_modality_at_finalize(
-            stub,
-            output_type="image",
-            stage_id=2,
-            replica_id=0,
-            stage_metrics=stage_metrics,
-            engine_outputs=_Bag(),
-        )
-        assert ("observe_diffusion_preprocess", "2", "0", 0.05) in stub.calls
-        assert ("observe_diffusion_exec", "2", "0", 1.2) in stub.calls
-        assert ("observe_diffusion_postprocess", "2", "0", 0.3) in stub.calls
-        assert ("observe_diffusion_exec_per_step", "2", "0", pytest.approx(0.024)) in stub.calls
-
-    def test_diffusion_per_step_skipped_without_num_inference_steps(self):
-        stub = _StubModMetrics()
-        stage_metrics = _Bag(
-            stage_gen_time_ms=1000.0,
-            diffusion_metrics={
-                "diffusion_engine_exec_time_s": 1.2,
-                "preprocess_time_s": 0.05,
-                "postprocess_time_s": 0.3,
-            },
-        )
-        observe_modality_at_finalize(
-            stub,
-            output_type="image",
-            stage_id=2,
-            replica_id=0,
-            stage_metrics=stage_metrics,
-            engine_outputs=_Bag(),
-        )
-        assert ("observe_diffusion_exec", "2", "0", 1.2) in stub.calls
-        assert not any(c[0] == "observe_diffusion_exec_per_step" for c in stub.calls)
-
-    def test_diffusion_metrics_none_skipped(self):
-        stub = _StubModMetrics()
-        observe_modality_at_finalize(
-            stub,
-            output_type="image",
-            stage_id=2,
-            replica_id=0,
-            stage_metrics=_Bag(stage_gen_time_ms=500.0, diffusion_metrics=None),
-            engine_outputs=_Bag(),
-        )
-        assert not any(c[0].startswith("observe_diffusion") for c in stub.calls)
-
-    def test_diffusion_zero_values_observed(self):
-        stub = _StubModMetrics()
-        stage_metrics = _Bag(
-            stage_gen_time_ms=500.0,
-            diffusion_metrics={
-                "preprocess_time_s": 0.0,
-                "diffusion_engine_exec_time_s": 0.0,
-                "postprocess_time_s": 0.0,
-            },
-        )
-        observe_modality_at_finalize(
-            stub,
-            output_type="image",
-            stage_id=2,
-            replica_id=0,
-            stage_metrics=stage_metrics,
-            engine_outputs=_Bag(),
-        )
-        assert ("observe_diffusion_preprocess", "2", "0", 0.0) in stub.calls
-        assert ("observe_diffusion_exec", "2", "0", 0.0) in stub.calls
-        assert ("observe_diffusion_postprocess", "2", "0", 0.0) in stub.calls
-
-    def test_non_audio_output_type_skips_audio_but_still_checks_diffusion(self):
+    def test_non_audio_output_type_skipped(self):
+        # Image / video / text output types are out of scope for this module.
        stub = _StubModMetrics()
        for output_type in ("text", "image", "video"):
            observe_modality_at_finalize(
@@ -395,7 +251,7 @@ class TestObserveModalityAtFinalize:
                stage_metrics=_Bag(stage_gen_time_ms=100.0),
                engine_outputs=_Bag(),
            )
-        assert not any(c[0].startswith("observe_audio") or c[0].startswith("inc_audio") for c in stub.calls)
+        assert stub.calls == []

    def test_replica_id_none_skipped(self):
        stub = _StubModMetrics()
@@ -138,7 +138,6 @@ def format_diffusion_outputs(
        ),
        "width": request.sampling_params.width,
        "height": request.sampling_params.height,
-        "num_inference_steps": request.sampling_params.num_inference_steps,
    }

    # Only the primary payload determines the response type. Some image models
@@ -568,7 +568,6 @@ class OmniBase(PDDisaggregationMixin):
            msg_id = id(result)
            consumed = self._consumed_metric_message_ids(req_id)
            if msg_id not in consumed:
-                metrics.accumulate_diffusion_metrics(stage_meta.stage_type, req_id, engine_outputs)
                metrics.on_stage_metrics(stage_id, req_id, _m, output_type)
                consumed.add(msg_id)

@@ -134,15 +134,6 @@ AUDIO_CONTINUITY_OK_METRIC = METRIC_PREFIX + AUDIO_CONTINUITY_OK
 AUDIO_SKIPPED_REQUESTS_METRIC = METRIC_PREFIX + AUDIO_SKIPPED_REQUESTS


-# ============================================================================
-# Diffusion family (per-stage + per-replica diffusion timing breakdowns)
-# ============================================================================
-DIFFUSION_EXEC_S = METRIC_PREFIX + "diffusion_exec_s"
-DIFFUSION_EXEC_PER_STEP_S = METRIC_PREFIX + "diffusion_exec_per_step_s"
-DIFFUSION_PREPROCESS_S = METRIC_PREFIX + "diffusion_preprocess_s"
-DIFFUSION_POSTPROCESS_S = METRIC_PREFIX + "diffusion_postprocess_s"
-
-
 # ============================================================================
 # Cross-stage Transfer family (per-physical-hop TX/RX/in-flight timings)
 # ============================================================================
@@ -78,35 +78,6 @@ _audio_skipped_family = Counter(
 )


-# ----------------------------------------------------------------------------
-# Diffusion family
-# ----------------------------------------------------------------------------
-_diffusion_exec_family = Histogram(
-    defs.DIFFUSION_EXEC_S,
-    "DiT forward pass execution time per request in seconds.",
-    labelnames=_stage_labels,
-    buckets=defs.SECONDS_BUCKETS,
-)
-_diffusion_exec_per_step_family = Histogram(
-    defs.DIFFUSION_EXEC_PER_STEP_S,
-    "DiT forward pass execution time per denoising step in seconds.",
-    labelnames=_stage_labels,
-    buckets=defs.SECONDS_FAST_BUCKETS,
-)
-_diffusion_preprocess_family = Histogram(
-    defs.DIFFUSION_PREPROCESS_S,
-    "Diffusion input preprocessing time per request in seconds.",
-    labelnames=_stage_labels,
-    buckets=defs.SECONDS_FAST_BUCKETS,
-)
-_diffusion_postprocess_family = Histogram(
-    defs.DIFFUSION_POSTPROCESS_S,
-    "Diffusion output postprocessing (VAE decode) time per request in seconds.",
-    labelnames=_stage_labels,
-    buckets=defs.SECONDS_FAST_BUCKETS,
-)
-
-
 class OmniModalityMetrics:
    """Per-modality observe API. Stage/replica are passed at observe time
    because a single OmniModalityMetrics instance per pipeline serves all
@@ -168,44 +139,6 @@ class OmniModalityMetrics:
            reason=reason or "unknown",
        ).inc()

-    # ---- Diffusion --------------------------------------------------------
-
-    def observe_diffusion_exec(self, stage: str, replica: str, seconds: float) -> None:
-        if not self._log_stats:
-            return
-        _diffusion_exec_family.labels(
-            model_name=self._model_name,
-            stage=stage,
-            replica=replica,
-        ).observe(seconds)
-
-    def observe_diffusion_exec_per_step(self, stage: str, replica: str, seconds: float) -> None:
-        if not self._log_stats:
-            return
-        _diffusion_exec_per_step_family.labels(
-            model_name=self._model_name,
-            stage=stage,
-            replica=replica,
-        ).observe(seconds)
-
-    def observe_diffusion_preprocess(self, stage: str, replica: str, seconds: float) -> None:
-        if not self._log_stats:
-            return
-        _diffusion_preprocess_family.labels(
-            model_name=self._model_name,
-            stage=stage,
-            replica=replica,
-        ).observe(seconds)
-
-    def observe_diffusion_postprocess(self, stage: str, replica: str, seconds: float) -> None:
-        if not self._log_stats:
-            return
-        _diffusion_postprocess_family.labels(
-            model_name=self._model_name,
-            stage=stage,
-            replica=replica,
-        ).observe(seconds)
-

 def observe_modality_at_finalize(
    mod_metrics: OmniModalityMetrics,
@@ -228,49 +161,43 @@ def observe_modality_at_finalize(
    """
    if replica_id is None or stage_metrics is None or output_type is None:
        return
+    if output_type != "audio":
+        return

    stage_label = str(stage_id)
    replica_label = str(replica_id)
    gen_time_s = float(getattr(stage_metrics, "stage_gen_time_ms", 0.0)) / 1000.0
    mm_out = extract_mm_output(engine_outputs)

-    if output_type == "audio":
-        sample_rate = defs.resolve_audio_sample_rate(mm_out)
-        n_frames = int(getattr(stage_metrics, "audio_generated_frames", 0) or 0)
-        if n_frames == 0:
-            n_frames = count_audio_frames(mm_out)
-        mod_metrics.inc_audio_frames(stage_label, replica_label, n_frames)
-        duration_s = n_frames / sample_rate if sample_rate > 0 else 0.0
-        if duration_s > 0:
-            mod_metrics.observe_audio_duration(stage_label, replica_label, duration_s)
-            mod_metrics.observe_audio_rtf(
-                stage_label,
-                replica_label,
-                defs.compute_audio_rtf(gen_time_s, duration_s),
-            )
-        else:
-            mod_metrics.inc_audio_skipped(stage_label, replica_label, "no_audio_data")
-
-    dm = getattr(stage_metrics, "diffusion_metrics", None)
-    if dm:
-        _key_map = {
-            "diffusion_engine_exec_time_s": mod_metrics.observe_diffusion_exec,
-            "preprocess_time_s": mod_metrics.observe_diffusion_preprocess,
-            "postprocess_time_s": mod_metrics.observe_diffusion_postprocess,
-        }
-        for key, observe_fn in _key_map.items():
-            val = dm.get(key)
-            if val is not None:
-                observe_fn(stage_label, replica_label, float(val))
-
-        exec_time = dm.get("diffusion_engine_exec_time_s")
-        num_steps = dm.get("num_inference_steps")
-        if exec_time is not None and num_steps and num_steps > 0:
-            mod_metrics.observe_diffusion_exec_per_step(
-                stage_label,
-                replica_label,
-                float(exec_time) / int(num_steps),
-            )
+    sample_rate = defs.resolve_audio_sample_rate(mm_out)
+    # `stage_metrics.audio_generated_frames` is the legacy per-chunk
+    # accumulator field on StageRequestStats. No production path currently
+    # fills it, so the fallback below is the live source — but we leave the
+    # field lookup in place in case the accumulator gets re-wired upstream.
+    n_frames = int(getattr(stage_metrics, "audio_generated_frames", 0) or 0)
+    if n_frames == 0:
+        n_frames = count_audio_frames(mm_out)
+    mod_metrics.inc_audio_frames(stage_label, replica_label, n_frames)
+    duration_s = n_frames / sample_rate if sample_rate > 0 else 0.0
+    if duration_s > 0:
+        mod_metrics.observe_audio_duration(stage_label, replica_label, duration_s)
+        mod_metrics.observe_audio_rtf(
+            stage_label,
+            replica_label,
+            defs.compute_audio_rtf(gen_time_s, duration_s),
+        )
+    else:
+        # Request completed (finish_reason ∈ {stop, length} — error paths
+        # don't reach finalize) but no audio samples were produced. Covers
+        # silent `return None` skips in the talker→code2wav stage
+        # processors and the `parsed.append((0,0))` malformed-length path
+        # in qwen3-tts code2wav. raise-paths surface via the upstream
+        # vllm:request_success_total{finished_reason="error"} channel and
+        # never reach this branch.
+        mod_metrics.inc_audio_skipped(stage_label, replica_label, "no_audio_data")
+    # audio_underrun / continuity are emitted from the streaming path in
+    # observe_audio_streaming_finalize; finalize is too late for the
+    # per-chunk timeline they need.


 def observe_audio_first_packet(
@@ -44,7 +44,7 @@ class StageRequestStats:
    final_output_type: str | None = None
    request_id: str | None = None
    postprocess_time_ms: float = 0.0
-    diffusion_metrics: dict[str, float] = None
+    diffusion_metrics: dict[str, int] = None
    audio_generated_frames: int = 0
    audio_sample_rate: int = 0
    audio_duration_s: float = 0.0
@@ -579,7 +579,7 @@ class OrchestratorAggregator:
        if final_output_type is not None:
            stats.final_output_type = final_output_type
        stats.diffusion_metrics = (
-            {k: float(v) for k, v in self.diffusion_metrics.pop(req_id, {}).items()}
+            {k: int(v) for k, v in self.diffusion_metrics.pop(req_id, {}).items()}
            if req_id in self.diffusion_metrics
            else None
        )
@@ -644,19 +644,8 @@ class OrchestratorAggregator:
        if isinstance(diffusion_metrics, list):
            diffusion_metrics = diffusion_metrics[0]
        if diffusion_metrics:
-            _MS_TO_S = {
-                "diffusion_engine_exec_time_ms": "diffusion_engine_exec_time_s",
-                "preprocess_time_ms": "preprocess_time_s",
-                "postprocess_time_ms": "postprocess_time_s",
-                "diffusion_engine_total_time_ms": "diffusion_engine_total_time_s",
-            }
            for key, value in diffusion_metrics.items():
-                if value is None:
-                    continue
-                if key in _MS_TO_S:
-                    self.diffusion_metrics[req_id][_MS_TO_S[key]] = float(value) / 1000.0
-                else:
-                    self.diffusion_metrics[req_id][key] = value
+                self.diffusion_metrics[req_id][key] += value

    def on_forward(
        self,
Author	SHA1	Message	Date
Hongsheng Liu	fab29e2873	[Docs] Redirect the former PD guide URL Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>	2026-08-13 12:48:43 +08:00
Hongsheng Liu	1ade14d147	[Docs] Document current PD design Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>	2026-08-13 12:44:42 +08:00
Hongsheng Liu	b6313918d6	Merge branch 'main' into codex-pd-navigation	2026-08-13 12:23:56 +08:00
Hongsheng Liu	f27e70dce7	[Docs] Move PD guide into Features Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>	2026-08-13 06:54:34 +08:00
Hongsheng Liu	c46abfef59	[Docs] Mark PD disaggregation as experimental Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>	2026-08-13 06:50:41 +08:00