docs: clarify quantization support matrix

Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>

docs/design/metrics.md

@@ -1,6 +1,6 @@
 # Prometheus Metrics Design
 
-This document describes how vLLM-Omni exposes Prometheus metrics for multi-stage pipelines, the constraints that shaped the design, and how the pipeline-level metrics coexist with upstream vLLM per-engine metrics. This iteration covers the text and audio surface; diffusion / image / video families land in a follow-up.
+This document describes how vLLM-Omni exposes Prometheus metrics for multi-stage pipelines, the constraints that shaped the design, and how the pipeline-level metrics coexist with upstream vLLM per-engine metrics.
 
 ## Objectives
 

docs/usage/faq.md

@@ -14,7 +14,10 @@ A: At first, you can check current [issues](https://github.com/vllm-project/vllm
 
 > Q: Does vLLM-Omni support AWQ or any other quantization?
 
-A: We plan to introduce GGUF FP8 prequantized models and online FP8 quantization in version 0.16.0. Support for other quantization types will follow in future releases. For details, please see our [Q1 quantization roadmap](https://github.com/vllm-project/vllm-omni/issues/1057).
+A: AWQ is not currently listed as a validated vLLM-Omni path. Quantization
+support depends on the model, pipeline stage, checkpoint format, and hardware
+backend. See the current [quantization support matrix](../user_guide/quantization/overview.md)
+for online methods, pre-quantized checkpoints, and model-level validation.
 
 > Q: Does vLLM-Omni support multimodal streaming input and output?
 

docs/usage/metrics.md

@@ -1,23 +1,23 @@
 # Production Metrics
 
-vLLM-Omni exposes Prometheus metrics via the `/metrics` endpoint on the OpenAI-compatible API server. This page covers the text and audio surface; diffusion / image / video metrics are tracked in a follow-up PR.
+vLLM-Omni exposes Prometheus metrics via the `/metrics` endpoint on the OpenAI-compatible API server.
 
 ```bash
 vllm-omni serve Qwen/Qwen3-Omni-30B-A3B-Instruct --port 8000 --log-stats
 curl http://localhost:8000/metrics
 ```
 
-**`--log-stats` is required to populate metric data.** Without the flag, the endpoint still returns `200 OK` but the upstream `vllm:*` wrap is not registered at all, and the 15 `vllm:omni_*` families are registered as placeholders with no data written to them. This keeps the runtime cost essentially zero for deployments that don't need monitoring. With the flag, all ~80 families populate.
+**`--log-stats` is required to populate metric data.** Without the flag, the endpoint still returns `200 OK` but the upstream `vllm:*` wrap is not registered at all, and the 15 `vllm_omni:*` families are registered as placeholders with no data written to them. This keeps the runtime cost essentially zero for deployments that don't need monitoring. With the flag, all ~80 families populate.
 
 ## Metric Namespaces
 
 | Prefix | Source | Present when |
 |--------|--------|--------------|
-| `vllm:omni_` | vLLM-Omni orchestrator / audio modality / cross-stage transfer | Pipeline-dependent |
+| `vllm_omni:` | vLLM-Omni orchestrator / audio modality / cross-stage transfer | Pipeline-dependent |
 | `vllm:` | Upstream vLLM engine, wrapped by `OmniPrometheusStatLogger` to expose `{stage, replica}` | Pipeline includes an LLM (AR) stage |
 | `http_` / `process_` | Uvicorn / Python runtime | Always |
 
-## Pipeline-Level Metrics (`vllm:omni_`)
+## Pipeline-Level Metrics (`vllm_omni:`)
 
 Defined in `vllm_omni/metrics/prometheus.py`. Track request lifecycle across the full multi-stage pipeline.
 
@@ -25,42 +25,66 @@ Defined in `vllm_omni/metrics/prometheus.py`. Track request lifecycle across the
 
 | Metric | Type | Labels | Description |
 |--------|------|--------|-------------|
-| `vllm:omni_num_requests_running` | Gauge | `model_name` | Pipeline-global in-flight requests (dispatched to engine, not yet finalized) |
+| `vllm_omni:num_requests_running` | Gauge | `model_name` | Pipeline-global in-flight requests (dispatched to engine, not yet finalized) |
-| `vllm:omni_num_requests_waiting` | Gauge | `model_name` | Requests waiting in the Orchestrator queue |
+| `vllm_omni:num_requests_waiting` | Gauge | `model_name` | Requests waiting in the Orchestrator queue |
-| `vllm:omni_requests_success_total` | Counter | `model_name`, `finished_reason` | Total requests by completion reason. `finished_reason` ∈ {`stop`, `length`, `abort`, ...} mirroring upstream `vllm:request_success_total`; aborts cover client disconnect / cancellation paths in addition to upstream `FinishReason.ABORT` |
+| `vllm_omni:requests_success_total` | Counter | `model_name`, `finished_reason` | Total requests by completion reason. `finished_reason` ∈ {`stop`, `length`, `abort`, ...} mirroring upstream `vllm:request_success_total`; aborts cover client disconnect / cancellation paths in addition to upstream `FinishReason.ABORT` |
 
 ### Latency
 
 | Metric | Type | Labels | Description |
 |--------|------|--------|-------------|
-| `vllm:omni_e2e_request_latency_s` | Histogram | `model_name` | Pipeline-global end-to-end request latency in seconds |
+| `vllm_omni:e2e_request_latency_s` | Histogram | `model_name` | Pipeline-global end-to-end request latency in seconds |
 
-## Audio Modality Metrics (`vllm:omni_`)
+## Audio Modality Metrics (`vllm_omni:`)
 
 Emitted at request finalize, except for `audio_ttfp_s` (streaming-hook at the first audio packet) and `audio_underrun_s` / `audio_continuity_ok_total` (streaming finalize, after the chunk stream is exhausted). All carry `{model_name, stage, replica}` plus the listed extra label.
 
 | Metric | Type | Extra label | Description |
 |--------|------|-------------|-------------|
-| `vllm:omni_audio_ttfp_s` | Histogram | — | Time from request arrival to first audio packet/frame |
+| `vllm_omni:audio_ttfp_s` | Histogram | — | Time from request arrival to first audio packet/frame |
-| `vllm:omni_audio_duration_s` | Histogram | — | Audio content duration (`audio_frames / sample_rate`) |
+| `vllm_omni:audio_duration_s` | Histogram | — | Audio content duration (`audio_frames / sample_rate`) |
-| `vllm:omni_audio_rtf` | Histogram | — | Real-time factor (`stage_gen_time_s / audio_duration_s`); streaming TTS SLO red line `< 1`; uses `RTF_BUCKETS` |
+| `vllm_omni:audio_rtf` | Histogram | — | Real-time factor (`stage_gen_time_s / audio_duration_s`); streaming TTS SLO red line `< 1`; uses `RTF_BUCKETS` |
-| `vllm:omni_audio_frames_total` | Counter | — | Cumulative audio frame count; throughput via `rate()` |
+| `vllm_omni:audio_frames_total` | Counter | — | Cumulative audio frame count; throughput via `rate()` |
-| `vllm:omni_audio_underrun_s` | Histogram | — | Per-request worst-case player deficit; `> 0` indicates listener heard silent gaps |
+| `vllm_omni:audio_underrun_s` | Histogram | — | Per-request worst-case player deficit; `> 0` indicates listener heard silent gaps |
-| `vllm:omni_audio_continuity_ok_total` | Counter | `threshold_ms` | Incremented when the request's worst underrun stayed below `threshold_ms` |
+| `vllm_omni:audio_continuity_ok_total` | Counter | `threshold_ms` | Incremented when the request's worst underrun stayed below `threshold_ms` |
-| `vllm:omni_audio_skipped_requests_total` | Counter | `reason` | Silent-loss counter — code2wav rejected malformed codec input and returned `200 OK` with empty audio |
+| `vllm_omni:audio_skipped_requests_total` | Counter | `reason` | Silent-loss counter — code2wav rejected malformed codec input and returned `200 OK` with empty audio |
 
 The continuity math comes from `vllm_omni/benchmarks/audio_continuity.py::compute_continuity_stats` so the server-side observation aligns with the bench-side definition.
 
-## Cross-Stage Transfer Metrics (`vllm:omni_`)
+## Diffusion Metrics (`vllm_omni:`)
+
+Per-request timing breakdowns for diffusion (image/video) stages. Emitted at request finalize when `stage_metrics.diffusion_metrics` is present. All carry `{model_name, stage, replica}`.
+
+| Metric | Type | Description |
+|--------|------|-------------|
+| `vllm_omni:diffusion_exec_s` | Histogram | DiT forward pass execution time per request in seconds |
+| `vllm_omni:diffusion_exec_per_step_s` | Histogram | DiT forward pass execution time per denoising step in seconds (`exec_s / num_inference_steps`) |
+| `vllm_omni:diffusion_preprocess_s` | Histogram | Diffusion input preprocessing time per request in seconds |
+| `vllm_omni:diffusion_postprocess_s` | Histogram | Diffusion output postprocessing (VAE decode) time per request in seconds |
+
+`diffusion_exec_s` uses `SECONDS_BUCKETS` (50 ms → 300 s); the other three use `SECONDS_FAST_BUCKETS` (1 ms → 60 s). `diffusion_exec_per_step_s` is skipped when `num_inference_steps` is not set.
+
+### Example PromQL
+
+```promql
+# P99 DiT exec time by stage
+histogram_quantile(0.99, rate(vllm_omni:diffusion_exec_s_bucket[5m]))
+
+# Ratio of preprocess to total exec wall time
+rate(vllm_omni:diffusion_preprocess_s_sum[5m])
+  / rate(vllm_omni:diffusion_exec_s_sum[5m])
+```
+
+## Cross-Stage Transfer Metrics (`vllm_omni:`)
 
 Per-physical-transfer histograms tracking the data hop between adjacent stages. Labels `{model_name, from_stage, from_replica, to_stage, to_replica}` let dashboards attribute latency to specific replica edges. `from_replica` / `to_replica` are resolved from the orchestrator's sticky-routing binding (`stage_pool.get_bound_replica_id(request_id)`), so no extra plumbing through `TransferEdgeStats` is needed.
 
 | Metric | Type | Description |
 |--------|------|-------------|
-| `vllm:omni_transfer_size_bytes` | Histogram | Per-transfer payload size in bytes |
+| `vllm_omni:transfer_size_bytes` | Histogram | Per-transfer payload size in bytes |
-| `vllm:omni_transfer_tx_s` | Histogram | Sender-side time (serialize + submit to connector) |
+| `vllm_omni:transfer_tx_s` | Histogram | Sender-side time (serialize + submit to connector) |
-| `vllm:omni_transfer_rx_s` | Histogram | Receiver-side time (recv + deserialize) |
+| `vllm_omni:transfer_rx_s` | Histogram | Receiver-side time (recv + deserialize) |
-| `vllm:omni_transfer_in_flight_s` | Histogram | Network in-flight time (TX done → RX recv start) |
+| `vllm_omni:transfer_in_flight_s` | Histogram | Network in-flight time (TX done → RX recv start) |
 
 ## vLLM Engine Metrics (`vllm:`)
 
@@ -83,9 +107,9 @@ For the full list of upstream metrics, see [the vLLM docs](https://github.com/vl
 
 | Metric group | Multi-stage LLM (Qwen3-Omni) |
 |---|---|
-| `vllm:omni_` request tracking + latency | With `--log-stats` |
+| `vllm_omni:` request tracking + latency | With `--log-stats` |
-| `vllm:omni_` audio modality | With `--log-stats`, if pipeline has a talker stage |
+| `vllm_omni:` audio modality | With `--log-stats`, if pipeline has a talker stage |
-| `vllm:omni_` transfer | With `--log-stats`, if pipeline has ≥ 2 stages |
+| `vllm_omni:` transfer | With `--log-stats`, if pipeline has ≥ 2 stages |
 | `vllm:` engine metrics (per `(stage, replica)`) | With `--log-stats` |
 | `vllm:` MFU metrics | With `--log-stats --enable-mfu-metrics` |
 
@@ -94,5 +118,5 @@ For the full list of upstream metrics, see [the vLLM docs](https://github.com/vl
 - All time-bearing metrics use the `_s` suffix (values in seconds). Buckets are `SECONDS_BUCKETS` for e2e / generation-style values and `SECONDS_FAST_BUCKETS` (1 ms → 60 s) for the fine-grained transfer and audio-underrun values.
 - Counters use the `_total` suffix (auto-appended by `prometheus_client`).
 - Sizes use the `_bytes` suffix.
-- All omni-specific families are prefixed `vllm:omni_`. The upstream `unregister_vllm_metrics()` function is monkey-patched (see `vllm_omni/patch.py`) to a scoped version that still strips upstream `vllm:*` collectors so multi-engine init within one process does not crash on duplicate registration, but preserves anything prefixed `vllm:omni_` / `vllm_omni`.
+- All omni-specific families are prefixed `vllm_omni:`. The upstream `unregister_vllm_metrics()` function is monkey-patched (see `vllm_omni/patch.py`) to a scoped version that still strips upstream `vllm:*` collectors so multi-engine init within one process does not crash on duplicate registration, but preserves anything prefixed `vllm_omni:`.
-- Text and audio first-output use distinct families (`vllm:time_to_first_token_seconds` reused from upstream for text; `vllm:omni_audio_ttfp_s` for audio) rather than a single metric with a `modality` label.
+- Text and audio first-output use distinct families (`vllm:time_to_first_token_seconds` reused from upstream for text; `vllm_omni:audio_ttfp_s` for audio) rather than a single metric with a `modality` label.

docs/user_guide/diffusion_features.md

@@ -92,7 +92,7 @@ current support.
 |--------|--------------|-------------|----------|
 | **[FP8](quantization/fp8.md)** | `quantization="fp8"` | FP8 W8A8 on validated transformer stages | Memory reduction, inference speedup |
 | **[INT8](quantization/int8.md)** | `quantization="int8"` | INT8 W8A8 on validated transformer stages | Memory reduction, broad GPU compatibility |
-| **[GGUF](quantization/gguf.md)** | `quantization="gguf"` | Native GGUF transformer-only weights (Q4, Q8, etc.) | Memory reduction on consumer GPUs |
+| **[BitsAndBytes W4](quantization/bitsandbytes.md)** | `quantization="bitsandbytes"` | Online NF4/FP4 weight-only quantization on validated CUDA transformer stages | Reducing transformer weight memory without preparing a separate checkpoint |
 
 ## Supported Models
 

docs/user_guide/examples/offline_inference/text_to_video.md

@@ -143,7 +143,8 @@ python text_to_video.py \
 - `--enable-layerwise-offload`: enable layerwise offloading on DiT modules.
 - `--frame-rate`: generation FPS for pipelines that require it (e.g., LTX2).
 - `--audio-sample-rate`: fallback audio sample rate when the pipeline returns audio.
-- `--quantization`: quantization method (`fp8` for FP8, `gguf` for GGUF).
+- `--quantization`: online quantization method, for example `fp8`; supported
+  pre-quantized checkpoints are normally auto-detected from checkpoint metadata.
 - `--flow-shift`: scheduler flow_shift parameter.
 - `--lora-path`: path to PEFT LoRA adapter folder or checkpoint file.
 - `--lora-scale`: scale factor for LoRA weights.

docs/user_guide/examples/online_serving/diffusers_pipeline_adapter.md

@@ -90,8 +90,8 @@ vllm serve "Qwen/Qwen-Image" \
     --quantization-config '{"method": "fp8"}'
 ```
 
-Other vLLM-Omni quantization methods, such as `gguf`, `modelopt`, `mxfp4`,
+Other vLLM-Omni quantization methods, such as `modelopt`, `mxfp4`, `mxfp8`,
-`mxfp8`, serialized checkpoints, static FP8 configs, and layer-name skip lists
+serialized checkpoints, static FP8 configs, and layer-name skip lists
 such as `ignored_layers`, are intentionally not translated by this compatibility
 shortcut. Use Diffusers-native configuration through `diffusers_load_kwargs` or
 a native vLLM-Omni pipeline for those cases.

docs/user_guide/quantization/autoround.md

@@ -10,6 +10,10 @@ checkpoint's `config.json` and auto-detects
 AutoRound is static quantization: no `--quantization` flag is needed at
 inference time when the checkpoint already contains the quantization config.
 
+The validation terms on this page follow the
+[support levels](overview.md#support-levels) defined in the overview. A listed
+checkpoint is not enough by itself to establish end-to-end model support.
+
 ## Hardware Support
 
 | Device | Support |
@@ -21,46 +25,57 @@ inference time when the checkpoint already contains the quantization config.
 | Intel XPU | ✅ |
 | Ascend NPU | ❌ |
 
-Legend: `✅` supported, `❌` unsupported, `⭕` not verified in this
+Legend: `✅` backend available, `❌` unsupported, `⭕` not verified in this
-guide. AutoRound is Intel-supported.
+guide. This table is method-level backend availability. AutoRound is an Intel
+project; vLLM selects a compatible CUDA or XPU compute backend at load time.
 
 ## Model Type Support
 
 ### Diffusion Model (Qwen-Image, Wan2.2)
 
-| Model | Checkpoint | Scope | Scheme | Backend |
+| Model | Checkpoint | Scope | Scheme | Validation |
-|-------|------------|-------|--------|---------|
+|-------|------------|-------|--------|------------|
-| FLUX.1-dev | `vllm-project-org/FLUX.1-dev-AutoRound-w4a16` | Diffusion transformer | W4A16 | GPTQ-Marlin or Intel-supported AutoRound backend |
+| FLUX.1-dev | `vllm-project-org/FLUX.1-dev-AutoRound-w4a16` | Diffusion transformer | W4A16 | Validated |
-| Qwen-Image | Not listed | Diffusion transformer | W4A16 | Not validated |
+| Qwen-Image | `INC4AI/Qwen-Image-AutoRound-W4A16` | Diffusion transformer | W4A16 | Validated with end-to-end generation and quality comparison |
-| Wan2.2-I2V | `Intel/Wan2.2-I2V-A14B-Diffusers-int4-AutoRound` | Diffusion transformer | W4A16 | GPTQ-Marlin or Intel-supported AutoRound backend |
+| Wan2.2-I2V | `Intel/Wan2.2-I2V-A14B-Diffusers-int4-AutoRound` | Both diffusion transformers | W4A16 | CI-backed |
-| Wan2.2-T2V | `Intel/Wan2.2-T2V-A14B-Diffusers-int4-AutoRound` | Diffusion transformer | W4A16 | GPTQ-Marlin or Intel-supported AutoRound backend |
+| Wan2.2-T2V | `Intel/Wan2.2-T2V-A14B-Diffusers-int4-AutoRound` | Both diffusion transformers | W4A16 | CI-backed |
-| Wan2.2-TI2V | `Intel/Wan2.2-TI2V-5B-Diffusers-int4-AutoRound` | Diffusion transformer | W4A16 | GPTQ-Marlin or Intel-supported AutoRound backend |
+| Wan2.2-TI2V | `Intel/Wan2.2-TI2V-5B-Diffusers-int4-AutoRound` | Diffusion transformer | W4A16 | Validated; not in the scheduled Wan AutoRound job |
+
+CUDA execution uses GPTQ-Marlin or another compatible vLLM AutoRound backend;
+Intel XPU uses the Intel-supported backend. The text encoder and VAE stay in
+BF16 for all entries above.
 
 ### World Model (Cosmos3)
 
-| Model | Checkpoint | Scope | Scheme | Backend |
+| Model | Checkpoint | Scope | Scheme | Validation |
-|-------|------------|-------|--------|---------|
+|-------|------------|-------|--------|------------|
-| Cosmos3-Nano | `Intel/Cosmos3-Nano-int4-AutoRound` | World-model transformer | W4A16 | GPTQ-Marlin or Intel-supported AutoRound backend |
+| Cosmos3-Nano | `Intel/Cosmos3-Nano-int4-AutoRound` | World-model transformer | W4A16 | Integrated; no model-specific end-to-end test is maintained in-tree |
-| Cosmos3-Super | `Intel/Cosmos3-Super-int4-AutoRound` | World-model transformer | W4A16 | GPTQ-Marlin or Intel-supported AutoRound backend |
+| Cosmos3-Super | `Intel/Cosmos3-Super-int4-AutoRound` | World-model transformer | W4A16 | Validated with a manual serving run |
+
+The Cosmos VAE and guardrail components stay in BF16.
 
 ### Multi-Stage Omni/TTS Model (Qwen3-Omni, Qwen3-TTS)
 
-| Model | Checkpoint | Scope | Scheme | Backend |
+| Model | Checkpoint | Scope | Scheme | Validation |
-|-------|------------|-------|--------|---------|
+|-------|------------|-------|--------|------------|
-| Qwen2.5-Omni-7B | `Intel/Qwen2.5-Omni-7B-int4-AutoRound` | Language-model stage | W4A16 | AutoRound |
+| Qwen2.5-Omni-7B | `Intel/Qwen2.5-Omni-7B-int4-AutoRound` | Thinker language-model stage | W4A16 | Validated outside scheduled CI |
-| Qwen3-Omni-30B-A3B-Instruct | `Intel/Qwen3-Omni-30B-A3B-Instruct-int4-AutoRound` | Thinker language-model stage | W4A16 | AutoRound |
+| Qwen3-Omni-30B-A3B-Instruct | `Intel/Qwen3-Omni-30B-A3B-Instruct-int4-AutoRound` | Thinker language-model stage | W4A16 | Validated outside scheduled CI |
 | Qwen3-TTS | Not listed | TTS language-model stage | W4A16 | Not validated |
 
 AutoRound support is checkpoint-driven. A model is supported when its
 checkpoint uses a compatible INC/AutoRound config and the target stage maps to
 vLLM-Omni's runtime module names.
 
+For Qwen-Omni, only the thinker language model uses the AutoRound checkpoint.
+Audio and vision encoders, the talker, and waveform-decoder stages stay in
+BF16.
+
 ### Multi-Stage Diffusion Model (BAGEL, GLM-Image)
 
-| Model | Scope | Status | Notes |
+| Model | Checkpoint | Scope | Validation |
-|-------|-------|--------|-------|
+|-------|------------|-------|------------|
-| GLM-Image | Diffusion transformer  | ✅ | `Intel/GLM-Image-int4-AutoRound` |
+| GLM-Image | `Intel/GLM-Image-int4-AutoRound` | Diffusion transformer | CI-backed |
-| BAGEL | Checkpoint-defined diffusion or transformer stage | Not validated | Requires a compatible AutoRound checkpoint |
+| BAGEL | Not listed | Checkpoint-defined diffusion or transformer stage | Not validated |
 
 ## Configuration
 
@@ -119,6 +134,14 @@ The checkpoint should contain a config like:
 At load time, vLLM-Omni builds an `OmniINCConfig`, remaps checkpoint block names
 to runtime module names, and selects the matching vLLM compute backend.
 
+Checkpoint auto-detection and successful layer construction establish the
+**Integrated** level. Promote a model to **Validated** only after the named
+checkpoint completes an end-to-end generation or multimodal request and its
+output passes a model-appropriate sanity or quality check. Compare against the
+BF16 baseline for quality-sensitive image and video paths. Promote it to
+**CI-backed** only when that named checkpoint is selected by scheduled hardware
+CI.
+
 Example checkpoint creation:
 
 ```bash

docs/user_guide/quantization/bitsandbytes.md

@@ -9,6 +9,10 @@ stay in BF16/FP16. No pre-quantized checkpoint is required.
 This is an online (dynamic) path: load the normal HuggingFace checkpoint and
 quantize during model loading via the `bitsandbytes` CUDA kernels.
 
+The validation terms on this page follow the
+[support levels](overview.md#support-levels). BitsAndBytes kernel support does
+not imply that every diffusion transformer has been quality-validated.
+
 ## Hardware Support
 
 | Device | Support |
@@ -21,7 +25,7 @@ quantize during model loading via the `bitsandbytes` CUDA kernels.
 | Intel XPU | ❌ |
 | Ascend NPU | ❌ |
 
-Legend: `✅` supported, `❌` unsupported. Non-CUDA platforms raise in
+Legend: `✅` backend available, `❌` unsupported. Non-CUDA platforms raise in
 `get_quant_method()`; CUDA requires compute capability 7.5+ (`SM 75+`).
 
 Requires the optional `bitsandbytes` package (`pip install bitsandbytes`).
@@ -30,11 +34,11 @@ Requires the optional `bitsandbytes` package (`pip install bitsandbytes`).
 
 ### Diffusion Model (Qwen-Image, Wan2.2)
 
-| Model | HF models | CUDA | Mode | Recommendation |
+| Model | HF models | Scope | Level | Recommendation |
-|-------|-----------|:----:|------|----------------|
+|-------|-----------|-------|-------|----------------|
-| Z-Image | `Tongyi-MAI/Z-Image-Turbo` | Yes | Online W4 weight-only | All heavy linear layers; sensitive embedders stay BF16 |
+| Z-Image | `Tongyi-MAI/Z-Image-Turbo` | Diffusion transformer | Validated | All heavy linear layers; sensitive embedders stay BF16 |
-| Qwen-Image | `Qwen/Qwen-Image`, `Qwen/Qwen-Image-2512` | Not validated | Online W4 weight-only | Compare vs BF16 before enabling |
+| Qwen-Image | `Qwen/Qwen-Image`, `Qwen/Qwen-Image-2512` | Diffusion transformer | Not validated | Compare with BF16 before enabling |
-| Wan2.2 | Wan2.2 diffusion pipelines | Not validated | Online W4 weight-only | Validate before enabling in docs |
+| Wan2.2 | Wan2.2 diffusion pipelines | One or both diffusion transformers | Not validated | Validate quality and cascade behavior before enabling |
 
 Other diffusion models may work if their transformer uses supported linear
 layers, but they are not validated in this guide.
@@ -104,6 +108,10 @@ Multi-GPU tensor parallelism (`tensor_parallel_size` > 1) is not validated for
 BitsAndBytes in diffusion models; each rank quantizes its own weight shard
 independently.
 
+Z-Image-Turbo is validated outside scheduled full-model BitsAndBytes CI. The
+real CUDA kernel smoke tests cover quantization and matrix multiplication, but
+do not establish model-level quality for unlisted architectures.
+
 If quality regresses, use `ignored_layers` to keep sensitive projections in BF16
 (for example `to_out` or `w2`).
 

docs/user_guide/quantization/modelopt.md

@@ -13,6 +13,11 @@ components, such as the tokenizer, scheduler, text encoder, vision/audio
 encoder, and VAE, on the base checkpoint unless a model-specific guide says
 otherwise.
 
+The validation terms on this page follow the
+[support levels](overview.md#support-levels) defined in the overview. ModelOpt
+format detection and checkpoint adaptation are shared runtime integrations;
+each model and stage still requires its own end-to-end validation.
+
 !!! note
     ModelOpt checkpoints are pre-quantized checkpoints. Do not pass
     `--quantization fp8` for these checkpoints. The checkpoint
@@ -58,11 +63,11 @@ stay unquantized unless the model-specific recipe validates otherwise.
 | Intel XPU | ⭕ |
 | Ascend NPU | ❌ |
 
-Legend: `✅` supported, `❌` unsupported, `⭕` not verified in this guide.
+Legend: `✅` backend available, `❌` unsupported, `⭕` not verified in this
-The optional CUTLASS FP8 runtime override requires CUDA SM89+. ModelOpt NVFP4
+guide. The optional CUTLASS FP8 runtime override requires CUDA SM89+. ModelOpt
-and mixed FP8/NVFP4 diffusion checkpoints are currently validated on Blackwell
+NVFP4 and mixed FP8/NVFP4 diffusion checkpoints are currently validated on
-CUDA systems in the recipes below; other CUDA generations require separate
+Blackwell CUDA systems in the recipes below; other CUDA generations require
-backend and quality validation.
+separate backend and quality validation.
 
 ## Model Type Support
 
@@ -79,6 +84,9 @@ backend and quality validation.
 | HunyuanImage-3.0 | `feizhai123/hunyuan-image3-modelopt-mixed-experts-nvfp4-dense-fp8` | MoE diffusion transformer | Validated for ModelOpt mixed FP8/NVFP4 checkpoints |
 | Wan2.2 | Not available | Diffusion transformer | Not validated |
 
+The validated diffusion entries above have named checkpoint and recipe
+evidence. They are not all exercised by scheduled full-model CI.
+
 For full serving commands and benchmark context, see
 [`recipes/Qwen/Qwen-Image.md`](https://github.com/vllm-project/vllm-omni/blob/main/recipes/Qwen/Qwen-Image.md)
 and
@@ -88,8 +96,8 @@ and
 
 | Model | Scope | Status |
 |-------|-------|--------|
-| Qwen3-Omni | Thinker language-model stage | ModelOpt FP8 checkpoint path |
+| Qwen3-Omni | Thinker language-model stage (FP8) | Validated outside scheduled CI |
-| Qwen3-Omni | Thinker language-model stage (W4A4 NVFP4) | Validated; see [Qwen3-Omni NVFP4 W4A4](#qwen3-omni-nvfp4-w4a4-thinker) below |
+| Qwen3-Omni | Thinker language-model stage (W4A4 NVFP4) | CI-backed for the H100 Marlin fallback; native Blackwell path is recipe-validated |
 | Qwen3-TTS | TTS language-model stage | Not validated |
 
 Audio encoder, vision encoder, talker, and code2wav stages stay in BF16 unless
@@ -102,9 +110,9 @@ Qwen3-Omni-30B-A3B-Instruct thinker language model. The thinker text body
 (attention + MoE experts) is quantized to NVFP4 with FP8 per-tensor input
 scales; the audio encoder, vision encoder, talker, and code2wav stay in BF16.
 
-| Variant | HF checkpoint | Hardware |
+| Variant | HF checkpoint | Hardware and validation |
-|---------|---------------|----------|
+|---------|---------------|-------------------------|
-| W4A4 NVFP4 (full thinker) | `YihongJin/Qwen3-Omni-30B-A3B-Instruct-NVFP4-W4A4-full-thinker-awqclip` | sm_100+ (Blackwell, FlashInfer FP4 GEMM) |
+| W4A4 NVFP4 (full thinker) | `YihongJin/Qwen3-Omni-30B-A3B-Instruct-NVFP4-W4A4-full-thinker-awqclip` | sm_100+ native FlashInfer FP4 GEMM; H100 Marlin fallback is CI-backed |
 
 Calibration uses ModelOpt `mtq.NVFP4_DEFAULT_CFG` with the `awq_clip` algorithm
 on 1024 ultrachat samples chat-templated through the Qwen3-Omni tokenizer.
@@ -232,3 +240,6 @@ omni = Omni(
 5. Keep `--quantization fp8` for online FP8 from BF16 checkpoints; use this
    ModelOpt path only when the checkpoint already contains ModelOpt quantized
    weights and scales.
+6. The Qwen3-Omni full-model CI smoke exercises the H100 Marlin fallback. A
+   scheduled Blackwell test is still required before treating the native
+   FlashInfer FP4 kernel path as CI-backed.

docs/user_guide/quantization/msmodelslim.md

@@ -3,12 +3,17 @@
 ## Overview
 
 [msModelSlim](https://github.com/Ascend/msmodelslim) is an Ascend compression
-toolkit for producing pre-quantized model checkpoints. In vLLM-Omni, these
+toolkit for producing pre-quantized model checkpoints. vLLM-Omni consumes its
-checkpoints run through the Ascend/NPU path with `--quantization ascend`.
+output either through a compatible generic Ascend checkpoint path or through
+the native MXFP8 and MXFP4 merge workflows documented in this guide.
 
 msModelSlim is static quantization: quantized weights are generated offline
 before vLLM-Omni inference starts.
 
+The validation terms on this page follow the
+[support levels](overview.md#support-levels). A producer recipe or hardware
+backend alone is an integration point, not model-level validation.
+
 ## Hardware Support
 
 | Device | Support |
@@ -20,23 +25,28 @@ before vLLM-Omni inference starts.
 | Intel XPU | ❌ |
 | Ascend NPU | ✅ |
 
-Legend: `✅` supported, `❌` unsupported, `⭕` not verified in this
+Legend: `✅` backend available, `❌` unsupported, `⭕` not verified in this
 guide.
 
 ## Model Type Support
 
 ### Diffusion Model (Qwen-Image, Wan2.2)
 
-| Model | Base model | Scope | Hardware | Notes |
+| Model | Base model | Scope | Level | Notes |
-|-------|------------|-------|----------|-------|
+|-------|------------|-------|-------|-------|
-| Wan2.2 | Wan2.2 diffusion weights | DiT or diffusion stage | Ascend NPU | Upstream msModelSlim provides a Wan2.2 quantization recipe; vLLM-Omni inference validation is not listed |
+| Wan2.2 | Wan2.2 diffusion weights | DiT or diffusion stage | Integrated | Upstream msModelSlim provides a producer recipe; use the [MXFP8](mxfp8.md) or [MXFP4](mxfp4.md) workflow for vLLM-Omni's native merged formats |
-| Qwen-Image | `Qwen/Qwen-Image`, `Qwen/Qwen-Image-2512` | DiT or diffusion stage | Ascend NPU | Not validated in this guide |
+| Qwen-Image | `Qwen/Qwen-Image`, `Qwen/Qwen-Image-2512` | DiT or diffusion stage | Not validated | No supported checkpoint workflow is documented |
-| HunyuanImage-3.0 | `tencent/HunyuanImage-3.0`, `tencent/HunyuanImage-3.0-Instruct` | DiT or diffusion stage | Ascend A2/A3 NPU | Generate quantized weights with the HunyuanImage-3.0 msModelSlim adaptation |
+| HunyuanImage-3.0 | `tencent/HunyuanImage-3.0`, `tencent/HunyuanImage-3.0-Instruct` | DiT or diffusion stage | Integrated | Generate quantized weights with the HunyuanImage-3.0 msModelSlim adaptation and validate against BF16 |
 
 Public Hugging Face quantized weights are not available yet. Use the
 [HunyuanImage-3.0 msModelSlim adaptation](https://gitcode.com/betta18/msmodelslim/tree/hyimage3_mxfp8)
 to generate the checkpoint manually.
 
+msModelSlim is also the producer used by the native offline MXFP8 and MXFP4
+DualScale workflows. Those workflows preprocess and annotate each transformer
+checkpoint and are auto-detected at inference; follow their method pages rather
+than passing `--quantization ascend`.
+
 ### Multi-Stage Omni/TTS Model (Qwen3-Omni, Qwen3-TTS)
 
 | Model | Scope | Status | Notes |
@@ -53,25 +63,32 @@ to generate the checkpoint manually.
 
 ## Configuration
 
+For a compatible generic Ascend checkpoint, select the Ascend quantization
+method explicitly:
+
 Offline inference:
 
 ```bash
-python text_to_image.py --model <quantized-model-path> --quantization ascend
+python text_to_image.py --model <compatible-ascend-checkpoint> --quantization ascend
 ```
 
 Online serving:
 
 ```bash
-vllm serve <quantized-model-path> --omni --quantization ascend
+vllm serve <compatible-ascend-checkpoint> --omni --quantization ascend
 ```
 
 ## Parameters
 
 | Parameter | Type | Default | Description |
 |-----------|------|---------|-------------|
-| `--quantization` | str | - | Use `ascend` for msModelSlim-produced checkpoints |
+| `--quantization` | str | - | Use `ascend` only for a compatible generic Ascend-format checkpoint; omit it for native merged MXFP8/MXFP4 checkpoints |
 | `model` | str | - | Path to the quantized checkpoint generated by Ascend tooling |
 
+Do not pass `--quantization ascend` for a Wan checkpoint processed by the native
+MXFP8 or MXFP4 merge tools. Their transformer metadata selects `mxfp8` or
+`mxfp4_dualscale` automatically.
+
 Example msModelSlim command for a Wan2.2 W8A8 checkpoint:
 
 ```bash
@@ -93,3 +110,6 @@ For HunyuanImage-3.0, use the Hunyuan-specific adaptation linked above.
    tooling; it is not a load-time CUDA quantizer.
 3. Keep the quantized checkpoint aligned with the same model architecture and
    stage config used for BF16 inference.
+4. No named full-model msModelSlim checkpoint is currently scheduled in NPU
+   CI. Treat new producer outputs as **Integrated** until an end-to-end BF16
+   comparison establishes **Validated** support.

docs/user_guide/quantization/mxfp4.md

@@ -13,6 +13,10 @@ vLLM-Omni provides two quantization methods with different scale structures:
 | `mxfp4` | Single-scale (per-32 fine only) | Online only | Quick accuracy baseline; no checkpoint prep needed |
 | `mxfp4_dualscale` | Dual-scale (fine per-32 + coarse per-512 + per-channel `mul_scale`) | Online + Offline | Production; better accuracy; offline recommended |
 
+The validation terms on this page follow the
+[support levels](overview.md#support-levels). Current model guidance covers
+Wan diffusion transformers only; the text encoder and VAE stay in BF16.
+
 !!! tip "Recommended: `mxfp4_dualscale` offline"
     For production deployments, use the `mxfp4_dualscale` offline mode with a
     pre-quantized checkpoint produced by msModelSlim. Offline checkpoints load
@@ -34,26 +38,45 @@ vLLM-Omni provides two quantization methods with different scale structures:
 
 ## Hardware Support
 
-| Device | Support |
+| Device | Online `mxfp4` | Online `mxfp4_dualscale` | Offline `mxfp4_dualscale` |
-|--------|---------|
+|--------|-----------------|---------------------------|----------------------------|
-| NVIDIA Blackwell GPU (SM 100+) | ⭕ |
+| NVIDIA Blackwell GPU (SM 100+) | ❌ | ❌ | ❌ |
-| NVIDIA Ada/Hopper GPU (SM 89+) | ⭕ |
+| NVIDIA Ada/Hopper GPU (SM 89+) | ❌ | ❌ | ❌ |
-| NVIDIA Ampere GPU (SM 80+) | ⭕ |
+| NVIDIA Ampere GPU (SM 80+) | ❌ | ❌ | ❌ |
-| AMD ROCm (gfx950 / MI355X) | ✅ |
+| AMD ROCm (gfx950 / MI355X) | ✅ | ❌ | ❌ |
-| Intel XPU | ⭕ |
+| Intel XPU | ❌ | ❌ | ❌ |
-| Ascend NPU (Atlas 950 A5) | ✅ |
+| Ascend NPU (Atlas 950 A5) | ✅ | ✅ | ✅ |
 
-Legend: `✅` supported, `❌` unsupported, `⭕` not verified in this guide.
+Legend: `✅` backend available, `❌` unsupported, `⭕` not verified in this
+guide. Backend availability does not imply full-model checkpoint validation.
+
+!!! note "Why NVIDIA is marked unsupported"
+    This table describes the methods currently wired into vLLM-Omni, not every
+    low-precision format NVIDIA hardware could execute. `DiffusionMXFP4Config`
+    has runtime implementations only for Ascend and ROCm gfx950; on CUDA it
+    raises `NotImplementedError` because no CUDA MXFP4 linear method or
+    checkpoint loader is registered.
+
+    On NVIDIA, use [ModelOpt](modelopt.md) NVFP4 for the documented 4-bit
+    checkpoint path. NVFP4 and OCP MXFP4 have different checkpoint metadata,
+    scale layouts, and kernel contracts, so they are not interchangeable. A
+    native CUDA MXFP4 path would require a CUDA linear implementation, loader
+    contract, and model-level output validation.
+
+The ROCm path dispatches online single-scale `mxfp4` to AITER on gfx950. It has
+kernel and dispatch coverage, but this guide does not yet claim end-to-end
+quality validation for a named Wan checkpoint on ROCm. DualScale and
+pre-quantized MXFP4 checkpoints are not supported on ROCm.
 
 ## Model Type Support
 
 ### Diffusion Model (Wan2.2)
 
-| Model | Online | Offline | Notes |
+| Model | Online (Ascend) | Offline (Ascend) | Validation and notes |
-|-------|--------|---------|-------|
+|-------|-----------------|------------------|----------------------|
-| Wan2.2-T2V-A14B | `mxfp4` / `mxfp4_dualscale` | `mxfp4_dualscale` | MoE cascade (`transformer` + `transformer_2`); both transformers quantized with the same config |
+| Wan2.2-T2V-A14B | `mxfp4` / `mxfp4_dualscale` | `mxfp4_dualscale` | Validated outside scheduled full-model offline NPU CI; both cascade transformers are quantized |
-| Wan2.2-I2V-A14B | `mxfp4` / `mxfp4_dualscale` | `mxfp4_dualscale` | MoE cascade; same scheme as T2V-A14B |
+| Wan2.2-I2V-A14B | `mxfp4` / `mxfp4_dualscale` | `mxfp4_dualscale` | Validated outside scheduled full-model offline NPU CI; same cascade scheme as T2V-A14B |
-| Wan2.2-TI2V-5B | ❌ | ❌ | Parameter count too small; W4A4 causes unacceptable accuracy loss |
+| Wan2.2-TI2V-5B | ❌ | ❌ | Unsupported because W4A4 causes unacceptable accuracy loss |
 
 The choice between `mxfp4` and `mxfp4_dualscale` in **online mode** is about
 quantization quality, not model compatibility — both work on cascade (A14B) and
@@ -83,12 +106,13 @@ by the merge script); there is no offline `mxfp4` single-scale format.
 
 ### `mxfp4` — Single-Scale Online Mode
 
-Online mode requires no pre-processing. vLLM-Omni quantizes BF16 weights to
+Online mode requires no pre-processing. On Ascend, vLLM-Omni quantizes BF16
-MXFP4 at load time using `npu_dynamic_mx_quant`. A single block scale
+weights at load time using `npu_dynamic_mx_quant`. On ROCm gfx950, the same
+`mxfp4` config dispatches to the AITER online kernel. A single block scale
 (`float8_e8m0fnu`, one per 32 K elements) is computed on the fly; no
-calibration `mul_scale` is available. Applies equally to single-transformer
+calibration `mul_scale` is available. The config applies equally to
-and cascade (A14B) models — both transformers in a cascade receive the same
+single-transformer and cascade (A14B) models, but named Wan model validation is
-quantization config automatically.
+currently limited to the documented Ascend path.
 
 ```python
 from vllm_omni import Omni
@@ -111,10 +135,10 @@ vllm serve <your-model> --omni --quantization mxfp4
 
 ### `mxfp4_dualscale` — DualScale Online Mode
 
-Online DualScale mode computes both fine and coarse scales on the fly from BF16
+Ascend-only online DualScale mode computes both fine and coarse scales on the
-weights using `npu_dynamic_dual_level_mx_quant`. Applies equally to
+fly from BF16 weights using `npu_dynamic_dual_level_mx_quant`. It applies
-single-transformer and cascade (A14B) models. Compared to `mxfp4` online,
+equally to single-transformer and cascade (A14B) models. Compared to `mxfp4`
-DualScale provides better quantization accuracy at higher compute cost.
+online, DualScale provides better quantization accuracy at higher compute cost.
 
 The default configuration keeps the leading 5 transformer blocks in BF16
 (`num_bf16_fallback_layers=5`). Accuracy evaluation on Wan2.2-A14B shows this
@@ -341,6 +365,11 @@ unless they appear in `ignored_layers`.
    quantization. Use `ignored_layers` and `num_bf16_fallback_layers` to trade
    off compression vs. accuracy for precision-sensitive layers.
 
+8. The native methods, merge mappings, and per-transformer cascade metadata
+   have unit coverage. The current Buildkite configuration does not schedule a
+   named full-model offline MXFP4 checkpoint on NPU, so this path is
+   **Validated**, not **CI-backed**.
+
 ## Adapting MXFP4 for a New Model
 
 This section is aimed at developers who want to add MXFP4 support to a model

docs/user_guide/quantization/mxfp8.md

@@ -15,30 +15,45 @@ This method supports three modes:
 | **Offline (Native)** | msModelSlim-exported MXFP8 weights converted to diffusers format via `merge_mxfp8_checkpoint.py` — weights and scales are loaded directly from the preprocessed checkpoint |
 | **Offline (AutoRound)** | AutoRound MXFP8 checkpoints with `data_type="mx_fp"` — auto-detected from `config.json` |
 
+The validation terms on this page follow the
+[support levels](overview.md#support-levels). Current model guidance covers
+Wan diffusion transformers only; the text encoder and VAE stay in BF16.
+
 ## Hardware Support
 
 | Device | Online | Offline (Native) | Offline (AutoRound) |
 |--------|--------|------------------|---------------------|
-| NVIDIA Blackwell GPU (SM 100+) | ⭕ | ⭕ | ⭕ |
+| NVIDIA Blackwell GPU (SM 100+) | ❌ | ❌ | ⭕ |
-| NVIDIA Ada/Hopper GPU (SM 89+) | ⭕ | ⭕ | ⭕ |
+| NVIDIA Ada/Hopper GPU (SM 89+) | ❌ | ❌ | ⭕ |
-| NVIDIA Ampere GPU (SM 80+) | ⭕ | ⭕ | ⭕ |
+| NVIDIA Ampere GPU (SM 80+) | ❌ | ❌ | ⭕ |
-| AMD ROCm | ⭕ | ⭕ | ⭕ |
+| AMD ROCm | ❌ | ❌ | ⭕ |
 | Intel XPU | ✅ | ❌ | ✅ |
 | Ascend NPU (Atlas 950 A5) | ✅ | ✅ | ⭕ |
 
-Legend: `✅` supported, `❌` unsupported, `⭕` not verified in this guide.
+Legend: `✅` backend available, `❌` unsupported, `⭕` not verified in this
+guide. Backend availability does not imply full-model checkpoint validation.
 
-**Note**: Intel XPU only supports AutoRound MXFP8 for offline mode. Use AutoRound quantized checkpoints or online mode for XPU.
+The native `DiffusionMXFP8Config` currently dispatches only to Intel XPU and
+Ascend NPU implementations. CUDA reaches `NotImplementedError`, so the NVIDIA
+rows are marked unsupported even though this is an implementation gap rather
+than a general claim about NVIDIA hardware capabilities. On NVIDIA, use
+[FP8](fp8.md) for load-time quantization or [ModelOpt](modelopt.md) FP8 for
+pre-quantized checkpoints. Adding native CUDA MXFP8 requires a registered CUDA
+linear method, a compatible checkpoint contract, and model-level validation.
+
+Native `mxfp8` is available online on Intel XPU and online or offline on
+Ascend. Intel XPU only supports AutoRound MXFP8 for offline mode. The AutoRound
+column is a separate checkpoint format and compute path.
 
 ## Model Type Support
 
 ### Diffusion Model (Wan2.2)
 
-| Model | Mode | Notes |
+| Model | Mode | Scope | Validation |
-|-------|------|-------|
+|-------|------|-------|------------|
-| Wan2.2-T2V-A14B | Online + Offline | MoE cascade; quantizes two transformers (`transformer` + `transformer_2`) |
+| Wan2.2-T2V-A14B | Online + Offline | MoE cascade: `transformer` + `transformer_2` | Validated outside scheduled full-model offline NPU CI |
-| Wan2.2-I2V-A14B | Online + Offline | MoE cascade; quantizes two transformers (`transformer` + `transformer_2`) |
+| Wan2.2-I2V-A14B | Online + Offline | MoE cascade: `transformer` + `transformer_2` | Validated outside scheduled full-model offline NPU CI |
-| Wan2.2-TI2V-5B | Online + Offline | Single transformer |
+| Wan2.2-TI2V-5B | Online + Offline | Single diffusion transformer | Validated outside scheduled full-model offline NPU CI |
 
 ### Multi-Stage Omni/TTS Model (Qwen3-Omni, Qwen3-TTS)
 
@@ -218,3 +233,7 @@ omni = Omni(model="<autoround-mxfp8-model>")
    `--output-dir`), regenerate it with the current script. The old script
    wrote a separate `quantization_config.json` that is not read by vLLM-Omni;
    the current script injects the config directly into `transformer/config.json`.
+4. The native loader, merge mappings, and Wan cascade config propagation have
+   unit coverage. The current Buildkite configuration does not schedule a
+   named full-model offline MXFP8 checkpoint on NPU, so this path is
+   **Validated**, not **CI-backed**.

docs/user_guide/quantization/online.md

@@ -6,7 +6,7 @@ Online quantization means vLLM-Omni computes quantized weights and scales while
 loading the model. Use it when you want memory savings without preparing a
 separate quantized checkpoint.
 
-This mode is different from pre-quantized checkpoint formats such as GGUF,
+This mode is different from pre-quantized checkpoint formats such as ModelOpt,
 AutoRound, msModelSlim, or serialized Int8 checkpoints. Those formats are
 prepared before serving and are documented in their method-specific guides.
 For MXFP8 and MXFP4, use this page for load-time quantization from BF16
@@ -15,18 +15,22 @@ by msModelSlim and the merge tools.
 
 ## Hardware Support
 
-| Device | FP8 W8A8 | Int8 W8A8 | MXFP8 W8A8 | MXFP4 W4A4 |
+| Device | FP8 W8A8 | Int8 W8A8 | BitsAndBytes W4 | MXFP8 W8A8 | MXFP4 W4A4 |
-|--------|----------|-----------|------------|------------|
+|--------|----------|-----------|------------------|------------|------------|
-| NVIDIA Blackwell GPU (SM 100+) | ✅ | ✅ | ⭕ | ⭕ |
+| NVIDIA Blackwell GPU (SM 100+) | ✅ | ✅ | ✅ | ❌ | ❌ |
-| NVIDIA Ada/Hopper GPU (SM 89+) | ✅ | ✅ | ⭕ | ⭕ |
+| NVIDIA Ada/Hopper GPU (SM 89+) | ✅ | ✅ | ✅ | ❌ | ❌ |
-| NVIDIA Ampere GPU (SM 80+) | ✅ | ✅ | ⭕ | ⭕ |
+| NVIDIA Ampere GPU (SM 80+) | ✅ | ✅ | ✅ | ❌ | ❌ |
-| AMD ROCm | ⭕ | ⭕ | ⭕ | ⭕ |
+| AMD ROCm (gfx950) | ⭕ | ⭕ | ❌ | ❌ | ✅ |
-| Intel XPU | ⭕ | ⭕ | ✅ | ⭕ |
+| Intel XPU | ⭕ | ⭕ | ❌ | ✅ | ❌ |
-| Ascend NPU | ❌ | ✅ | ✅ | ✅ |
+| Ascend NPU | ❌ | ✅ | ❌ | ✅ | ✅ |
 
-Legend: `✅` supported, `❌` unsupported, `⭕` not verified in this
+Legend: `✅` backend available, `❌` unsupported, `⭕` not verified in this
-guide. FP8 on Ampere may use a weight-only path where available. MXFP8 and
+guide. These entries describe method-level backend availability, not validation
-MXFP4 are documented for the Ascend NPU path.
+for every model. FP8 on Ampere may use a weight-only path where available.
+On ROCm gfx950, MXFP4 means online single-scale `mxfp4`; online and offline
+`mxfp4_dualscale` are Ascend-only. The native MXFP8/MXFP4 configs do not
+currently register CUDA implementations; use FP8 or ModelOpt FP8/NVFP4 on
+NVIDIA instead.
 
 ## Model Type Support
 
@@ -36,8 +40,9 @@ MXFP4 are documented for the Ascend NPU path.
 |--------|-------|----------------|--------|
 | FP8 W8A8 | [FP8](fp8.md) | Qwen-Image; Wan2.2 is not validated | Validated for Qwen-Image family and other DiT models |
 | Int8 W8A8 | [Int8](int8.md) | Qwen-Image; Wan2.2 is not validated | Validated for Qwen-Image and Z-Image |
+| BitsAndBytes W4 | [BitsAndBytes](bitsandbytes.md) | Z-Image-Turbo; Qwen-Image and Wan2.2 are not validated | Validated for Z-Image-Turbo |
 | MXFP8 W8A8 | [MXFP8](mxfp8.md) | Wan2.2-T2V-A14B, Wan2.2-I2V-A14B, Wan2.2-TI2V-5B | Validated on Ascend NPU and Intel XPU |
-| MXFP4 W4A4 | [MXFP4](mxfp4.md) | Wan2.2-T2V-A14B, Wan2.2-I2V-A14B | Ascend NPU only; TI2V-5B is not supported |
+| MXFP4 W4A4 | [MXFP4](mxfp4.md) | Wan2.2-T2V-A14B, Wan2.2-I2V-A14B | Validated on Ascend NPU; the ROCm gfx950 backend is integrated but not model-validated; TI2V-5B is unsupported |
 
 ### Multi-Stage Omni/TTS Model (Qwen3-Omni, Qwen3-TTS)
 
@@ -59,6 +64,7 @@ from vllm_omni import Omni
 
 omni_fp8 = Omni(model="<your-model>", quantization="fp8")
 omni_int8 = Omni(model="<your-model>", quantization="int8")
+omni_bnb = Omni(model="Tongyi-MAI/Z-Image-Turbo", quantization="bitsandbytes")
 omni_mxfp8 = Omni(model="<your-model>", quantization="mxfp8")
 omni_mxfp4 = Omni(model="<your-model>", quantization="mxfp4")
 omni_mxfp4_dualscale = Omni(model="<your-model>", quantization="mxfp4_dualscale")
@@ -69,6 +75,7 @@ CLI:
 ```bash
 vllm serve <your-model> --omni --quantization fp8
 vllm serve <your-model> --omni --quantization int8
+vllm serve Tongyi-MAI/Z-Image-Turbo --omni --quantization bitsandbytes
 vllm serve <your-model> --omni --quantization mxfp8
 vllm serve <your-model> --omni --quantization mxfp4
 vllm serve <your-model> --omni --quantization mxfp4_dualscale
@@ -85,14 +92,19 @@ config = build_quant_config({
 })
 ```
 
+Component routing is an extension mechanism, not automatic support for every
+pipeline component. See [Quantization Scope](overview.md#quantization-scope)
+before applying it to an encoder, VAE, or decoder.
+
 ## Parameters
 
 | Parameter | Methods | Description |
 |-----------|---------|-------------|
-| `method` | FP8, Int8, MXFP8, MXFP4 | Quantization method: `"fp8"`, `"int8"`, `"mxfp8"`, `"mxfp4"`, or `"mxfp4_dualscale"` |
+| `method` | FP8, Int8, BitsAndBytes, MXFP8, MXFP4 | Quantization method: `"fp8"`, `"int8"`, `"bitsandbytes"`, `"mxfp8"`, `"mxfp4"`, or `"mxfp4_dualscale"` |
-| `ignored_layers` | FP8, Int8, MXFP8, MXFP4 | Layer name patterns to keep in BF16/FP16 |
+| `ignored_layers` | FP8, Int8, BitsAndBytes, MXFP8, MXFP4 | Layer name patterns to keep in BF16/FP16 |
 | `activation_scheme` | FP8, Int8 | The runtime value `"dynamic"` selects online activation scaling |
 | `weight_block_size` | FP8 | Optional block-wise FP8 weight quantization size |
+| `quant_type` | BitsAndBytes | Weight format: `"nf4"` (default) or `"fp4"` |
 | `num_bf16_fallback_layers` | MXFP4 DualScale | Leading transformer blocks to keep in BF16 for online `mxfp4_dualscale`; defaults to `5` |
 
 ## Validation and Notes

docs/user_guide/quantization/overview.md

@@ -16,72 +16,123 @@ For the internal architecture and backend extension points, see the
 | Runtime attention quantization | [Quantized KV Cache](quantized_kvcache.md) | vLLM-Omni dynamically quantizes eligible diffusion Flash Attention tensors during inference. | FP8 FA |
 | Pre-quantized checkpoints | Method-specific guides | The checkpoint or an offline quantizer provides quantized weights and scales before serving. | ModelOpt, AutoRound, msModelSlim, serialized Int8, offline MXFP8, offline MXFP4 DualScale |
 
-## Hardware Support
+Online quantization starts from a normal BF16/FP16 checkpoint and repeats the
+conversion on each fresh model load. It is the simplest path for experiments.
+Pre-quantized checkpoints store the packed weights and scales produced by a
+separate tool; they are easier to reuse across deployments and can carry
+calibrated or mixed-precision policies that load-time conversion cannot
+reproduce.
 
-| Device | FP8 W8A8 | Int8 W8A8 | BitsAndBytes W4 | ModelOpt | MXFP8 W8A8 | MXFP4 W4A4 | AutoRound | msModelSlim |
+## Support Levels
-|--------|----------|-----------|------------------|----------|------------|------------|-----------|-------------|
-| NVIDIA Blackwell GPU (SM 100+) | ✅ | ✅ | ✅ | ✅ | ⭕ | ⭕ | ✅ | ❌ |
-| NVIDIA Ada/Hopper GPU (SM 89+) | ✅ | ✅ | ✅ | ✅ | ⭕ | ⭕ | ✅ | ❌ |
-| NVIDIA Ampere GPU (SM 80+) | ✅ | ✅ | ✅ | ⭕ | ⭕ | ⭕ | ✅ | ❌ |
-| AMD ROCm | ⭕ | ⭕ | ❌ | ⭕ | ⭕ | ✅ | ⭕ | ❌ |
-| Intel XPU | ⭕ | ⭕ | ❌ | ⭕ | ⭕ | ⭕ | ✅ | ❌ |
-| Ascend NPU | ❌ | ✅ | ❌ | ❌ | ✅ | ✅ | ❌ | ✅ |
 
-Legend: `✅` supported, `❌` unsupported, `⭕` not verified in this
+Quantization support is specific to a model, component, checkpoint format, and
-guide. FP8 on Ampere may use a weight-only path where available.
+hardware backend. The tables in this guide use the following levels:
 
-## Model Type Support
+| Level | Meaning |
+|-------|---------|
+| **CI-backed** | A named model and quantized checkpoint are exercised by scheduled full-model hardware CI. |
+| **Validated** | A named model and checkpoint have end-to-end or quality-validation evidence, but the check is not necessarily scheduled in CI. |
+| **Integrated** | The config, loader, quantized layers, or model wiring exists, but this guide has no completed end-to-end validation for the named model and checkpoint. |
+| **Not validated** | There is no current support recommendation for this model/component combination. |
+| **Unsupported** | The combination is intentionally rejected or known not to work. |
 
-### Diffusion Model (Qwen-Image, Wan2.2)
+A method being available on a device does not make every model on that device
+supported. Likewise, successfully detecting a checkpoint's
+`quantization_config` establishes loader compatibility, not output quality.
 
-These models run a diffusion transformer as the primary inference module. The
+## Hardware × Quantization Method
-default quantization target is the transformer; tokenizer, scheduler, text
-encoder, and VAE stay on the base checkpoint unless a method guide says
-otherwise.
 
-| Method | Guide | Mode | Example models | Status |
+This matrix uses concrete inference paths as columns. Producer toolkits are not
-|--------|-------|------|----------------|--------|
+separate methods: NVIDIA ModelOpt produces the ModelOpt FP8/NVFP4 formats,
-| FP8 W8A8 | [FP8](fp8.md) | Online W8A8 or checkpoint FP8 | Qwen-Image; Wan2.2 is not validated | Validated for Qwen-Image family and other DiT models |
+Intel AutoRound produces AutoRound checkpoints, and Ascend msModelSlim produces
-| Int8 W8A8 | [Int8](int8.md) | Online or serialized W8A8 | Qwen-Image; Wan2.2 is not validated | Validated for Qwen-Image and Z-Image |
+checkpoints consumed by the native MXFP or compatible Ascend paths.
-| BitsAndBytes W4 | [BitsAndBytes](bitsandbytes.md) | Online W4 weight-only | Z-Image-Turbo; Qwen-Image/Wan2.2 not validated | Validated for Z-Image-Turbo |
-| ModelOpt | [ModelOpt](modelopt.md) | Pre-quantized FP8 checkpoints | Qwen-Image, Z-Image, FLUX.2, HunyuanImage-3.0 | Validated for ModelOpt FP8 diffusion checkpoints |
-| MXFP8 W8A8 | [MXFP8](mxfp8.md) | Online W8A8 or offline pre-quantized | Wan2.2-T2V-A14B, I2V-A14B, TI2V-5B | Ascend NPU only; validated for Wan2.2 |
-| MXFP4 W4A4 | [MXFP4](mxfp4.md) | `mxfp4`: online single-scale only; `mxfp4_dualscale`: online or offline dual-scale (offline recommended) | Wan2.2-T2V-A14B, I2V-A14B | Ascend NPU only; validated for Wan2.2 A14B cascade models; TI2V-5B not supported; offline `mxfp4_dualscale` uses calibrated `mul_scale` for best accuracy |
-| AutoRound | [AutoRound](autoround.md) | Pre-quantized W4A16 checkpoints | FLUX.1-dev; Qwen-Image/Wan2.2 not validated | Checkpoint-driven |
-| msModelSlim | [msModelSlim](msmodelslim.md) | Pre-quantized Ascend checkpoints | Wan2.2 recipe; HunyuanImage-3.0 inference target | Ascend/NPU path |
 
-### Multi-Stage Omni/TTS Model (Qwen3-Omni, Qwen3-TTS)
+| Hardware | [Online FP8](fp8.md) | [Int8](int8.md) | [BitsAndBytes W4](bitsandbytes.md) | [ModelOpt FP8](modelopt.md) | [ModelOpt NVFP4](modelopt.md#supported-modelopt-checkpoint-formats) | [AutoRound W4A16](autoround.md) | [OCP MXFP8](mxfp8.md) | [OCP MXFP4](mxfp4.md) |
+|----------|-----------------------|------------------|------------------------------------------|--------------------------------|-------------------------------------------------------------------------|--------------------------------------|--------------------------|--------------------------|
+| NVIDIA Blackwell (SM 100+) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ |
+| NVIDIA Hopper (SM 90) | ✅ | ✅ | ✅ | ✅ | ✅ (Marlin fallback) | ✅ | ❌ | ❌ |
+| NVIDIA Ada (SM 89) | ✅ | ✅ | ✅ | ✅ | ⭕ | ✅ | ❌ | ❌ |
+| NVIDIA Ampere (SM 80+) | ✅ | ✅ | ✅ | ⭕ | ⭕ | ✅ | ❌ | ❌ |
+| AMD ROCm (gfx950 for MXFP4) | ⭕ | ⭕ | ❌ | ⭕ | ⭕ | ⭕ | ❌ | ✅ (online only) |
+| Intel XPU | ⭕ | ⭕ | ❌ | ⭕ | ⭕ | ✅ | ✅ | ❌ |
+| Ascend NPU | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ |
 
-These models combine an AR language model with audio, vision, talker, or TTS
+Legend: `✅` backend available, `❌` rejected or no implementation, `⭕` not
-stages. Quantization is scoped to the AR language-model stage when the
+verified in this guide. This matrix does not establish model-level support; use
-checkpoint contains a supported `quantization_config`; the non-AR stages stay
+the model-and-stage tables below for that.
-in BF16 unless the model guide explicitly adds support.
 
-| Method | Guide | Scope | Example models | Status |
+The Int8 column covers online and serialized Int8 paths. OCP MXFP8 is online on
-|--------|-------|-------|----------------|--------|
+Intel XPU and online/offline on Ascend; Intel's separate AutoRound MXFP8
-| ModelOpt | [ModelOpt](modelopt.md) | Thinker or language-model checkpoint config | Qwen3-Omni thinker | ModelOpt checkpoint path |
+checkpoint path is described in the [MXFP8 guide](mxfp8.md). OCP MXFP4 on AMD
-| Int8 | [Int8](int8.md) | Not currently validated for omni/TTS stages | Qwen3-Omni, Qwen3-TTS | Not validated |
+means online single-scale `mxfp4` on gfx950, while MXFP4 DualScale is
-| BitsAndBytes | [BitsAndBytes](bitsandbytes.md) | Not currently validated for omni/TTS stages | Qwen3-Omni, Qwen3-TTS | Not validated |
+Ascend-only. NVIDIA's NVFP4 and OCP MXFP4 have different checkpoint metadata,
-| MXFP8 | [MXFP8](mxfp8.md) | Not currently validated for omni/TTS stages | Qwen3-Omni, Qwen3-TTS | Not validated |
+scale layouts, and kernel contracts. Mixed ModelOpt FP8/NVFP4 checkpoints also
-| MXFP4 | [MXFP4](mxfp4.md) | Not currently validated for omni/TTS stages | Qwen3-Omni, Qwen3-TTS | Not validated |
+require a model-specific mixed-precision policy and validation. FP8 on Ampere
-| AutoRound | [AutoRound](autoround.md) | Thinker or language-model checkpoint config | Qwen2.5-Omni, Qwen3-Omni | Supported through AutoRound checkpoints |
+may use a weight-only path where available.
-| msModelSlim | [msModelSlim](msmodelslim.md) | Not currently validated for omni/TTS stages | Qwen3-Omni, Qwen3-TTS | Not validated |
 
-### Multi-Stage Diffusion Model (BAGEL, GLM-Image)
+## Current Model and Stage Support
 
-These models split generation across multiple stages. Quantization must be
+The current offline checkpoint guidance is transformer-first. A listed
-attached to the intended stage rather than applied globally.
+checkpoint quantizes the stage named in the **Scope** column; it does not imply
+that the model's encoders, VAE, decoder, or other stages are also quantized.
 
-| Method | Guide | Scope | Example models | Status |
+### Vendor and Runtime Ecosystems
-|--------|-------|-------|----------------|--------|
+
-| FP8 | [FP8](fp8.md) | Stage-specific DiT or transformer module | BAGEL, GLM-Image | Requires model-specific validation |
+The producer and runtime backend are related, but they are not the same support
-| Int8 | [Int8](int8.md) | Stage-specific DiT or transformer module | BAGEL, GLM-Image | Requires model-specific validation |
+claim. A producer can emit several checkpoint formats, and each format still
-| BitsAndBytes | [BitsAndBytes](bitsandbytes.md) | Stage-specific transformer or DiT module | BAGEL, GLM-Image | Not validated |
+needs a compatible vLLM-Omni loader, model-stage wiring, and validation. A
-| ModelOpt | [ModelOpt](modelopt.md) | Checkpoint-defined diffusion stage | BAGEL, GLM-Image | Requires model-specific validation |
+runtime may also provide online quantization without an offline producer path.
-| MXFP8 | [MXFP8](mxfp8.md) | Stage-specific DiT or transformer module | BAGEL, GLM-Image | Not validated |
+
-| MXFP4 | [MXFP4](mxfp4.md) | Stage-specific DiT or transformer module | BAGEL, GLM-Image | Not validated |
+| Ecosystem | Current paths | Current vLLM-Omni scope | Model evidence |
-| AutoRound | [AutoRound](autoround.md) | Checkpoint-defined stage | BAGEL, GLM-Image | No validated checkpoint listed |
+|-----------|---------------|-------------------------|----------------|
-| msModelSlim | [msModelSlim](msmodelslim.md) | Ascend-generated stage weights | GLM-Image | Requires model-specific adaptation |
+| NVIDIA | ModelOpt FP8, NVFP4, and mixed FP8/NVFP4 | Diffusion transformer or Qwen-Omni thinker | Qwen-Image and other named diffusion checkpoints are validated; Qwen3-Omni NVFP4 has H100 CI for the Marlin fallback |
+| Intel | AutoRound W4A16 and AutoRound MXFP8 | Diffusion/world-model transformer or Qwen-Omni thinker | Wan A14B and GLM-Image W4A16 are CI-backed; Qwen-Image, Qwen-Omni, and Cosmos3-Super have validation outside scheduled CI |
+| AMD ROCm | Online OCP MXFP4 through AITER on gfx950; no pre-quantized MXFP4 or AMD Quark workflow is integrated | Diffusion transformer linear layers | Integrated backend with dispatch and weight-allocation unit coverage; no named-model quality validation or scheduled gfx950 full-model CI |
+| Ascend | msModelSlim output through native MXFP8/MXFP4 merge workflows or a compatible generic Ascend format | Wan diffusion transformers | Native Wan MXFP8/MXFP4 paths are validated outside scheduled full-model NPU CI; other model families remain integrated or unvalidated |
+
+### Pre-Quantized Model Matrix
+
+| Model family | Method | Scope | Level | Notes |
+|--------------|--------|-------|-------|-------|
+| Qwen-Image | [ModelOpt](modelopt.md) FP8 or mixed FP8/NVFP4 | Diffusion transformer | Validated | Named Qwen-Image 2512 checkpoints and recipes are available. Text encoder and VAE stay BF16. |
+| Qwen-Image | [AutoRound](autoround.md) W4A16 | Diffusion transformer | Validated | Uses `INC4AI/Qwen-Image-AutoRound-W4A16`; text encoder and VAE stay BF16. |
+| Wan2.2 I2V/T2V A14B | [AutoRound](autoround.md) W4A16 | Both diffusion transformers | CI-backed | The high-noise and low-noise DiTs are quantized; text encoder and VAE stay BF16. |
+| Wan2.2 TI2V 5B | [AutoRound](autoround.md) W4A16 | Diffusion transformer | Validated | Named checkpoint available; not covered by the scheduled Wan AutoRound job. |
+| Wan2.2 | [MXFP8](mxfp8.md) | One or both diffusion transformers | Validated | Native Ascend and AutoRound checkpoint formats are integrated; no scheduled full-model offline NPU job. |
+| Wan2.2 I2V/T2V A14B | [MXFP4 DualScale](mxfp4.md) | Both diffusion transformers | Validated | Offline calibrated checkpoints are recommended; no scheduled full-model offline NPU job. |
+| Qwen2.5-Omni | [AutoRound](autoround.md) W4A16 | Thinker language model | Validated | Encoders and output/audio stages stay BF16. |
+| Qwen3-Omni | [ModelOpt](modelopt.md) FP8 | Thinker language model | Validated | Tested outside scheduled CI. |
+| Qwen3-Omni | [ModelOpt](modelopt.md) NVFP4 | Thinker language model | CI-backed | H100 CI exercises the Marlin fallback; the native Blackwell FP4 path has separate recipe validation. |
+| Qwen3-Omni | [AutoRound](autoround.md) W4A16 | Thinker language model | Validated | Audio encoder, vision encoder, talker, and Code2Wav stay BF16. |
+| GLM-Image | [AutoRound](autoround.md) W4A16 | Diffusion transformer stage | CI-backed | Text-to-image and image-to-image full-model smoke coverage. |
+| Cosmos3-Super | [AutoRound](autoround.md) W4A16 | World-model transformer | Validated | Manually validated serving path; VAE and other components stay BF16. |
+| Cosmos3-Nano | [AutoRound](autoround.md) W4A16 | World-model transformer | Integrated | Named checkpoint is listed, but no model-specific end-to-end test is maintained in-tree. |
+| MiniMax H3 | None | None | Not validated | No offline vendor checkpoint path is documented; online FP8 targets the DiT only. |
+
+The method pages contain the complete checkpoint lists, including FLUX,
+Z-Image, and HunyuanImage-3.0 variants.
+
+### Online Quantization
+
+| Model family | Methods | Scope | Level |
+|--------------|---------|-------|-------|
+| Qwen-Image | FP8, Int8 | Diffusion transformer | Validated |
+| Z-Image | Int8, BitsAndBytes W4 | Diffusion transformer | Validated |
+| Wan2.2 | MXFP8 | One or both diffusion transformers | Validated on the documented XPU/NPU paths |
+| Wan2.2 A14B | MXFP4 | Both diffusion transformers | Validated on Ascend; the ROCm gfx950 online backend is integrated but not model-validated |
+| MiniMax H3 | FP8 | DiT and optional reference DiT | Validated |
+| Qwen3-Omni | Generic online methods, such as FP8 | Thinker language model only | Integrated; not model-validated as an online recommendation |
+
+See [Online Quantization](online.md) for configuration and hardware details.
+
+### Unlisted and Long-Tail Models
+
+For models not listed above, generic quantized linear layers or compatible
+checkpoint metadata are only an integration starting point. The default support
+target is the model's primary transformer, DiT, or AR language-model stage.
+Text/audio/vision encoders, VAEs, vocoders, decoders, and output stages remain
+BF16 until a model-specific integration supplies quant-aware layers, checkpoint
+mapping, end-to-end output validation, and backend coverage.
 
 !!! note
     "Online quantization" means vLLM-Omni computes the quantization data while
@@ -92,8 +143,8 @@ attached to the intended stage rather than applied globally.
 
 ### Diffusion Model (Qwen-Image, Wan2.2)
 
-The default target is the diffusion transformer. Component routing is available
+The default target is the diffusion transformer. `build_quant_config()` can
-through `build_quant_config()`:
+construct a component router:
 
 ```python
 from vllm_omni.quantization import build_quant_config
@@ -104,11 +155,19 @@ config = build_quant_config({
 })
 ```
 
+!!! warning "Routing syntax is not a support claim"
+    A component dictionary only selects a quantization config by module prefix.
+    It does not make an encoder or VAE quantizable. The model must construct
+    that component from quantization-aware layers, and an offline checkpoint
+    must contain matching packed weights and scales. Current user guidance
+    keeps diffusion text encoders and VAEs in BF16.
+
 | Component | Default quantized? | Notes |
 |-----------|--------------------|-------|
-| Diffusion transformer | Yes | Primary target for FP8, Int8, BitsAndBytes, ModelOpt, MXFP8, MXFP4, AutoRound, and msModelSlim |
+| Diffusion transformer | Yes, for listed model/method pairs | Primary target for FP8, Int8, BitsAndBytes, ModelOpt, MXFP8, MXFP4, AutoRound, and msModelSlim |
+| Second diffusion transformer | Yes, for listed Wan2.2 A14B methods | `transformer` and `transformer_2` read their own offline checkpoint metadata; arbitrary mixed methods are not currently validated |
 | Text encoder | No | Keep BF16 unless a method-specific guide documents support |
-| VAE | No | Keep BF16; storage-only paths are method-specific |
+| VAE | No | Keep BF16 |
 | Scheduler/tokenizer | No | Loaded from the base model repository |
 
 ### Multi-Stage Omni/TTS Model (Qwen3-Omni, Qwen3-TTS)
@@ -121,6 +180,18 @@ config = build_quant_config({
 | Talker or TTS stage | No | BF16 unless model-specific support is documented |
 | Code2Wav | No | BF16 |
 
+An explicit `ComponentQuantizationConfig` can be used by model developers to
+route different configs to `audio_tower`, `visual`, and `language_model`, but
+the supported Qwen-Omni checkpoints currently quantize only the thinker
+language model.
+
+### World Model (Cosmos3)
+
+| Component | Default quantized? | Notes |
+|-----------|--------------------|-------|
+| World-model transformer | AutoRound checkpoints only | Cosmos3-Super is validated; Cosmos3-Nano is integrated but not end-to-end tested in-tree |
+| VAE and guardrail components | No | BF16 |
+
 ### Multi-Stage Diffusion Model (BAGEL, GLM-Image)
 
 | Component | Default quantized? | Notes |
@@ -141,6 +212,7 @@ build_quant_config("fp8")
 build_quant_config({"method": "fp8", "activation_scheme": "static"})
 build_quant_config("bitsandbytes")
 build_quant_config("auto-round", bits=4, group_size=128)
+# Component routing syntax; model-specific support is still required.
 build_quant_config({"transformer": {"method": "fp8"}, "vae": None})
 build_quant_config(None)
 ```

tests/diffusion/test_diffusion_output_formatter.py

@@ -95,6 +95,7 @@ def test_formatter_preserves_single_video_audio_actions_and_metadata(
         "resolution": 512,
         "width": None,
         "height": None,
+        "num_inference_steps": 1,
     }
 
 

tests/metrics/test_modality.py

@@ -42,6 +42,10 @@ _EXPECTED_FAMILIES = [
     defs.AUDIO_UNDERRUN_S,
     defs.AUDIO_CONTINUITY_OK_METRIC,
     defs.AUDIO_SKIPPED_REQUESTS_METRIC,
+    defs.DIFFUSION_EXEC_S,
+    defs.DIFFUSION_EXEC_PER_STEP_S,
+    defs.DIFFUSION_PREPROCESS_S,
+    defs.DIFFUSION_POSTPROCESS_S,
 ]
 
 
@@ -55,6 +59,10 @@ class TestRegistration:
         mod.observe_audio_underrun("s", "r", 0.01)
         mod.inc_audio_continuity_ok("s", "r", 100)
         mod.inc_audio_skipped("s", "r", "malformed_codec")
+        mod.observe_diffusion_exec("s", "r", 0.5)
+        mod.observe_diffusion_exec_per_step("s", "r", 0.01)
+        mod.observe_diffusion_preprocess("s", "r", 0.01)
+        mod.observe_diffusion_postprocess("s", "r", 0.2)
 
         out = generate_latest(REGISTRY).decode()
         for name in _EXPECTED_FAMILIES:
@@ -156,6 +164,50 @@ class TestAudio:
         assert unknown == 1.0
 
 
+# ---------------------------------------------------------------------------
+# Diffusion observe API
+# ---------------------------------------------------------------------------
+
+
+class TestDiffusion:
+    def test_diffusion_exec_observed(self, mod: OmniModalityMetrics) -> None:
+        stage, replica = "dit_exec", "0"
+        mod.observe_diffusion_exec(stage, replica, 1.25)
+        out = generate_latest(REGISTRY).decode()
+        prefix = f'{defs.DIFFUSION_EXEC_S}_sum{{model_name="{_MODEL}",replica="{replica}",stage="{stage}"}}'
+        assert _sample_value(out, prefix) == 1.25
+
+    def test_diffusion_preprocess_observed(self, mod: OmniModalityMetrics) -> None:
+        stage, replica = "dit_pre", "0"
+        mod.observe_diffusion_preprocess(stage, replica, 0.03)
+        out = generate_latest(REGISTRY).decode()
+        prefix = f'{defs.DIFFUSION_PREPROCESS_S}_sum{{model_name="{_MODEL}",replica="{replica}",stage="{stage}"}}'
+        assert _sample_value(out, prefix) == pytest.approx(0.03)
+
+    def test_diffusion_postprocess_observed(self, mod: OmniModalityMetrics) -> None:
+        stage, replica = "dit_post", "0"
+        mod.observe_diffusion_postprocess(stage, replica, 0.8)
+        out = generate_latest(REGISTRY).decode()
+        prefix = f'{defs.DIFFUSION_POSTPROCESS_S}_sum{{model_name="{_MODEL}",replica="{replica}",stage="{stage}"}}'
+        assert _sample_value(out, prefix) == 0.8
+
+    def test_diffusion_exec_per_step_observed(self, mod: OmniModalityMetrics) -> None:
+        stage, replica = "dit_per_step", "0"
+        mod.observe_diffusion_exec_per_step(stage, replica, 0.078)
+        out = generate_latest(REGISTRY).decode()
+        prefix = f'{defs.DIFFUSION_EXEC_PER_STEP_S}_sum{{model_name="{_MODEL}",replica="{replica}",stage="{stage}"}}'
+        assert _sample_value(out, prefix) == pytest.approx(0.078)
+
+    def test_log_stats_false_skips(self) -> None:
+        silent = OmniModalityMetrics(model_name="silent-model", log_stats=False)
+        silent.observe_diffusion_exec("s", "r", 1.0)
+        silent.observe_diffusion_exec_per_step("s", "r", 0.02)
+        silent.observe_diffusion_preprocess("s", "r", 0.01)
+        silent.observe_diffusion_postprocess("s", "r", 0.5)
+        out = generate_latest(REGISTRY).decode()
+        assert 'model_name="silent-model"' not in out
+
+
 # ---------------------------------------------------------------------------
 # Stub-driven routing tests for the finalize / streaming helpers.
 # ---------------------------------------------------------------------------
@@ -183,6 +235,18 @@ class _StubModMetrics:
     def inc_audio_skipped(self, s, r, reason):
         self.calls.append(("inc_audio_skipped", s, r, reason))
 
+    def observe_diffusion_exec(self, s, r, v):
+        self.calls.append(("observe_diffusion_exec", s, r, v))
+
+    def observe_diffusion_exec_per_step(self, s, r, v):
+        self.calls.append(("observe_diffusion_exec_per_step", s, r, v))
+
+    def observe_diffusion_preprocess(self, s, r, v):
+        self.calls.append(("observe_diffusion_preprocess", s, r, v))
+
+    def observe_diffusion_postprocess(self, s, r, v):
+        self.calls.append(("observe_diffusion_postprocess", s, r, v))
+
 
 class _Bag:
     def __init__(self, **kw):
@@ -239,8 +303,88 @@ class TestObserveModalityAtFinalize:
         )
         assert ("observe_audio_duration", "1", "0", 1.0) in stub.calls
 
-    def test_non_audio_output_type_skipped(self):
+    def test_diffusion_path_full(self):
-        # Image / video / text output types are out of scope for this module.
+        stub = _StubModMetrics()
+        stage_metrics = _Bag(
+            stage_gen_time_ms=1000.0,
+            diffusion_metrics={
+                "preprocess_time_s": 0.05,
+                "diffusion_engine_exec_time_s": 1.2,
+                "postprocess_time_s": 0.3,
+                "image_num": 1,
+                "resolution": 1024,
+                "num_inference_steps": 50,
+            },
+        )
+        observe_modality_at_finalize(
+            stub,
+            output_type="image",
+            stage_id=2,
+            replica_id=0,
+            stage_metrics=stage_metrics,
+            engine_outputs=_Bag(),
+        )
+        assert ("observe_diffusion_preprocess", "2", "0", 0.05) in stub.calls
+        assert ("observe_diffusion_exec", "2", "0", 1.2) in stub.calls
+        assert ("observe_diffusion_postprocess", "2", "0", 0.3) in stub.calls
+        assert ("observe_diffusion_exec_per_step", "2", "0", pytest.approx(0.024)) in stub.calls
+
+    def test_diffusion_per_step_skipped_without_num_inference_steps(self):
+        stub = _StubModMetrics()
+        stage_metrics = _Bag(
+            stage_gen_time_ms=1000.0,
+            diffusion_metrics={
+                "diffusion_engine_exec_time_s": 1.2,
+                "preprocess_time_s": 0.05,
+                "postprocess_time_s": 0.3,
+            },
+        )
+        observe_modality_at_finalize(
+            stub,
+            output_type="image",
+            stage_id=2,
+            replica_id=0,
+            stage_metrics=stage_metrics,
+            engine_outputs=_Bag(),
+        )
+        assert ("observe_diffusion_exec", "2", "0", 1.2) in stub.calls
+        assert not any(c[0] == "observe_diffusion_exec_per_step" for c in stub.calls)
+
+    def test_diffusion_metrics_none_skipped(self):
+        stub = _StubModMetrics()
+        observe_modality_at_finalize(
+            stub,
+            output_type="image",
+            stage_id=2,
+            replica_id=0,
+            stage_metrics=_Bag(stage_gen_time_ms=500.0, diffusion_metrics=None),
+            engine_outputs=_Bag(),
+        )
+        assert not any(c[0].startswith("observe_diffusion") for c in stub.calls)
+
+    def test_diffusion_zero_values_observed(self):
+        stub = _StubModMetrics()
+        stage_metrics = _Bag(
+            stage_gen_time_ms=500.0,
+            diffusion_metrics={
+                "preprocess_time_s": 0.0,
+                "diffusion_engine_exec_time_s": 0.0,
+                "postprocess_time_s": 0.0,
+            },
+        )
+        observe_modality_at_finalize(
+            stub,
+            output_type="image",
+            stage_id=2,
+            replica_id=0,
+            stage_metrics=stage_metrics,
+            engine_outputs=_Bag(),
+        )
+        assert ("observe_diffusion_preprocess", "2", "0", 0.0) in stub.calls
+        assert ("observe_diffusion_exec", "2", "0", 0.0) in stub.calls
+        assert ("observe_diffusion_postprocess", "2", "0", 0.0) in stub.calls
+
+    def test_non_audio_output_type_skips_audio_but_still_checks_diffusion(self):
         stub = _StubModMetrics()
         for output_type in ("text", "image", "video"):
             observe_modality_at_finalize(
@@ -251,7 +395,7 @@ class TestObserveModalityAtFinalize:
                 stage_metrics=_Bag(stage_gen_time_ms=100.0),
                 engine_outputs=_Bag(),
             )
-        assert stub.calls == []
+        assert not any(c[0].startswith("observe_audio") or c[0].startswith("inc_audio") for c in stub.calls)
 
     def test_replica_id_none_skipped(self):
         stub = _StubModMetrics()

vllm_omni/diffusion/output_formatter.py

@@ -138,6 +138,7 @@ def format_diffusion_outputs(
         ),
         "width": request.sampling_params.width,
         "height": request.sampling_params.height,
+        "num_inference_steps": request.sampling_params.num_inference_steps,
     }
 
     # Only the primary payload determines the response type. Some image models

vllm_omni/entrypoints/omni_base.py

@@ -568,6 +568,7 @@ class OmniBase(PDDisaggregationMixin):
             msg_id = id(result)
             consumed = self._consumed_metric_message_ids(req_id)
             if msg_id not in consumed:
+                metrics.accumulate_diffusion_metrics(stage_meta.stage_type, req_id, engine_outputs)
                 metrics.on_stage_metrics(stage_id, req_id, _m, output_type)
                 consumed.add(msg_id)
 

vllm_omni/metrics/definitions.py

@@ -134,6 +134,15 @@ AUDIO_CONTINUITY_OK_METRIC = METRIC_PREFIX + AUDIO_CONTINUITY_OK
 AUDIO_SKIPPED_REQUESTS_METRIC = METRIC_PREFIX + AUDIO_SKIPPED_REQUESTS
 
 
+# ============================================================================
+# Diffusion family (per-stage + per-replica diffusion timing breakdowns)
+# ============================================================================
+DIFFUSION_EXEC_S = METRIC_PREFIX + "diffusion_exec_s"
+DIFFUSION_EXEC_PER_STEP_S = METRIC_PREFIX + "diffusion_exec_per_step_s"
+DIFFUSION_PREPROCESS_S = METRIC_PREFIX + "diffusion_preprocess_s"
+DIFFUSION_POSTPROCESS_S = METRIC_PREFIX + "diffusion_postprocess_s"
+
+
 # ============================================================================
 # Cross-stage Transfer family (per-physical-hop TX/RX/in-flight timings)
 # ============================================================================

vllm_omni/metrics/modality.py

@@ -78,6 +78,35 @@ _audio_skipped_family = Counter(
 )
 
 
+# ----------------------------------------------------------------------------
+# Diffusion family
+# ----------------------------------------------------------------------------
+_diffusion_exec_family = Histogram(
+    defs.DIFFUSION_EXEC_S,
+    "DiT forward pass execution time per request in seconds.",
+    labelnames=_stage_labels,
+    buckets=defs.SECONDS_BUCKETS,
+)
+_diffusion_exec_per_step_family = Histogram(
+    defs.DIFFUSION_EXEC_PER_STEP_S,
+    "DiT forward pass execution time per denoising step in seconds.",
+    labelnames=_stage_labels,
+    buckets=defs.SECONDS_FAST_BUCKETS,
+)
+_diffusion_preprocess_family = Histogram(
+    defs.DIFFUSION_PREPROCESS_S,
+    "Diffusion input preprocessing time per request in seconds.",
+    labelnames=_stage_labels,
+    buckets=defs.SECONDS_FAST_BUCKETS,
+)
+_diffusion_postprocess_family = Histogram(
+    defs.DIFFUSION_POSTPROCESS_S,
+    "Diffusion output postprocessing (VAE decode) time per request in seconds.",
+    labelnames=_stage_labels,
+    buckets=defs.SECONDS_FAST_BUCKETS,
+)
+
+
 class OmniModalityMetrics:
     """Per-modality observe API. Stage/replica are passed at observe time
     because a single OmniModalityMetrics instance per pipeline serves all
@@ -139,6 +168,44 @@ class OmniModalityMetrics:
             reason=reason or "unknown",
         ).inc()
 
+    # ---- Diffusion --------------------------------------------------------
+
+    def observe_diffusion_exec(self, stage: str, replica: str, seconds: float) -> None:
+        if not self._log_stats:
+            return
+        _diffusion_exec_family.labels(
+            model_name=self._model_name,
+            stage=stage,
+            replica=replica,
+        ).observe(seconds)
+
+    def observe_diffusion_exec_per_step(self, stage: str, replica: str, seconds: float) -> None:
+        if not self._log_stats:
+            return
+        _diffusion_exec_per_step_family.labels(
+            model_name=self._model_name,
+            stage=stage,
+            replica=replica,
+        ).observe(seconds)
+
+    def observe_diffusion_preprocess(self, stage: str, replica: str, seconds: float) -> None:
+        if not self._log_stats:
+            return
+        _diffusion_preprocess_family.labels(
+            model_name=self._model_name,
+            stage=stage,
+            replica=replica,
+        ).observe(seconds)
+
+    def observe_diffusion_postprocess(self, stage: str, replica: str, seconds: float) -> None:
+        if not self._log_stats:
+            return
+        _diffusion_postprocess_family.labels(
+            model_name=self._model_name,
+            stage=stage,
+            replica=replica,
+        ).observe(seconds)
+
 
 def observe_modality_at_finalize(
     mod_metrics: OmniModalityMetrics,
@@ -161,19 +228,14 @@ def observe_modality_at_finalize(
     """
     if replica_id is None or stage_metrics is None or output_type is None:
         return
-    if output_type != "audio":
-        return
 
     stage_label = str(stage_id)
     replica_label = str(replica_id)
     gen_time_s = float(getattr(stage_metrics, "stage_gen_time_ms", 0.0)) / 1000.0
     mm_out = extract_mm_output(engine_outputs)
 
+    if output_type == "audio":
         sample_rate = defs.resolve_audio_sample_rate(mm_out)
-    # `stage_metrics.audio_generated_frames` is the legacy per-chunk
-    # accumulator field on StageRequestStats. No production path currently
-    # fills it, so the fallback below is the live source — but we leave the
-    # field lookup in place in case the accumulator gets re-wired upstream.
         n_frames = int(getattr(stage_metrics, "audio_generated_frames", 0) or 0)
         if n_frames == 0:
             n_frames = count_audio_frames(mm_out)
@@ -187,17 +249,28 @@ def observe_modality_at_finalize(
                 defs.compute_audio_rtf(gen_time_s, duration_s),
             )
         else:
-        # Request completed (finish_reason ∈ {stop, length} — error paths
-        # don't reach finalize) but no audio samples were produced. Covers
-        # silent `return None` skips in the talker→code2wav stage
-        # processors and the `parsed.append((0,0))` malformed-length path
-        # in qwen3-tts code2wav. raise-paths surface via the upstream
-        # vllm:request_success_total{finished_reason="error"} channel and
-        # never reach this branch.
             mod_metrics.inc_audio_skipped(stage_label, replica_label, "no_audio_data")
-    # audio_underrun / continuity are emitted from the streaming path in
+
-    # observe_audio_streaming_finalize; finalize is too late for the
+    dm = getattr(stage_metrics, "diffusion_metrics", None)
-    # per-chunk timeline they need.
+    if dm:
+        _key_map = {
+            "diffusion_engine_exec_time_s": mod_metrics.observe_diffusion_exec,
+            "preprocess_time_s": mod_metrics.observe_diffusion_preprocess,
+            "postprocess_time_s": mod_metrics.observe_diffusion_postprocess,
+        }
+        for key, observe_fn in _key_map.items():
+            val = dm.get(key)
+            if val is not None:
+                observe_fn(stage_label, replica_label, float(val))
+
+        exec_time = dm.get("diffusion_engine_exec_time_s")
+        num_steps = dm.get("num_inference_steps")
+        if exec_time is not None and num_steps and num_steps > 0:
+            mod_metrics.observe_diffusion_exec_per_step(
+                stage_label,
+                replica_label,
+                float(exec_time) / int(num_steps),
+            )
 
 
 def observe_audio_first_packet(

vllm_omni/metrics/stats.py

@@ -44,7 +44,7 @@ class StageRequestStats:
     final_output_type: str | None = None
     request_id: str | None = None
     postprocess_time_ms: float = 0.0
-    diffusion_metrics: dict[str, int] = None
+    diffusion_metrics: dict[str, float] = None
     audio_generated_frames: int = 0
     audio_sample_rate: int = 0
     audio_duration_s: float = 0.0
@@ -579,7 +579,7 @@ class OrchestratorAggregator:
         if final_output_type is not None:
             stats.final_output_type = final_output_type
         stats.diffusion_metrics = (
-            {k: int(v) for k, v in self.diffusion_metrics.pop(req_id, {}).items()}
+            {k: float(v) for k, v in self.diffusion_metrics.pop(req_id, {}).items()}
             if req_id in self.diffusion_metrics
             else None
         )
@@ -644,8 +644,19 @@ class OrchestratorAggregator:
         if isinstance(diffusion_metrics, list):
             diffusion_metrics = diffusion_metrics[0]
         if diffusion_metrics:
+            _MS_TO_S = {
+                "diffusion_engine_exec_time_ms": "diffusion_engine_exec_time_s",
+                "preprocess_time_ms": "preprocess_time_s",
+                "postprocess_time_ms": "postprocess_time_s",
+                "diffusion_engine_total_time_ms": "diffusion_engine_total_time_s",
+            }
             for key, value in diffusion_metrics.items():
-                self.diffusion_metrics[req_id][key] += value
+                if value is None:
+                    continue
+                if key in _MS_TO_S:
+                    self.diffusion_metrics[req_id][_MS_TO_S[key]] = float(value) / 1000.0
+                else:
+                    self.diffusion_metrics[req_id][key] = value
 
     def on_forward(
         self,