[Diffusion][Quantization] Enable MiniMax-H3 global FP8 with DLO (#5910)

Signed-off-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
This commit is contained in:
SYLARandHongsheng Liu authored and GitHub committed 2026-08-15 23:01:26 +08:00
1 parent 09b6a1604a
commit d1e230c95b
18 files changed
+833 -165

No files matched your search

+11 -3
View File
@@ -76,6 +76,12 @@ equivalent. `auto-round` and `auto_round` use the Omni INC/AutoRound adapter.
ModelOpt checkpoint metadata is detected from its method, producer metadata, or
`quant_algo`, then mapped to the corresponding vLLM ModelOpt configuration.
Unless the input is a per-component dictionary, the resolved configuration is
passed unchanged to every quantization-aware component constructed by the
pipeline. It affects only layers supported by that component's quantization
method; it does not rewrite arbitrary `torch.nn` modules. A per-component
dictionary narrows this global scope through runtime layer-prefix routing.
The factory uses Omni-specific overrides for methods whose diffusion behavior
or platform dispatch is not provided by the vLLM registry. The current override
set includes INT8, BitsAndBytes, MXFP8, MXFP4, MXFP4 dual-scale, and INC/
@@ -154,9 +160,11 @@ quantizable vLLM layers. Each method configuration is responsible for:
Model code must keep precision-sensitive or unsupported modules unquantized.
The shared `safe_quant_config()` helper is used by model integrations for
normalization and modulation layers where a generic FP8 configuration would be
unsafe. Text encoders, audio/vision encoders, VAEs, schedulers, and tokenizers
are not implicitly quantized just because the main transformer is quantized;
they require explicit component routing and method support.
unsafe. Encoders exposed through the quantization factory receive a global
configuration just like the main transformer, but only their supported vLLM
quantizable layers are affected. Ordinary `torch.nn` encoder layers, VAEs,
schedulers, and tokenizers are not rewritten automatically. A component map can
select a narrower scope, but it cannot add quantization support to a layer.
Weight quantization and quantized KV cache are separate paths. The former is
resolved through `quantization_config`; runtime attention/KV-cache quantization