[Diffusion][Quantization] Enable MiniMax-H3 global FP8 with DLO (#5910)
Signed-off-by: lishunyang12 <lishunyang12@163.com> Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
This commit is contained in:
1 parent
09b6a1604a
commit
d1e230c95b
18 files changed
+833
-165
No files matched your search
@@ -76,6 +76,12 @@ equivalent. `auto-round` and `auto_round` use the Omni INC/AutoRound adapter.
|
||||
ModelOpt checkpoint metadata is detected from its method, producer metadata, or
|
||||
`quant_algo`, then mapped to the corresponding vLLM ModelOpt configuration.
|
||||
|
||||
Unless the input is a per-component dictionary, the resolved configuration is
|
||||
passed unchanged to every quantization-aware component constructed by the
|
||||
pipeline. It affects only layers supported by that component's quantization
|
||||
method; it does not rewrite arbitrary `torch.nn` modules. A per-component
|
||||
dictionary narrows this global scope through runtime layer-prefix routing.
|
||||
|
||||
The factory uses Omni-specific overrides for methods whose diffusion behavior
|
||||
or platform dispatch is not provided by the vLLM registry. The current override
|
||||
set includes INT8, BitsAndBytes, MXFP8, MXFP4, MXFP4 dual-scale, and INC/
|
||||
@@ -154,9 +160,11 @@ quantizable vLLM layers. Each method configuration is responsible for:
|
||||
Model code must keep precision-sensitive or unsupported modules unquantized.
|
||||
The shared `safe_quant_config()` helper is used by model integrations for
|
||||
normalization and modulation layers where a generic FP8 configuration would be
|
||||
unsafe. Text encoders, audio/vision encoders, VAEs, schedulers, and tokenizers
|
||||
are not implicitly quantized just because the main transformer is quantized;
|
||||
they require explicit component routing and method support.
|
||||
unsafe. Encoders exposed through the quantization factory receive a global
|
||||
configuration just like the main transformer, but only their supported vLLM
|
||||
quantizable layers are affected. Ordinary `torch.nn` encoder layers, VAEs,
|
||||
schedulers, and tokenizers are not rewritten automatically. A component map can
|
||||
select a narrower scope, but it cannot add quantization support to a layer.
|
||||
|
||||
Weight quantization and quantized KV cache are separate paths. The former is
|
||||
resolved through `quantization_config`; runtime attention/KV-cache quantization
|
||||
|
||||
@@ -95,7 +95,7 @@ warmup_quack_fp8([(14040, 2048, 6144), (14040, 2048, 2048)])
|
||||
| HunyuanImage-3.0 | `tencent/HunyuanImage-3.0`, `tencent/HunyuanImage-3.0-Instruct` | Yes | Yes | All layers; use the Hunyuan stage config for multi-stage runs | None | |
|
||||
| HunyuanVideo-1.5 | `hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-480p_t2v`, `720p_t2v`, `480p_i2v` | Yes | Yes | All layers | None | |
|
||||
| Cosmos3 | `nvidia/Cosmos3-Nano`, `nvidia/Cosmos3-Super` | Yes | Not validated | All layers | None | |
|
||||
| MiniMax H3 | `MiniMaxAI/MiniMax-H3` | Yes | Not validated | DiT linears except patch, timestep, and final projections; not compatible with layerwise offload | None | |
|
||||
| MiniMax-H3 | `MiniMaxAI/MiniMax-H3` (`FL2VA` / `Ref2VA`) | Yes | Not validated | `quantization="fp8"` quantizes eligible DiT and text-encoder linears; mixed-precision input/output heads stay FP32 | None | ✅︎ |
|
||||
|
||||
### Multi-Stage Omni/TTS Model (Qwen3-Omni, Qwen3-TTS)
|
||||
|
||||
@@ -138,6 +138,39 @@ outputs = omni.generate(
|
||||
)
|
||||
```
|
||||
|
||||
### Global and per-component scope
|
||||
|
||||
A plain, global quantization configuration is passed unchanged to every
|
||||
quantization-aware component constructed by a pipeline. This includes an
|
||||
eligible encoder when that encoder is implemented with vLLM quantizable
|
||||
layers; it does not rewrite arbitrary `torch.nn` modules. A structured
|
||||
component map is the only way to narrow that scope. Pipeline integrations that
|
||||
do not yet expose an encoder through the quantization factory remain DiT-only.
|
||||
|
||||
For a pipeline that exposes both a transformer and a quantization-aware text
|
||||
encoder, the scope is:
|
||||
|
||||
| Configuration | Transformer | Text encoder | Components without supported quantizable layers |
|
||||
|---------------|-------------|--------------|-------------------------------------------------|
|
||||
| `quantization="fp8"` | FP8 | FP8 | checkpoint precision |
|
||||
| `{"transformer": {"method": "fp8"}}` | FP8 | checkpoint precision | checkpoint precision |
|
||||
| `{"text_encoder": {"method": "fp8"}}` | checkpoint precision | FP8 | checkpoint precision |
|
||||
|
||||
Use `quantization_config` for component-selective Python configuration. For
|
||||
example, quantize only the text decoder:
|
||||
|
||||
```python
|
||||
omni = Omni(
|
||||
model="<your-model>",
|
||||
quantization_config={"text_encoder": {"method": "fp8"}},
|
||||
)
|
||||
```
|
||||
|
||||
Component keys are runtime prefixes exposed by the pipeline integration; common
|
||||
keys include `transformer` and `text_encoder`. Entries may be combined, and
|
||||
`ignored_layers` can keep named eligible layers in checkpoint precision. Check
|
||||
the model recipe for supported components and their runtime prefixes.
|
||||
|
||||
CLI:
|
||||
|
||||
```bash
|
||||
|
||||
Reference in new issue
Block a user