vllm-omni/docs/user_guide/quantization/msmodelslim.md

msModelSlim Quantization

Overview

msModelSlim is an Ascend compression toolkit for producing pre-quantized model checkpoints. vLLM-Omni consumes its output either through a compatible generic Ascend checkpoint path or through the native MXFP8 and MXFP4 merge workflows documented in this guide.

msModelSlim is static quantization: quantized weights are generated offline before vLLM-Omni inference starts.

The validation terms on this page follow the support levels. A producer recipe or hardware backend alone is an integration point, not model-level validation.

Hardware Support

Device	Support
NVIDIA Blackwell GPU (SM 100+)	❌
NVIDIA Ada/Hopper GPU (SM 89+)	❌
NVIDIA Ampere GPU (SM 80+)	❌
AMD ROCm	❌
Intel XPU	❌
Ascend NPU	✅

Legend: ✅ backend available, ❌ unsupported, ⭕ not verified in this guide.

Model Type Support

Diffusion Model (Qwen-Image, Wan2.2)

Model	Base model	Scope	Level	Notes
Wan2.2	Wan2.2 diffusion weights	DiT or diffusion stage	Integrated	Upstream msModelSlim provides a producer recipe; use the MXFP8 or MXFP4 workflow for vLLM-Omni's native merged formats
Qwen-Image	Qwen/Qwen-Image, Qwen/Qwen-Image-2512	DiT or diffusion stage	Not validated	No supported checkpoint workflow is documented
HunyuanImage-3.0	tencent/HunyuanImage-3.0, tencent/HunyuanImage-3.0-Instruct	DiT or diffusion stage	Integrated	Generate quantized weights with the HunyuanImage-3.0 msModelSlim adaptation and validate against BF16

Public Hugging Face quantized weights are not available yet. Use the HunyuanImage-3.0 msModelSlim adaptation to generate the checkpoint manually.

msModelSlim is also the producer used by the native offline MXFP8 and MXFP4 DualScale workflows. Those workflows preprocess and annotate each transformer checkpoint and are auto-detected at inference; follow their method pages rather than passing --quantization ascend.

Multi-Stage Omni/TTS Model (Qwen3-Omni, Qwen3-TTS)

Model	Scope	Status	Notes
Qwen3-Omni	Thinker or language-model stage	Not validated	No msModelSlim omni checkpoint path is documented
Qwen3-TTS	TTS language-model stage	Not validated	No msModelSlim TTS checkpoint path is documented

Multi-Stage Diffusion Model (BAGEL, GLM-Image)

Model	Scope	Status	Notes
BAGEL	Stage-specific diffusion or transformer weights	Not validated	Requires a model-specific Ascend adaptation
GLM-Image	Stage-specific diffusion or transformer weights	Not validated	Requires a model-specific Ascend adaptation

Configuration

For a compatible generic Ascend checkpoint, select the Ascend quantization method explicitly:

Offline inference:

python text_to_image.py --model <compatible-ascend-checkpoint> --quantization ascend

Online serving:

vllm serve <compatible-ascend-checkpoint> --omni --quantization ascend

Parameters

Parameter	Type	Default	Description
--quantization	str	-	Use ascend only for a compatible generic Ascend-format checkpoint; omit it for native merged MXFP8/MXFP4 checkpoints
model	str	-	Path to the quantized checkpoint generated by Ascend tooling

Do not pass --quantization ascend for a Wan checkpoint processed by the native MXFP8 or MXFP4 merge tools. Their transformer metadata selects mxfp8 or mxfp4_dualscale automatically.

Example msModelSlim command for a Wan2.2 W8A8 checkpoint:

msmodelslim quant \
  --model_path /path/to/wan2_2_t2v_float_weights \
  --save_path /path/to/wan2_2_t2v_quantized_weights \
  --device npu \
  --model_type Wan2_2 \
  --config_path /path/to/wan2_2_w8a8f8_mxfp_t2v.yaml \
  --trust_remote_code True

For HunyuanImage-3.0, use the Hunyuan-specific adaptation linked above.

Validation and Notes

1. Run with the Ascend/NPU installation and environment.
2. The ascend quantization method expects weights produced by the Ascend tooling; it is not a load-time CUDA quantizer.
3. Keep the quantized checkpoint aligned with the same model architecture and stage config used for BF16 inference.
4. No named full-model msModelSlim checkpoint is currently scheduled in NPU CI. Treat new producer outputs as Integrated until an end-to-end BF16 comparison establishes Validated support.