vllm-omni/.claude/skills/quantization/references/adding-models.md

Adding Quantization Support to a Model

Use this checklist when extending quantization to a new architecture or stage.

Classify Ownership

Case	Owner
New generic AR method, kernel, or loader semantics	upstream vllm first
Existing upstream method needs diffusion wiring	vllm-omni model and config integration
GGUF or ModelOpt tensor mapping for a diffusion architecture	vllm-omni adapter
NPU MXFP or msModelSlim diffusion path	vllm-omni platform/method wrapper and docs
Multi-stage omni checkpoint quantizes only the thinker LM	vllm-omni component scoping

Avoid building a private quantization stack in model code.

Model Wiring Checklist

1. Find the closest supported model implementation.
2. Confirm the model uses vLLM linear layers that accept quant_config.
3. Add quant_config to the transformer constructor.
4. Pass stable prefix values through every linear layer.
5. Handle fused modules and packed mappings explicitly.
6. Ensure load_weights() and any WeightsMapper remap ignored_layers or modules_to_not_convert.
7. Keep tokenizer, scheduler, VAE, text encoder, audio/vision encoders, and talker BF16 unless method-specific docs validate them.
8. Add or update a method doc under docs/user_guide/quantization/.

Checkpoint Adapter Checklist

For pre-quantized checkpoints:

• Inspect transformer/config.json first.
• Check quant_method, method, producer.name, quant_algo, serialized flags, ignored_layers, and any method-specific fields.
• Verify tensor name mapping before generation.
• Verify packed modules such as to_qkv, add_kv_proj, and w13.
• Confirm scale tensor names and shapes.
• Allow dequantization back to BF16 for intentionally skipped target layers.

Sensitive Layer Discovery

Start conservative. Quantize the backbone first and keep output-side or routing layers BF16 until quality is proven.

Frequently sensitive areas:

• attention output projections such as to_out
• FFN down projections such as w2
• final projections such as proj_out, final_layer, and output heads
• modulation, timestep, position, image, or text embedders
• refiner blocks
• MoE routers, experts selected by routing, and layernorm-adjacent modules
• VAE, text/vision towers, audio encoders, and codec/talker stages

Do not reuse one model family's ignore list blindly.

Tests and Evidence

Minimum evidence for a PR:

• loader/config test or smoke command
• fixed-seed BF16 vs quantized output comparison
• quality metrics and saved outputs for precision-changing work
• memory and latency comparison under identical execution mode
• failure behavior for unsupported models or missing checkpoint fields

When the PR adds a method to a support table, include the exact model, hardware, command, quality threshold, and artifact path used for validation.