Files
vllm-omni/.claude/skills/review-pr/references/modules/quantization.md
T
2026-08-08 22:18:54 +08:00

23 lines
1.1 KiB
Markdown

# Quantization
Primary design: [Quantization](https://docs.vllm.ai/projects/vllm-omni/en/latest/design/module/quantization/).
Use for method registration, checkpoint interpretation, layer replacement,
reduced precision, platform capability, and runtime constraints.
## Contract checks
- Validate checkpoint metadata, selected method, supported layers, device
capability, required dependencies, and parallel layout before loading.
- Preserve loader and weight-name mapping semantics, dtype, scale/granularity,
zero point, padding, and buffer handling through the real kernel consumer.
- Fail clearly for unsupported combinations; do not silently choose a different
method or precision.
- Keep online and pre-quantized checkpoint paths explicit and ensure patches or
fallbacks do not bypass model-owned loaders.
- Require numerical comparison against a defined reference with explicit
tolerance before treating performance results as valid.
Test registration, loading, representative layers, metadata, platform guards,
parallel combinations, and numerical correctness separately from performance.