2026-02-11 21:01:35 +08:00
|
|
|
# Quantization for Diffusion Transformers
|
|
|
|
|
|
|
|
|
|
vLLM-Omni supports quantization of DiT linear layers to reduce memory usage and accelerate inference.
|
|
|
|
|
|
|
|
|
|
## Supported Methods
|
|
|
|
|
|
|
|
|
|
| Method | Guide |
|
|
|
|
|
|--------|-------|
|
|
|
|
|
| FP8 | [FP8](fp8.md) |
|
2026-02-27 11:39:14 +08:00
|
|
|
| GGUF | [GGUF](gguf.md) |
|
2026-02-11 21:01:35 +08:00
|
|
|
|
|
|
|
|
## Device Compatibility
|
|
|
|
|
|
|
|
|
|
| GPU Generation | Example GPUs | FP8 Mode |
|
|
|
|
|
|---------------|-------------------|----------|
|
|
|
|
|
| Ada/Hopper (SM 89+) | RTX 4090, H100, H200 | Full W8A8 with native hardware |
|
|
|
|
|
|
|
|
|
|
Kernel selection is automatic.
|
