[Quantization] Migrate GGUF diffusion model support to OOT plugin (#4769)

Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
This commit is contained in:
Isotr0py authored and GitHub committed 2026-07-11 18:48:43 +00:00
1 parent ff6f0da271
commit b95905deb1
35 files changed
+272 -1614

No files matched your search

+9
View File
@@ -9,6 +9,15 @@ for the transformer.
GGUF is static quantization: the quantized weights are produced before serving.
!!! note
GGUF support has migrated to OOT [vllm-gguf-plugin](https://github.com/vllm-project/vllm-gguf-plugin). Make sure you have GGUF plugin installed before serving a GGUF model.
Before serving a GGUF model, make sure to install the [vllm-gguf-plugin](https://github.com/vllm-project/vllm-gguf-plugin):
```bash
uv pip install vllm-gguf-plugin>=0.0.3
```
## Hardware Support
| Device | Support |