[Diffusion] Add Diffusers quantization config bridge (#3837)
Signed-off-by: CAICAIIs <3360776475@qq.com>
This commit is contained in:
1 parent
06329c6450
commit
b443504d61
7 files changed
+816
-9
No files matched your search
@@ -9,7 +9,8 @@ vLLM-Omni supports running diffusion models with the diffusers backend, directly
|
||||
|
||||
The diffusers backend is a black-box adapter. Its primary focus is to serve diffusion models online.
|
||||
Currently, the following features are NOT yet supported.
|
||||
It is not guaranteed whether they will be supported in the future.
|
||||
Community contributions are welcome to add these features when they can be
|
||||
implemented with clear Diffusers or vLLM-Omni behavior.
|
||||
|
||||
- CFG parallel execution
|
||||
- Sequence parallel execution
|
||||
@@ -66,6 +67,35 @@ This is suitable for sampling parameters not available through the vLLM-Omni int
|
||||
When a parameter is available in the vLLM-Omni interface, it will be adapted here.
|
||||
But if that parameter is simultaneously set in both the vLLM-Omni interface and `diffusers_call_kwargs`, the **former** will take precedence (because it is set at request time).
|
||||
|
||||
### Quantization
|
||||
|
||||
Use `diffusers_load_kwargs` for Diffusers-native quantization options. If
|
||||
`diffusers_load_kwargs["quantization_config"]` is provided as a dictionary, the
|
||||
diffusers backend builds a Diffusers quantization config and lets Diffusers
|
||||
validate it before calling `DiffusionPipeline.from_pretrained()`.
|
||||
|
||||
The diffusers backend also provides a small compatibility shortcut for
|
||||
vLLM-Omni quantization configs when `diffusers_load_kwargs` does not already
|
||||
contain `quantization_config`. Currently this maps online/dynamic `fp8` and
|
||||
online/dynamic `int8` to Diffusers/TorchAO dynamic quantization for transformer
|
||||
components such as `transformer` or `transformer_2`. This path requires
|
||||
`torchao` to be installed.
|
||||
|
||||
For example, the CLI can request dynamic FP8 through the vLLM-Omni interface:
|
||||
|
||||
```bash
|
||||
vllm serve "Qwen/Qwen-Image" \
|
||||
--omni \
|
||||
--diffusion-load-format diffusers \
|
||||
--quantization-config '{"method": "fp8"}'
|
||||
```
|
||||
|
||||
Other vLLM-Omni quantization methods, such as `gguf`, `modelopt`, `mxfp4`,
|
||||
`mxfp8`, serialized checkpoints, static FP8 configs, and layer-name skip lists
|
||||
such as `ignored_layers`, are intentionally not translated by this compatibility
|
||||
shortcut. Use Diffusers-native configuration through `diffusers_load_kwargs` or
|
||||
a native vLLM-Omni pipeline for those cases.
|
||||
|
||||
### Attention Backends
|
||||
|
||||
The diffusers backend converts
|
||||
|
||||
Reference in new issue
Block a user