docs: split diffusion feature guides
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
This commit is contained in:
1 parent
8b0a6779d4
commit
7872663bf0
21 files changed
+1138
-801
No files matched your search
@@ -0,0 +1,32 @@
|
||||
# Hugging Face Hub Attention Backends
|
||||
|
||||
Hub backends load the same published kernels commonly used with Hugging Face
|
||||
Diffusers. They are useful when training and serving must use matching kernel
|
||||
implementations to reduce numerical drift and sampling divergence.
|
||||
|
||||
| Backend | Kernel | Platform |
|
||||
| --- | --- | --- |
|
||||
| `FLASH_ATTN_HUB` | `kernels-community/flash-attn2` | Compatible CUDA GPUs |
|
||||
| `FLASH_ATTN_3_HUB` | `kernels-community/flash-attn3` | Hopper sm_90 or newer |
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install kernels==0.14.1
|
||||
```
|
||||
|
||||
If `kernels` is unavailable, vLLM-Omni warns and falls back to the
|
||||
corresponding local FlashAttention implementation. On CUDA devices older than
|
||||
Hopper, `FLASH_ATTN_3_HUB` falls back to `FLASH_ATTN_HUB`.
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
export DIFFUSION_ATTENTION_BACKEND=FLASH_ATTN_HUB
|
||||
|
||||
# Equivalent CLI selection
|
||||
vllm-omni serve <model> --diffusion-attention-backend FLASH_ATTN_HUB
|
||||
```
|
||||
|
||||
For per-role kernel alignment, configure the Hub backend on only the relevant
|
||||
role; see the [attention backend overview](../attention_backends.md#per-role-configuration).
|
||||
Reference in new issue
Block a user