[Docs] Split diffusion attention and CPU offload guides (#6075)

Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
This commit is contained in:
Hongsheng Liu authored and GitHub committed 2026-08-15 14:41:09 +08:00
1 parent 6f36fd3b78
commit 1260ceaae6
24 files changed
+1145 -804

No files matched your search

@@ -0,0 +1,32 @@
# Hugging Face Hub Attention Backends
Hub backends load the same published kernels commonly used with Hugging Face
Diffusers. They are useful when training and serving must use matching kernel
implementations to reduce numerical drift and sampling divergence.
| Backend | Kernel | Platform |
| --- | --- | --- |
| `FLASH_ATTN_HUB` | `kernels-community/flash-attn2` | Compatible CUDA GPUs |
| `FLASH_ATTN_3_HUB` | `kernels-community/flash-attn3` | Hopper sm_90 or newer |
## Installation
```bash
pip install kernels==0.14.1
```
If `kernels` is unavailable, vLLM-Omni warns and falls back to the
corresponding local FlashAttention implementation. On CUDA devices older than
Hopper, `FLASH_ATTN_3_HUB` falls back to `FLASH_ATTN_HUB`.
## Usage
```bash
export DIFFUSION_ATTENTION_BACKEND=FLASH_ATTN_HUB
# Equivalent CLI selection
vllm-omni serve <model> --diffusion-attention-backend FLASH_ATTN_HUB
```
For per-role kernel alignment, configure the Hub backend on only the relevant
role; see the [attention backend overview](../attention_backends.md#per-role-configuration).