docs: rename offloader guides
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
This commit is contained in:
1 parent
7872663bf0
commit
8b61499662
10 files changed
+19
-19
No files matched your search
+3
-3
@@ -62,9 +62,9 @@ nav:
|
||||
- Feature Compatibility: user_guide/feature_compatibility.md
|
||||
- CPU Offloading:
|
||||
- Overview: user_guide/diffusion/cpu_offload.md
|
||||
- Model-Level: user_guide/diffusion/cpu_offload/model_level.md
|
||||
- Layerwise: user_guide/diffusion/cpu_offload/layerwise.md
|
||||
- Distributed Layerwise: user_guide/diffusion/cpu_offload/distributed_layerwise.md
|
||||
- Model-Level: user_guide/diffusion/offloader/cpu_offload.md
|
||||
- Layerwise: user_guide/diffusion/offloader/layerwise_offload.md
|
||||
- Distributed Layerwise: user_guide/diffusion/offloader/distributed_layerwise_offload.md
|
||||
- Cache Acceleration:
|
||||
- TeaCache: user_guide/diffusion/cache_acceleration/teacache.md
|
||||
- Cache-DiT: user_guide/diffusion/cache_acceleration/cache_dit.md
|
||||
|
||||
@@ -5,9 +5,9 @@ The three strategies have separate user and design pages:
|
||||
|
||||
| Strategy | User guide | Design contract |
|
||||
| --- | --- | --- |
|
||||
| Model-level | [Guide](../../user_guide/diffusion/cpu_offload/model_level.md) | [Design](model_level_offload.md) |
|
||||
| Layerwise | [Guide](../../user_guide/diffusion/cpu_offload/layerwise.md) | [Design](layerwise_offload.md) |
|
||||
| Distributed layerwise | [Guide](../../user_guide/diffusion/cpu_offload/distributed_layerwise.md) | [Design](distributed_layerwise_offload.md) |
|
||||
| Model-level | [Guide](../../user_guide/diffusion/offloader/cpu_offload.md) | [Design](model_level_offload.md) |
|
||||
| Layerwise | [Guide](../../user_guide/diffusion/offloader/layerwise_offload.md) | [Design](layerwise_offload.md) |
|
||||
| Distributed layerwise | [Guide](../../user_guide/diffusion/offloader/distributed_layerwise_offload.md) | [Design](distributed_layerwise_offload.md) |
|
||||
|
||||
## Strategy selection
|
||||
|
||||
|
||||
@@ -7,7 +7,7 @@ either shard those host-side weights across an existing parallel group or keep
|
||||
the standard loader's rank-local weights and avoid an additional collective.
|
||||
|
||||
For user-facing commands, see the
|
||||
[distributed layerwise offloading guide](../../user_guide/diffusion/cpu_offload/distributed_layerwise.md)
|
||||
[distributed layerwise offloading guide](../../user_guide/diffusion/offloader/distributed_layerwise_offload.md)
|
||||
and the [Cosmos3 recipe](https://github.com/vllm-project/vllm-omni/blob/main/recipes/cosmos3/Cosmos3-DistOffload.md).
|
||||
|
||||
## Status
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
Layerwise offload streams transformer blocks through one accelerator. For
|
||||
commands and model support, see the
|
||||
[layerwise user guide](../../user_guide/diffusion/cpu_offload/layerwise.md).
|
||||
[layerwise user guide](../../user_guide/diffusion/offloader/layerwise_offload.md).
|
||||
|
||||
## Hook ring
|
||||
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
Model-level offload enforces mutual exclusion between pipeline component
|
||||
groups. For commands and model support, see the
|
||||
[model-level user guide](../../user_guide/diffusion/cpu_offload/model_level.md).
|
||||
[model-level user guide](../../user_guide/diffusion/offloader/cpu_offload.md).
|
||||
|
||||
## Generic hook path
|
||||
|
||||
|
||||
@@ -11,9 +11,9 @@ For the shared factory, discovery, and lifecycle contract, see the
|
||||
|
||||
| Strategy | Device residency | Parallel scope | Guide |
|
||||
| --- | --- | --- | --- |
|
||||
| Model-level (sequential) | One pipeline component group at a time | Single device | [Model-Level Offloading](cpu_offload/model_level.md) |
|
||||
| Layerwise (blockwise) | One transformer block, with next-block prefetch | Single device | [Layerwise Offloading](cpu_offload/layerwise.md) |
|
||||
| Distributed layerwise | Fixed two-block device buffer; optional host-weight sharding and AllGather | Multiple GPU/NPU ranks | [Distributed Layerwise Offloading](cpu_offload/distributed_layerwise.md) |
|
||||
| Model-level (sequential) | One pipeline component group at a time | Single device | [Model-Level Offloading](offloader/cpu_offload.md) |
|
||||
| Layerwise (blockwise) | One transformer block, with next-block prefetch | Single device | [Layerwise Offloading](offloader/layerwise_offload.md) |
|
||||
| Distributed layerwise | Fixed two-block device buffer; optional host-weight sharding and AllGather | Multiple GPU/NPU ranks | [Distributed Layerwise Offloading](offloader/distributed_layerwise_offload.md) |
|
||||
|
||||
All strategies use pinned host memory for faster transfers where applicable.
|
||||
Configuration priority is:
|
||||
@@ -27,12 +27,12 @@ higher-priority strategy is selected.
|
||||
|
||||
## Quick selection
|
||||
|
||||
- Use [model-level offloading](cpu_offload/model_level.md) when swapping whole
|
||||
- Use [model-level offloading](offloader/cpu_offload.md) when swapping whole
|
||||
encoders and DiTs is enough to fit and phase-boundary transfers are
|
||||
acceptable.
|
||||
- Use [layerwise offloading](cpu_offload/layerwise.md) for compute-heavy video
|
||||
- Use [layerwise offloading](offloader/layerwise_offload.md) for compute-heavy video
|
||||
DiTs where block transfers can overlap computation.
|
||||
- Use [distributed layerwise offloading](cpu_offload/distributed_layerwise.md)
|
||||
- Use [distributed layerwise offloading](offloader/distributed_layerwise_offload.md)
|
||||
when a multi-rank deployment also needs bounded device residency and,
|
||||
optionally, sharded host weights.
|
||||
|
||||
@@ -64,12 +64,12 @@ These headings preserve links to sections that moved into dedicated guides.
|
||||
|
||||
## Model-level (Sequential) Offloading
|
||||
|
||||
See [Model-Level Offloading](cpu_offload/model_level.md).
|
||||
See [Model-Level Offloading](offloader/cpu_offload.md).
|
||||
|
||||
## Layerwise (Blockwise) Offloading
|
||||
|
||||
See [Layerwise Offloading](cpu_offload/layerwise.md).
|
||||
See [Layerwise Offloading](offloader/layerwise_offload.md).
|
||||
|
||||
## Distributed Layerwise Offloading
|
||||
|
||||
See [Distributed Layerwise Offloading](cpu_offload/distributed_layerwise.md).
|
||||
See [Distributed Layerwise Offloading](offloader/distributed_layerwise_offload.md).
|
||||
File renamed without changes.
File renamed without changes.
File renamed without changes.
@@ -217,7 +217,7 @@ The following tables show which models support each feature:
|
||||
3. CPU Offloading (Layerwise) and CPU Offloading (Module-wise) are not compatible.
|
||||
4. The CPU Offloading (Layerwise) row describes local layerwise offload.
|
||||
Multi-device Distributed Layerwise Offload has a separate topology and
|
||||
compatibility matrix in the [Distributed Layerwise Offloading guide](diffusion/cpu_offload/distributed_layerwise.md).
|
||||
compatibility matrix in the [Distributed Layerwise Offloading guide](diffusion/offloader/distributed_layerwise_offload.md).
|
||||
5. The compatibility matrix uses FP8 as the representative quantization method.
|
||||
6. Step Execution is not compatible with any diffusion cache backend. LoRA is supported, but each scheduled batch must use a single adapter (requests with different `lora_request` or `lora_scale` are kept in separate batches).
|
||||
|
||||
|
||||
Reference in new issue
Block a user