docs: organize offloader feature designs
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
This commit is contained in:
1 parent
8b61499662
commit
a52c19ea8b
14 files changed
+29
-25
No files matched your search
@@ -406,7 +406,7 @@ the source for configuration, compatibility, and implementation details.
|
||||
stage contract.
|
||||
* **Quantization and memory efficiency:** [Quantization](feature/quantization.md)
|
||||
resolves per-pipeline or per-component quantization configurations, while
|
||||
[Distributed Layerwise Offload](feature/distributed_layerwise_offload.md)
|
||||
[Distributed Layerwise Offload](feature/offloader/distributed_layerwise_offload.md)
|
||||
streams diffusion blocks from host memory within the existing parallel
|
||||
topology.
|
||||
|
||||
|
||||
@@ -5,9 +5,9 @@ The three strategies have separate user and design pages:
|
||||
|
||||
| Strategy | User guide | Design contract |
|
||||
| --- | --- | --- |
|
||||
| Model-level | [Guide](../../user_guide/diffusion/offloader/cpu_offload.md) | [Design](model_level_offload.md) |
|
||||
| Layerwise | [Guide](../../user_guide/diffusion/offloader/layerwise_offload.md) | [Design](layerwise_offload.md) |
|
||||
| Distributed layerwise | [Guide](../../user_guide/diffusion/offloader/distributed_layerwise_offload.md) | [Design](distributed_layerwise_offload.md) |
|
||||
| Model-level | [Guide](../../../user_guide/diffusion/offloader/cpu_offload.md) | [Design](cpu_offload.md) |
|
||||
| Layerwise | [Guide](../../../user_guide/diffusion/offloader/layerwise_offload.md) | [Design](layerwise_offload.md) |
|
||||
| Distributed layerwise | [Guide](../../../user_guide/diffusion/offloader/distributed_layerwise_offload.md) | [Design](distributed_layerwise_offload.md) |
|
||||
|
||||
## Strategy selection
|
||||
|
||||
@@ -76,5 +76,5 @@ ownership.
|
||||
- Platform streams, events, synchronization, and cache management go through
|
||||
the vLLM-Omni platform abstraction.
|
||||
|
||||
The diffusion [Offloader module design](../module/diffusion/offloader.md)
|
||||
The diffusion [Offloader module design](../../module/diffusion/offloader.md)
|
||||
describes how these feature contracts fit into the larger diffusion runtime.
|
||||
+2
-2
@@ -2,7 +2,7 @@
|
||||
|
||||
Model-level offload enforces mutual exclusion between pipeline component
|
||||
groups. For commands and model support, see the
|
||||
[model-level user guide](../../user_guide/diffusion/offloader/cpu_offload.md).
|
||||
[model-level user guide](../../../user_guide/diffusion/offloader/cpu_offload.md).
|
||||
|
||||
## Generic hook path
|
||||
|
||||
@@ -53,4 +53,4 @@ attribute-name heuristics.
|
||||
residency is restored.
|
||||
|
||||
Shared selection and lifecycle behavior is defined in the
|
||||
[CPU Offloading design](cpu_offloading.md).
|
||||
[CPU Offloading design](README.md).
|
||||
+2
-2
@@ -7,7 +7,7 @@ either shard those host-side weights across an existing parallel group or keep
|
||||
the standard loader's rank-local weights and avoid an additional collective.
|
||||
|
||||
For user-facing commands, see the
|
||||
[distributed layerwise offloading guide](../../user_guide/diffusion/offloader/distributed_layerwise_offload.md)
|
||||
[distributed layerwise offloading guide](../../../user_guide/diffusion/offloader/distributed_layerwise_offload.md)
|
||||
and the [Cosmos3 recipe](https://github.com/vllm-project/vllm-omni/blob/main/recipes/cosmos3/Cosmos3-DistOffload.md).
|
||||
|
||||
## Status
|
||||
@@ -56,7 +56,7 @@ AllGather: [full N+1] [full N+2]
|
||||
Buffers: [current slot] [prefetch slot] [current slot]
|
||||
```
|
||||
|
||||

|
||||

|
||||
|
||||
The backend uses two shared device buffers, so accelerator weight residency is
|
||||
bounded by the largest streamed blocks rather than the complete model.
|
||||
+2
-2
@@ -2,7 +2,7 @@
|
||||
|
||||
Layerwise offload streams transformer blocks through one accelerator. For
|
||||
commands and model support, see the
|
||||
[layerwise user guide](../../user_guide/diffusion/offloader/layerwise_offload.md).
|
||||
[layerwise user guide](../../../user_guide/diffusion/offloader/layerwise_offload.md).
|
||||
|
||||
## Hook ring
|
||||
|
||||
@@ -54,4 +54,4 @@ streamed by this backend.
|
||||
[Distributed Layerwise Offload](distributed_layerwise_offload.md).
|
||||
|
||||
Shared selection and lifecycle behavior is defined in the
|
||||
[CPU Offloading design](cpu_offloading.md).
|
||||
[CPU Offloading design](README.md).
|
||||
@@ -186,7 +186,7 @@ Distributed layerwise offload has an additional boundary: its sharded mmap
|
||||
AllGather path is not compatible with online quantization because the path
|
||||
expects already materialized checkpoint weights. Use the rank-local
|
||||
`--dlo-no-use-allgather` path or a compatible pre-quantized checkpoint when
|
||||
combining these features. See the [DLO design document](distributed_layerwise_offload.md)
|
||||
combining these features. See the [DLO design document](offloader/distributed_layerwise_offload.md)
|
||||
for the full compatibility matrix.
|
||||
|
||||
## Adding a quantization backend
|
||||
|
||||
@@ -61,10 +61,10 @@ The design contracts separate selection mechanics from backend algorithms:
|
||||
|
||||
#### CPU offloading
|
||||
|
||||
- [Overview and Shared Contracts](feature/cpu_offloading.md)
|
||||
- [Model-Level Offload](feature/model_level_offload.md)
|
||||
- [Layerwise Offload](feature/layerwise_offload.md)
|
||||
- [Distributed Layerwise Offload](feature/distributed_layerwise_offload.md)
|
||||
- [Overview and Shared Contracts](feature/offloader/README.md)
|
||||
- [Model-Level Offload](feature/offloader/cpu_offload.md)
|
||||
- [Layerwise Offload](feature/offloader/layerwise_offload.md)
|
||||
- [Distributed Layerwise Offload](feature/offloader/distributed_layerwise_offload.md)
|
||||
|
||||
- [Cache-DiT](feature/cache_dit.md)
|
||||
- [TeaCache](feature/teacache.md)
|
||||
|
||||
Reference in new issue
Block a user