docs: organize offloader feature designs
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
This commit is contained in:
1 parent
8b61499662
commit
a52c19ea8b
14 files changed
+29
-25
No files matched your search
@@ -5,7 +5,7 @@ the coarsest strategy that meets your memory target: finer-grained streaming
|
||||
saves more device memory but adds more transfer and setup overhead.
|
||||
|
||||
For the shared factory, discovery, and lifecycle contract, see the
|
||||
[CPU Offloading design](../../design/feature/cpu_offloading.md).
|
||||
[CPU Offloading design](../../design/feature/offloader/README.md).
|
||||
|
||||
## Choose a strategy
|
||||
|
||||
|
||||
@@ -77,5 +77,5 @@ while reusing sequential `.to()` movers.
|
||||
- Higher cold-start latency.
|
||||
- Transfers between encoder and denoising phases add latency.
|
||||
|
||||
See the [model-level design](../../../design/feature/model_level_offload.md)
|
||||
See the [model-level design](../../../design/feature/offloader/cpu_offload.md)
|
||||
for lifecycle and extension invariants.
|
||||
@@ -5,7 +5,7 @@ deployments. With AllGather enabled, each rank stores roughly `1 / dp_size` of
|
||||
the host weights and reconstructs each layer at runtime. Without AllGather,
|
||||
each rank streams its standard-loader rank-local weights independently.
|
||||
|
||||
See the [DLO feature design](../../../design/feature/distributed_layerwise_offload.md)
|
||||
See the [DLO feature design](../../../design/feature/offloader/distributed_layerwise_offload.md)
|
||||
for the implementation contract and compatibility matrix.
|
||||
|
||||
## Execution model
|
||||
|
||||
@@ -53,7 +53,7 @@ class Flux2Transformer2DModel(nn.Module):
|
||||
]
|
||||
```
|
||||
|
||||
See the [layerwise design](../../../design/feature/layerwise_offload.md) for
|
||||
See the [layerwise design](../../../design/feature/offloader/layerwise_offload.md) for
|
||||
the discovery and hook invariants. `OffloadPlan` is a separate declarative
|
||||
topology path for distributed layerwise offload.
|
||||
|
||||
|
||||
Reference in new issue
Block a user