[Diffusion] Document TP host layout identity

Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
This commit is contained in:
Hongsheng Liu committed 2026-08-19 11:23:36 +08:00
1 parent 16642dc2e8
commit 42c6ae9007
3 files changed
+40

No files matched your search

@@ -143,6 +143,35 @@ is no-AllGather-only, and it selects the backend transfer protocol. This is
coordination through typed configuration and `HostWeightPlan`, not through
model-specific global state.
### Runtime host-layout identity (v1)
Host weight cache v1 identifies the loader-produced host representation, not
the topology used later to move weights to devices. These are separate
contracts:
- `RuntimeHostLayout` describes tensor ownership and representation after the
ordinary loader has applied its parallel-aware transforms. This is the
layout stored in the host weight cache.
- `DLOTransferLayout` describes whether each worker copies its local weights
directly or participates in the existing DLO AllGather protocol. It is not
part of the host weight cache in v1; AllGather continues to use its existing
process-owned host shards.
The v1 physical-sharing invariant is one cache entry per TP coordinate,
shared by equivalent DP and SP consumers on the same node. The identity
therefore includes TP world size and TP rank, because different TP coordinates
own different tensors. It excludes DP world size/rank and SP rank, because
those dimensions do not change the final tensor ownership supported by v1.
SP world size and implementation details remain conservative compatibility
guards until equivalent layouts across those configurations are proven.
PP, HSDP/DTensor, expert parallelism, and CFG parallelism are rejected rather
than assigned speculative cache identities. Supporting one of these layouts
requires first defining its loader-produced ownership, representation, and
sharing domain; it does not follow automatically from supporting its runtime
collectives. Consequently, this phase implements a TP-coordinate runtime host
layout, not a general parallel-layout cache.
### Shared startup and lifecycle
The shared control path selects host backing and transfers its ownership to