[Diffusion] Add final-layout BF16 Host Weight Runtime artifacts (#6445)

Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
This commit is contained in:
Hongsheng Liu authored and GitHub committed 2026-08-22 21:37:20 +08:00
1 parent 3781214675
commit 493fc64397
13 files changed
+2949 -5

No files matched your search

+58 -4
View File
@@ -15,8 +15,11 @@ specified in the [Host Weight Runtime module design](../module/host_weight_runti
## Status
The first implementation provides contracts and a CPU local-filesystem store.
It does not enable cached loading for a model by itself. A consumer must still
provide an exact identity, a representation producer, and a model restorer.
The initial diffusion consumer contract additionally defines typed,
representation-independent final-layout identity/restoration mechanics plus a
concrete BF16-with-preserved-FP32 policy for MiniMax H3. It remains
library-only: no loader or DLO path selects, publishes, restores, or transports
that artifact yet.
V1 includes:
@@ -30,8 +33,9 @@ V1 includes:
V1 does not include:
- a public CLI or default-on model integration;
- a generic BF16, FP8, quantized, or model-specific producer;
- a public CLI, loader activation, or default-on model integration;
- lease handoff to DLO or another transport consumer;
- FP8, quantized, merged-adaptation, or additional model producers;
- CUDA registration, pinned staging, H2D scheduling, or GPU kernels;
- a remote artifact provider or cross-node coordination;
- automatic eviction; or
@@ -216,6 +220,56 @@ Dynamic LoRA overlays are not part of a reusable base-weight artifact. A
statically merged adapter is cacheable only as a separate identity containing
the adapter fingerprint and merge semantics.
### Initial diffusion final-layout contract
The shared diffusion contract covers complete final-layout DiT parameters and
persistent buffers. Text encoders, VAEs, non-persistent derived state, and other
pipeline components remain outside the artifact. One explicit representation
policy selects allowed dtypes, tensor roles, physical layout identity, producer
ABI, manifest schema, and restoration schema.
The contract is intentionally separate from loader activation:
- `FinalLayoutRequest` contains typed loader identity/configuration fingerprints,
TP coordinate, and conservative SP semantics. It has no open metadata bag,
DP coordinate, SP rank, device identity, DLO transfer mode, registration
policy, or store path.
- `FinalLayoutArtifactSpec` binds one `WeightRepresentation` and runtime-layout
name to explicit producer/restorer schemas and a canonical, versioned
implementation ABI descriptor. Compatibility never depends on reflective
source inspection.
- `PreparedWeightSource` snapshots immutable revisions or exact local file
content plus a typed checkpoint-adapter identity before ordinary
materialization. Source replacement before or during production fails
publication. A hash-looking symlink basename is trusted only for an explicit
Hugging Face Hub source whose repository ID and
`models--.../snapshots/<revision> -> blobs/<hash>` topology validate; every
local or otherwise unverified symlink target is content-hashed.
- the tensor ownership digest records exact runtime names, kinds, shapes,
semantic roles, dtypes, and strides from a CPU or meta model skeleton;
- `FinalLayoutTensorRestorer` accepts only an exact lease identity, validates
complete policy-defined coverage without mutation, and returns a one-shot
commit plan;
- each model declares one dtype-neutral `FinalLayoutModelContract` with an
explicit implementation version and a post-commit validator; and
- `FinalLayoutBF16Producer` accepts only the matching identity context and a
finalized CPU model. It is `POST_LOAD_ONLY` and `SINGLE_PROCESS` per exact TP
coordinate. Its BF16 policy preserves model-declared FP32 parameters and
buffers and revalidates MiniMax H3 mixed-precision invariants.
Other representations reuse source identity, typed parallel identity, tensor
ownership, and exact restoration only when their policy proves those semantics.
For example, runtime FP8 needs a separate policy/producer for generated scales,
quantization metadata, and Cutlass physical layouts; it is not enabled by
changing a dtype string on the BF16 producer.
This stage makes no startup, sharing, or DLO performance claim. A following
consumer PR owns disabled/preferred/required precedence, mixed-component loader
transactions, warm-hit restoration, and transactional lease handoff. A TP2
prewarm deployment will require a matching TP2 producer cohort to populate both
TP-coordinate identities even though the store coordinates each artifact
independently.
## Host sharing and GPU transport
Every process receives its own virtual mappings and `HostWeightLease`, but