[Diffusion] Add final-layout BF16 Host Weight Runtime artifacts (#6445)
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
This commit is contained in:
1 parent
3781214675
commit
493fc64397
13 files changed
+2949
-5
No files matched your search
@@ -15,8 +15,11 @@ specified in the [Host Weight Runtime module design](../module/host_weight_runti
|
||||
## Status
|
||||
|
||||
The first implementation provides contracts and a CPU local-filesystem store.
|
||||
It does not enable cached loading for a model by itself. A consumer must still
|
||||
provide an exact identity, a representation producer, and a model restorer.
|
||||
The initial diffusion consumer contract additionally defines typed,
|
||||
representation-independent final-layout identity/restoration mechanics plus a
|
||||
concrete BF16-with-preserved-FP32 policy for MiniMax H3. It remains
|
||||
library-only: no loader or DLO path selects, publishes, restores, or transports
|
||||
that artifact yet.
|
||||
|
||||
V1 includes:
|
||||
|
||||
@@ -30,8 +33,9 @@ V1 includes:
|
||||
|
||||
V1 does not include:
|
||||
|
||||
- a public CLI or default-on model integration;
|
||||
- a generic BF16, FP8, quantized, or model-specific producer;
|
||||
- a public CLI, loader activation, or default-on model integration;
|
||||
- lease handoff to DLO or another transport consumer;
|
||||
- FP8, quantized, merged-adaptation, or additional model producers;
|
||||
- CUDA registration, pinned staging, H2D scheduling, or GPU kernels;
|
||||
- a remote artifact provider or cross-node coordination;
|
||||
- automatic eviction; or
|
||||
@@ -216,6 +220,56 @@ Dynamic LoRA overlays are not part of a reusable base-weight artifact. A
|
||||
statically merged adapter is cacheable only as a separate identity containing
|
||||
the adapter fingerprint and merge semantics.
|
||||
|
||||
### Initial diffusion final-layout contract
|
||||
|
||||
The shared diffusion contract covers complete final-layout DiT parameters and
|
||||
persistent buffers. Text encoders, VAEs, non-persistent derived state, and other
|
||||
pipeline components remain outside the artifact. One explicit representation
|
||||
policy selects allowed dtypes, tensor roles, physical layout identity, producer
|
||||
ABI, manifest schema, and restoration schema.
|
||||
|
||||
The contract is intentionally separate from loader activation:
|
||||
|
||||
- `FinalLayoutRequest` contains typed loader identity/configuration fingerprints,
|
||||
TP coordinate, and conservative SP semantics. It has no open metadata bag,
|
||||
DP coordinate, SP rank, device identity, DLO transfer mode, registration
|
||||
policy, or store path.
|
||||
- `FinalLayoutArtifactSpec` binds one `WeightRepresentation` and runtime-layout
|
||||
name to explicit producer/restorer schemas and a canonical, versioned
|
||||
implementation ABI descriptor. Compatibility never depends on reflective
|
||||
source inspection.
|
||||
- `PreparedWeightSource` snapshots immutable revisions or exact local file
|
||||
content plus a typed checkpoint-adapter identity before ordinary
|
||||
materialization. Source replacement before or during production fails
|
||||
publication. A hash-looking symlink basename is trusted only for an explicit
|
||||
Hugging Face Hub source whose repository ID and
|
||||
`models--.../snapshots/<revision> -> blobs/<hash>` topology validate; every
|
||||
local or otherwise unverified symlink target is content-hashed.
|
||||
- the tensor ownership digest records exact runtime names, kinds, shapes,
|
||||
semantic roles, dtypes, and strides from a CPU or meta model skeleton;
|
||||
- `FinalLayoutTensorRestorer` accepts only an exact lease identity, validates
|
||||
complete policy-defined coverage without mutation, and returns a one-shot
|
||||
commit plan;
|
||||
- each model declares one dtype-neutral `FinalLayoutModelContract` with an
|
||||
explicit implementation version and a post-commit validator; and
|
||||
- `FinalLayoutBF16Producer` accepts only the matching identity context and a
|
||||
finalized CPU model. It is `POST_LOAD_ONLY` and `SINGLE_PROCESS` per exact TP
|
||||
coordinate. Its BF16 policy preserves model-declared FP32 parameters and
|
||||
buffers and revalidates MiniMax H3 mixed-precision invariants.
|
||||
|
||||
Other representations reuse source identity, typed parallel identity, tensor
|
||||
ownership, and exact restoration only when their policy proves those semantics.
|
||||
For example, runtime FP8 needs a separate policy/producer for generated scales,
|
||||
quantization metadata, and Cutlass physical layouts; it is not enabled by
|
||||
changing a dtype string on the BF16 producer.
|
||||
|
||||
This stage makes no startup, sharing, or DLO performance claim. A following
|
||||
consumer PR owns disabled/preferred/required precedence, mixed-component loader
|
||||
transactions, warm-hit restoration, and transactional lease handoff. A TP2
|
||||
prewarm deployment will require a matching TP2 producer cohort to populate both
|
||||
TP-coordinate identities even though the store coordinates each artifact
|
||||
independently.
|
||||
|
||||
## Host sharing and GPU transport
|
||||
|
||||
Every process receives its own virtual mappings and `HostWeightLease`, but
|
||||
|
||||
Reference in new issue
Block a user