[Diffusion] Add final-layout BF16 Host Weight Runtime artifacts (#6445)

Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
This commit is contained in:
Hongsheng Liu authored and GitHub committed 2026-08-22 21:37:20 +08:00
1 parent 3781214675
commit 493fc64397
13 files changed
+2949 -5

No files matched your search

+58 -4
View File
@@ -15,8 +15,11 @@ specified in the [Host Weight Runtime module design](../module/host_weight_runti
## Status
The first implementation provides contracts and a CPU local-filesystem store.
It does not enable cached loading for a model by itself. A consumer must still
provide an exact identity, a representation producer, and a model restorer.
The initial diffusion consumer contract additionally defines typed,
representation-independent final-layout identity/restoration mechanics plus a
concrete BF16-with-preserved-FP32 policy for MiniMax H3. It remains
library-only: no loader or DLO path selects, publishes, restores, or transports
that artifact yet.
V1 includes:
@@ -30,8 +33,9 @@ V1 includes:
V1 does not include:
- a public CLI or default-on model integration;
- a generic BF16, FP8, quantized, or model-specific producer;
- a public CLI, loader activation, or default-on model integration;
- lease handoff to DLO or another transport consumer;
- FP8, quantized, merged-adaptation, or additional model producers;
- CUDA registration, pinned staging, H2D scheduling, or GPU kernels;
- a remote artifact provider or cross-node coordination;
- automatic eviction; or
@@ -216,6 +220,56 @@ Dynamic LoRA overlays are not part of a reusable base-weight artifact. A
statically merged adapter is cacheable only as a separate identity containing
the adapter fingerprint and merge semantics.
### Initial diffusion final-layout contract
The shared diffusion contract covers complete final-layout DiT parameters and
persistent buffers. Text encoders, VAEs, non-persistent derived state, and other
pipeline components remain outside the artifact. One explicit representation
policy selects allowed dtypes, tensor roles, physical layout identity, producer
ABI, manifest schema, and restoration schema.
The contract is intentionally separate from loader activation:
- `FinalLayoutRequest` contains typed loader identity/configuration fingerprints,
TP coordinate, and conservative SP semantics. It has no open metadata bag,
DP coordinate, SP rank, device identity, DLO transfer mode, registration
policy, or store path.
- `FinalLayoutArtifactSpec` binds one `WeightRepresentation` and runtime-layout
name to explicit producer/restorer schemas and a canonical, versioned
implementation ABI descriptor. Compatibility never depends on reflective
source inspection.
- `PreparedWeightSource` snapshots immutable revisions or exact local file
content plus a typed checkpoint-adapter identity before ordinary
materialization. Source replacement before or during production fails
publication. A hash-looking symlink basename is trusted only for an explicit
Hugging Face Hub source whose repository ID and
`models--.../snapshots/<revision> -> blobs/<hash>` topology validate; every
local or otherwise unverified symlink target is content-hashed.
- the tensor ownership digest records exact runtime names, kinds, shapes,
semantic roles, dtypes, and strides from a CPU or meta model skeleton;
- `FinalLayoutTensorRestorer` accepts only an exact lease identity, validates
complete policy-defined coverage without mutation, and returns a one-shot
commit plan;
- each model declares one dtype-neutral `FinalLayoutModelContract` with an
explicit implementation version and a post-commit validator; and
- `FinalLayoutBF16Producer` accepts only the matching identity context and a
finalized CPU model. It is `POST_LOAD_ONLY` and `SINGLE_PROCESS` per exact TP
coordinate. Its BF16 policy preserves model-declared FP32 parameters and
buffers and revalidates MiniMax H3 mixed-precision invariants.
Other representations reuse source identity, typed parallel identity, tensor
ownership, and exact restoration only when their policy proves those semantics.
For example, runtime FP8 needs a separate policy/producer for generated scales,
quantization metadata, and Cutlass physical layouts; it is not enabled by
changing a dtype string on the BF16 producer.
This stage makes no startup, sharing, or DLO performance claim. A following
consumer PR owns disabled/preferred/required precedence, mixed-component loader
transactions, warm-hit restoration, and transactional lease handoff. A TP2
prewarm deployment will require a matching TP2 producer cohort to populate both
TP-coordinate identities even though the store coordinates each artifact
independently.
## Host sharing and GPU transport
Every process receives its own virtual mappings and `HostWeightLease`, but
+2 -1
View File
@@ -56,7 +56,8 @@ The contracts have deliberately narrow ownership:
- `HostWeightRuntime` applies resolution and fallback policy and emits one
terminal resolution report. Explicit post-load publication emits a separate
report and may return a lease, but does not revise the completed resolution.
report and closes its store-returned lease before returning; it does not
revise the completed resolution.
The runtime does not own a model or call the canonical loader.
- `HostWeightStore` owns artifact lookup, coordinated construction,
publication, validation, quarantine, and lifecycle.
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,38 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""Typed diffusion adapters for runtime-ready final-layout artifacts."""
from .contracts import (
FinalLayoutArtifactSpec,
FinalLayoutLoaderIdentity,
FinalLayoutParallelIdentity,
FinalLayoutRequest,
ImplementationIdentity,
)
from .identity_adapter import (
FinalLayoutIdentityContext,
build_final_layout_identity,
)
from .producers import (
FINAL_LAYOUT_BF16_POLICY,
FINAL_LAYOUT_BF16_SPEC,
FinalLayoutBF16Producer,
)
from .restorer import FinalLayoutTensorRestorer
from .source_identity import PreparedWeightSource, WeightSourceKind
__all__ = [
"FINAL_LAYOUT_BF16_POLICY",
"FINAL_LAYOUT_BF16_SPEC",
"FinalLayoutArtifactSpec",
"FinalLayoutBF16Producer",
"FinalLayoutIdentityContext",
"FinalLayoutLoaderIdentity",
"FinalLayoutParallelIdentity",
"FinalLayoutRequest",
"FinalLayoutTensorRestorer",
"ImplementationIdentity",
"PreparedWeightSource",
"WeightSourceKind",
"build_final_layout_identity",
]
@@ -0,0 +1,324 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""Typed semantic contracts shared by final-layout artifact representations."""
from __future__ import annotations
import hashlib
from dataclasses import dataclass, field
from enum import Enum
from typing import TYPE_CHECKING, Protocol
import torch
from vllm_omni.diffusion.models.host_weight_contract import (
FINAL_LAYOUT_TENSOR_MODEL_CONTRACT_SCHEMA,
)
from vllm_omni.host_weight_runtime import (
AdaptationIdentity,
CanonicalJson,
FailureCode,
HostWeightError,
HostWeightFailure,
ProducerIdentity,
ResolutionStage,
TensorKind,
WeightRepresentation,
)
if TYPE_CHECKING:
from collections.abc import Sequence
from .tensor_layout import RuntimeTensorTarget
FINAL_LAYOUT_TENSOR_RESTORER_SCHEMA = "diffusion-final-layout-tensor-restorer-v1"
def _require_text(name: str, value: object) -> str:
if not isinstance(value, str) or not value:
raise ValueError(f"{name} must not be empty")
return value
def _require_positive_int(name: str, value: object) -> int:
if isinstance(value, bool) or not isinstance(value, int) or value < 1:
raise ValueError(f"{name} must be a positive integer")
return value
class FinalLayoutContractCode(str, Enum):
SOURCE_CHANGED = "source_changed"
SOURCE_COVERAGE_INVALID = "source_coverage_invalid"
MODEL_CONTRACT_UNSUPPORTED = "model_contract_unsupported"
OWNERSHIP_AMBIGUOUS = "ownership_ambiguous"
TENSOR_UNSUPPORTED = "tensor_unsupported"
DTYPE_UNSUPPORTED = "dtype_unsupported"
IDENTITY_INCOMPATIBLE = "identity_incompatible"
TENSOR_CONTRACT_CHANGED = "tensor_contract_changed"
class FinalLayoutContractError(ValueError):
"""Typed fail-closed error at a final-layout model integration boundary."""
def __init__(self, code: FinalLayoutContractCode, message: str) -> None:
if not isinstance(code, FinalLayoutContractCode):
raise ValueError("final-layout contract errors require FinalLayoutContractCode")
super().__init__(message)
self.code = code
def final_layout_producer_error(error: FinalLayoutContractError) -> HostWeightError:
"""Translate representation failures into the runtime's stable taxonomy."""
if error.code is FinalLayoutContractCode.SOURCE_CHANGED:
stage = ResolutionStage.CANONICAL_LOADING
code = FailureCode.CANONICAL_SOURCE_FAILED
elif error.code in {
FinalLayoutContractCode.MODEL_CONTRACT_UNSUPPORTED,
FinalLayoutContractCode.TENSOR_UNSUPPORTED,
FinalLayoutContractCode.DTYPE_UNSUPPORTED,
}:
stage = ResolutionStage.PRODUCTION
code = FailureCode.PRODUCER_UNSUPPORTED
else:
stage = ResolutionStage.PRODUCTION
code = FailureCode.PRODUCER_FAILED
return HostWeightError(
HostWeightFailure(
stage=stage,
code=code,
retryable=False,
message=str(error),
details=CanonicalJson.from_value({"final_layout_code": error.code.value}),
)
)
@dataclass(frozen=True)
class ImplementationIdentity:
"""Explicit stable identity for one semantic implementation boundary."""
implementation_id: str
version: str
fingerprint: str
def __post_init__(self) -> None:
_require_text("implementation_id", self.implementation_id)
_require_text("implementation version", self.version)
_require_text("implementation fingerprint", self.fingerprint)
def to_dict(self) -> dict[str, str]:
return {
"implementation_id": self.implementation_id,
"version": self.version,
"fingerprint": self.fingerprint,
}
@dataclass(frozen=True)
class FinalLayoutLoaderIdentity:
"""Loader ABI and exact configuration facts that may change final bytes."""
implementation: ImplementationIdentity
model_config_fingerprint: str
weight_transform_fingerprint: str
def __post_init__(self) -> None:
if not isinstance(self.implementation, ImplementationIdentity):
raise ValueError("loader implementation must use ImplementationIdentity")
_require_text("model config fingerprint", self.model_config_fingerprint)
_require_text("weight transform fingerprint", self.weight_transform_fingerprint)
def to_dict(self) -> dict[str, object]:
return {
"implementation": self.implementation.to_dict(),
"model_config_fingerprint": self.model_config_fingerprint,
"weight_transform_fingerprint": self.weight_transform_fingerprint,
}
@dataclass(frozen=True)
class FinalLayoutParallelIdentity:
"""Semantic parallel layout with DP and SP rank intentionally absent."""
tensor_parallel_size: int = 1
tensor_parallel_rank: int = 0
sequence_parallel_size: int = 1
ulysses_degree: int = 1
ring_degree: int = 1
allgather_degree: int = 1
ulysses_mode: str = "strict"
pipeline_parallel_size: int = 1
cfg_parallel_size: int = 1
use_hsdp: bool = False
enable_expert_parallel: bool = False
def __post_init__(self) -> None:
tp_size = _require_positive_int("tensor_parallel_size", self.tensor_parallel_size)
if isinstance(self.tensor_parallel_rank, bool) or not isinstance(self.tensor_parallel_rank, int):
raise ValueError("tensor_parallel_rank must be an integer")
if not 0 <= self.tensor_parallel_rank < tp_size:
raise ValueError("tensor_parallel_rank must be within tensor_parallel_size")
for name, value in (
("sequence_parallel_size", self.sequence_parallel_size),
("ulysses_degree", self.ulysses_degree),
("ring_degree", self.ring_degree),
("allgather_degree", self.allgather_degree),
("pipeline_parallel_size", self.pipeline_parallel_size),
("cfg_parallel_size", self.cfg_parallel_size),
):
_require_positive_int(name, value)
_require_text("ulysses_mode", self.ulysses_mode)
if not isinstance(self.use_hsdp, bool) or not isinstance(self.enable_expert_parallel, bool):
raise ValueError("HSDP and expert-parallel flags must be booleans")
@property
def sequence_parallel_backend(self) -> str:
if self.sequence_parallel_size == 1:
return "none"
active = [
name
for name, degree in (
("ulysses", self.ulysses_degree),
("ring", self.ring_degree),
("allgather", self.allgather_degree),
)
if degree > 1
]
return "+".join(active) if active else "replicated-sp"
def to_dict(self) -> dict[str, object]:
return {
"tensor_parallel_size": self.tensor_parallel_size,
"tensor_parallel_rank": self.tensor_parallel_rank,
"sequence_parallel_size": self.sequence_parallel_size,
"sequence_parallel_backend": self.sequence_parallel_backend,
"ulysses_degree": self.ulysses_degree,
"ring_degree": self.ring_degree,
"allgather_degree": self.allgather_degree,
"ulysses_mode": self.ulysses_mode,
"pipeline_parallel_size": self.pipeline_parallel_size,
"cfg_parallel_size": self.cfg_parallel_size,
"use_hsdp": self.use_hsdp,
"enable_expert_parallel": self.enable_expert_parallel,
}
@dataclass(frozen=True)
class FinalLayoutRequest:
"""Representation-independent semantic request assembled by the loader."""
model_id: str
loader: FinalLayoutLoaderIdentity
parallel: FinalLayoutParallelIdentity = field(default_factory=FinalLayoutParallelIdentity)
load_format: str = "default"
adaptation: AdaptationIdentity = field(default_factory=AdaptationIdentity)
def __post_init__(self) -> None:
_require_text("canonical model ID or path", self.model_id)
if not isinstance(self.loader, FinalLayoutLoaderIdentity):
raise ValueError("loader must use FinalLayoutLoaderIdentity")
if not isinstance(self.parallel, FinalLayoutParallelIdentity):
raise ValueError("parallel must use FinalLayoutParallelIdentity")
_require_text("load_format", self.load_format)
if not isinstance(self.adaptation, AdaptationIdentity):
raise ValueError("adaptation must use AdaptationIdentity")
def implementation_abi_fingerprint(abi: CanonicalJson) -> str:
if not isinstance(abi, CanonicalJson):
raise ValueError("implementation ABI must use CanonicalJson")
return hashlib.sha256(abi.encoded).hexdigest()
@dataclass(frozen=True)
class FinalLayoutArtifactSpec:
"""Concrete representation, layout, producer, and restoration ABI."""
representation: WeightRepresentation
producer: ProducerIdentity
implementation_abi: CanonicalJson
layout_name: str = "diffusion-final-module-layout-v1"
component_name: str = "diffusion-dit"
component_ownership: str = "complete-final-layout-tensors"
model_contract_schema: str = FINAL_LAYOUT_TENSOR_MODEL_CONTRACT_SCHEMA
def __post_init__(self) -> None:
if not isinstance(self.representation, WeightRepresentation):
raise ValueError("artifact representation must use WeightRepresentation")
if not isinstance(self.producer, ProducerIdentity):
raise ValueError("artifact producer must use ProducerIdentity")
if not isinstance(self.implementation_abi, CanonicalJson):
raise ValueError("artifact implementation ABI must use CanonicalJson")
for name, value in (
("layout_name", self.layout_name),
("component_name", self.component_name),
("component_ownership", self.component_ownership),
("model_contract_schema", self.model_contract_schema),
):
_require_text(name, value)
abi = self.implementation_abi.to_value()
required_abi_fields = {
"artifact_identity",
"model_contract",
"producer",
"representation_policy",
"restorer",
"source_identity",
"tensor_contract",
}
if not isinstance(abi, dict) or not required_abi_fields <= set(abi):
raise ValueError(
"artifact implementation ABI must version identity, source, tensor, policy, producer, "
"restorer, and model contracts"
)
for name in required_abi_fields:
_require_text(f"implementation ABI {name}", abi[name])
if self.producer.implementation_fingerprint != implementation_abi_fingerprint(self.implementation_abi):
raise ValueError("producer implementation fingerprint differs from its explicit ABI descriptor")
class FinalLayoutTensorPolicy(Protocol):
"""Representation-specific validation over shared final-layout mechanics."""
@property
def spec(self) -> FinalLayoutArtifactSpec: ...
def validate_request(self, request: FinalLayoutRequest) -> None: ...
def tensor_role(self, name: str, tensor: torch.Tensor, kind: TensorKind) -> str: ...
def validate_target(self, target: RuntimeTensorTarget) -> None: ...
def validate_collection(self, targets: Sequence[RuntimeTensorTarget]) -> None: ...
def build_format_metadata(
self,
*,
component_names: tuple[str, ...],
tensor_contract_digest: str,
tensor_count: int,
) -> CanonicalJson: ...
def validate_format_metadata(
self,
metadata: CanonicalJson,
*,
component_names: tuple[str, ...],
tensor_contract_digest: str,
tensor_count: int,
) -> None: ...
__all__ = [
"FinalLayoutArtifactSpec",
"FinalLayoutContractCode",
"FinalLayoutContractError",
"FinalLayoutLoaderIdentity",
"FinalLayoutParallelIdentity",
"FinalLayoutRequest",
"FinalLayoutTensorPolicy",
"FINAL_LAYOUT_TENSOR_RESTORER_SCHEMA",
"ImplementationIdentity",
"final_layout_producer_error",
"implementation_abi_fingerprint",
]
@@ -0,0 +1,257 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""Exact pre-load identity assembly for policy-defined final-layout artifacts."""
from __future__ import annotations
from collections.abc import Mapping, Sequence
from dataclasses import dataclass
from torch import nn
from vllm_omni.diffusion.models.host_weight_contract import FinalLayoutModelContract
from vllm_omni.host_weight_runtime import (
CanonicalJson,
ComponentIdentity,
RuntimeWeightLayout,
WeightArtifactIdentity,
WeightRepresentation,
)
from .contracts import (
FinalLayoutArtifactSpec,
FinalLayoutContractCode,
FinalLayoutContractError,
FinalLayoutRequest,
FinalLayoutTensorPolicy,
)
from .source_identity import (
FinalLayoutSourceContext,
PreparedWeightSource,
resolve_final_layout_source_identity,
)
from .tensor_layout import (
RuntimeTensorTarget,
collect_final_layout_targets,
tensor_contract_sha256,
validate_final_layout_model_contract,
)
@dataclass(frozen=True)
class FinalLayoutIdentityContext:
"""Exact semantic identity plus process-local transaction guards."""
identity: WeightArtifactIdentity
spec: FinalLayoutArtifactSpec
policy: FinalLayoutTensorPolicy
request: FinalLayoutRequest
tensor_names: frozenset[str]
dit_names: tuple[str, ...]
model_contracts: tuple[FinalLayoutModelContract, ...]
source: FinalLayoutSourceContext
pipeline_type: type[nn.Module]
dit_types: tuple[type[nn.Module], ...]
def sources_unchanged(self) -> bool:
return self.source.sources_unchanged()
def ensure_sources_unchanged(self) -> None:
self.source.ensure_sources_unchanged()
def _require_metadata_object(name: str, metadata: CanonicalJson) -> Mapping[str, object]:
value = metadata.to_value()
if not isinstance(value, dict):
raise ValueError(f"{name} must encode one JSON object")
return value
def _model_contract_values(contracts: Sequence[FinalLayoutModelContract]) -> list[dict[str, str]]:
return [contract.to_dict() for contract in contracts]
def build_final_layout_identity(
pipeline: nn.Module,
*,
dit_modules: Sequence[tuple[str, nn.Module]],
prepared_sources: Sequence[PreparedWeightSource],
request: FinalLayoutRequest,
policy: FinalLayoutTensorPolicy,
) -> FinalLayoutIdentityContext:
"""Build exact source, representation, layout, and implementation identity."""
if not isinstance(request, FinalLayoutRequest):
raise TypeError("final-layout identity requires FinalLayoutRequest")
policy.validate_request(request)
spec = policy.spec
bindings = validate_final_layout_model_contract(
dit_modules,
expected_schema=spec.model_contract_schema,
)
structural_targets = collect_final_layout_targets(
pipeline,
dit_modules,
policy=policy,
require_materialized=False,
)
target_names = frozenset(target.name for target in structural_targets)
source = resolve_final_layout_source_identity(
prepared_sources,
model_id=request.model_id,
target_names=target_names,
)
dit_names = tuple(name for name, _ in dit_modules)
model_contracts = tuple(binding.contract for binding in bindings)
representation = WeightRepresentation(
name=spec.representation.name,
dtype=spec.representation.dtype,
metadata=CanonicalJson.from_value(
{
"artifact_policy": _require_metadata_object(
"artifact representation metadata",
spec.representation.metadata,
),
"load_format": request.load_format,
"loader": request.loader.to_dict(),
}
),
)
identity = WeightArtifactIdentity(
schema_version=1,
source=source.identity,
component=ComponentIdentity(
name=spec.component_name,
ownership=spec.component_ownership,
metadata=CanonicalJson.from_value(
{
"component_names": list(dit_names),
"model_contracts": _model_contract_values(model_contracts),
}
),
),
representation=representation,
layout=RuntimeWeightLayout(
name=spec.layout_name,
tensor_parallel_size=request.parallel.tensor_parallel_size,
tensor_parallel_rank=request.parallel.tensor_parallel_rank,
sequence_parallel_size=request.parallel.sequence_parallel_size,
sequence_parallel_backend=request.parallel.sequence_parallel_backend,
metadata=CanonicalJson.from_value(
{
"model_contract_schema": spec.model_contract_schema,
"parallel": request.parallel.to_dict(),
"tensor_contract_sha256": tensor_contract_sha256(structural_targets),
}
),
),
adaptation=request.adaptation,
producer=spec.producer,
)
return FinalLayoutIdentityContext(
identity=identity,
spec=spec,
policy=policy,
request=request,
tensor_names=target_names,
dit_names=dit_names,
model_contracts=model_contracts,
source=source,
pipeline_type=type(pipeline),
dit_types=tuple(type(module) for _, module in dit_modules),
)
def validate_final_layout_identity(
context: FinalLayoutIdentityContext,
targets: Sequence[RuntimeTensorTarget],
) -> str:
"""Validate a context against current targets and return their digest."""
identity = context.identity
spec = context.spec
request = context.request
digest = tensor_contract_sha256(targets)
if identity.source != context.source.identity or identity.adaptation != request.adaptation:
raise FinalLayoutContractError(
FinalLayoutContractCode.IDENTITY_INCOMPATIBLE,
"identity source or adaptation differs from the exact context",
)
if identity.producer != spec.producer:
raise FinalLayoutContractError(
FinalLayoutContractCode.IDENTITY_INCOMPATIBLE,
"identity producer differs from the artifact specification",
)
if identity.representation.name != spec.representation.name or identity.representation.dtype != (
spec.representation.dtype
):
raise FinalLayoutContractError(
FinalLayoutContractCode.IDENTITY_INCOMPATIBLE,
"identity representation differs from the artifact specification",
)
if identity.component.name != spec.component_name or identity.component.ownership != spec.component_ownership:
raise FinalLayoutContractError(
FinalLayoutContractCode.IDENTITY_INCOMPATIBLE,
"identity component ownership differs from the artifact specification",
)
if identity.layout.name != spec.layout_name:
raise FinalLayoutContractError(
FinalLayoutContractCode.IDENTITY_INCOMPATIBLE,
"identity runtime layout differs from the artifact specification",
)
component = _require_metadata_object("component metadata", identity.component.metadata)
representation = _require_metadata_object("representation metadata", identity.representation.metadata)
layout = _require_metadata_object("layout metadata", identity.layout.metadata)
if representation != {
"artifact_policy": _require_metadata_object(
"artifact representation metadata",
spec.representation.metadata,
),
"load_format": request.load_format,
"loader": request.loader.to_dict(),
}:
raise FinalLayoutContractError(
FinalLayoutContractCode.IDENTITY_INCOMPATIBLE,
"identity loader or representation policy differs from the exact context",
)
if component.get("component_names") != list(context.dit_names) or component.get("model_contracts") != (
_model_contract_values(context.model_contracts)
):
raise FinalLayoutContractError(
FinalLayoutContractCode.IDENTITY_INCOMPATIBLE,
"identity model ownership differs from the exact context",
)
if layout.get("model_contract_schema") != spec.model_contract_schema:
raise FinalLayoutContractError(
FinalLayoutContractCode.IDENTITY_INCOMPATIBLE,
"identity model restore schema differs from the artifact specification",
)
if (
identity.layout.tensor_parallel_size != request.parallel.tensor_parallel_size
or identity.layout.tensor_parallel_rank != request.parallel.tensor_parallel_rank
or identity.layout.sequence_parallel_size != request.parallel.sequence_parallel_size
or identity.layout.sequence_parallel_backend != request.parallel.sequence_parallel_backend
or layout.get("parallel") != request.parallel.to_dict()
):
raise FinalLayoutContractError(
FinalLayoutContractCode.IDENTITY_INCOMPATIBLE,
"identity parallel semantics differ from the exact context",
)
if layout.get("tensor_contract_sha256") != digest:
raise FinalLayoutContractError(
FinalLayoutContractCode.TENSOR_CONTRACT_CHANGED,
"identity tensor contract differs from the current model",
)
if frozenset(target.name for target in targets) != context.tensor_names:
raise FinalLayoutContractError(
FinalLayoutContractCode.TENSOR_CONTRACT_CHANGED,
"current tensor ownership differs from the exact identity context",
)
return digest
__all__ = [
"FinalLayoutIdentityContext",
"build_final_layout_identity",
"validate_final_layout_identity",
]
@@ -0,0 +1,25 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""Diffusion Host Weight Runtime producers."""
from .final_layout_bf16 import (
FINAL_LAYOUT_BF16_MANIFEST_SCHEMA,
FINAL_LAYOUT_BF16_POLICY,
FINAL_LAYOUT_BF16_PRODUCER_ID,
FINAL_LAYOUT_BF16_REPRESENTATION,
FINAL_LAYOUT_BF16_SPEC,
FINAL_LAYOUT_BF16_VERSION,
FinalLayoutBF16Policy,
FinalLayoutBF16Producer,
)
__all__ = [
"FINAL_LAYOUT_BF16_MANIFEST_SCHEMA",
"FINAL_LAYOUT_BF16_POLICY",
"FINAL_LAYOUT_BF16_PRODUCER_ID",
"FINAL_LAYOUT_BF16_REPRESENTATION",
"FINAL_LAYOUT_BF16_SPEC",
"FINAL_LAYOUT_BF16_VERSION",
"FinalLayoutBF16Policy",
"FinalLayoutBF16Producer",
]
@@ -0,0 +1,315 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""Concrete BF16-with-preserved-FP32 final-layout artifact policy and producer."""
from __future__ import annotations
from collections.abc import Sequence
import torch
from torch import nn
from vllm_omni.diffusion.models.host_weight_contract import (
FINAL_LAYOUT_TENSOR_MODEL_CONTRACT_SCHEMA,
)
from vllm_omni.host_weight_runtime import (
ArtifactWriter,
CanonicalJson,
CoordinationScope,
LookupPhase,
ProducerIdentity,
ProductionMetadata,
ProductionSourceMode,
TensorKind,
TensorWriteSpec,
WeightProductionSpec,
WeightRepresentation,
)
from ..contracts import (
FINAL_LAYOUT_TENSOR_RESTORER_SCHEMA,
FinalLayoutArtifactSpec,
FinalLayoutContractCode,
FinalLayoutContractError,
FinalLayoutRequest,
final_layout_producer_error,
implementation_abi_fingerprint,
)
from ..identity_adapter import (
FinalLayoutIdentityContext,
validate_final_layout_identity,
)
from ..tensor_layout import (
RuntimeTensorTarget,
collect_final_layout_targets,
validate_final_layout_model_contract,
)
FINAL_LAYOUT_BF16_PRODUCER_ID = "vllm-omni.diffusion.final-layout-bf16"
FINAL_LAYOUT_BF16_VERSION = "1"
FINAL_LAYOUT_BF16_REPRESENTATION = "diffusion-final-layout-bf16"
FINAL_LAYOUT_BF16_MANIFEST_SCHEMA = "diffusion-final-layout-bf16-manifest-v1"
DEFAULT_SHARD_SIZE_BYTES = 5 * 1024**3
_BF16_PARAMETER_DTYPES = {torch.bfloat16, torch.float32}
_BF16_BUFFER_DTYPES = {
torch.bool,
torch.uint8,
torch.int8,
torch.int16,
torch.int32,
torch.int64,
torch.float16,
torch.bfloat16,
torch.float32,
torch.float64,
}
for _dtype_name in (
"float8_e4m3fn",
"float8_e5m2",
"float8_e4m3fnuz",
"float8_e5m2fnuz",
):
if (_dtype := getattr(torch, _dtype_name, None)) is not None:
_BF16_BUFFER_DTYPES.add(_dtype)
_BF16_IMPLEMENTATION_ABI = CanonicalJson.from_value(
{
"artifact_identity": "diffusion-final-layout-identity-v1",
"model_contract": FINAL_LAYOUT_TENSOR_MODEL_CONTRACT_SCHEMA,
"producer": "finalized-bf16-tensor-writer-v1",
"representation_policy": "bf16-with-preserved-fp32-v1",
"restorer": "exact-final-layout-tensor-rebind-v1",
"source_identity": "prepared-diffusion-weight-source-v2",
"tensor_contract": "complete-strided-tensor-ownership-v1",
}
)
FINAL_LAYOUT_BF16_SPEC = FinalLayoutArtifactSpec(
representation=WeightRepresentation(
name=FINAL_LAYOUT_BF16_REPRESENTATION,
dtype=str(torch.bfloat16),
metadata=CanonicalJson.from_value(
{
"format": FINAL_LAYOUT_BF16_REPRESENTATION,
"mixed_precision_policy": "bf16-with-preserved-fp32",
}
),
),
producer=ProducerIdentity(
producer_id=FINAL_LAYOUT_BF16_PRODUCER_ID,
version=FINAL_LAYOUT_BF16_VERSION,
implementation_fingerprint=implementation_abi_fingerprint(_BF16_IMPLEMENTATION_ABI),
manifest_schema=FINAL_LAYOUT_BF16_MANIFEST_SCHEMA,
restorer_schema=FINAL_LAYOUT_TENSOR_RESTORER_SCHEMA,
),
implementation_abi=_BF16_IMPLEMENTATION_ABI,
)
class FinalLayoutBF16Policy:
"""Concrete compatibility and metadata policy for finalized BF16 tensors."""
@property
def spec(self) -> FinalLayoutArtifactSpec:
return FINAL_LAYOUT_BF16_SPEC
def validate_request(self, request: FinalLayoutRequest) -> None:
if request.load_format != "default":
raise ValueError(f"final-layout BF16 artifacts require load_format='default', got {request.load_format!r}")
if request.adaptation.kind != "base" or request.adaptation.fingerprint is not None:
raise ValueError("merged or static LoRA requires a representation-specific producer")
parallel = request.parallel
if parallel.pipeline_parallel_size != 1:
raise ValueError("pipeline-parallel ownership is not supported by the final-layout BF16 producer")
if parallel.cfg_parallel_size != 1:
raise ValueError("CFG-parallel ownership is not supported by the final-layout BF16 producer")
if parallel.use_hsdp:
raise ValueError("HSDP/DTensor layouts are not supported by the final-layout BF16 producer")
if parallel.enable_expert_parallel:
raise ValueError("expert-parallel ownership is not supported by the final-layout BF16 producer")
def tensor_role(self, name: str, tensor: torch.Tensor, kind: TensorKind) -> str:
del name, tensor
if kind is TensorKind.PARAMETER:
return "weight"
return "persistent_buffer"
def validate_target(self, target: RuntimeTensorTarget) -> None:
if target.kind is TensorKind.PARAMETER and target.tensor.dtype not in _BF16_PARAMETER_DTYPES:
raise FinalLayoutContractError(
FinalLayoutContractCode.DTYPE_UNSUPPORTED,
f"{target.name!r} must be BF16 or an explicitly preserved FP32 parameter, got {target.tensor.dtype}",
)
if target.kind is TensorKind.BUFFER and target.tensor.dtype not in _BF16_BUFFER_DTYPES:
raise FinalLayoutContractError(
FinalLayoutContractCode.DTYPE_UNSUPPORTED,
f"{target.name!r} uses unsupported buffer dtype {target.tensor.dtype}",
)
def validate_collection(self, targets: Sequence[RuntimeTensorTarget]) -> None:
if not any(target.kind is TensorKind.PARAMETER and target.tensor.dtype is torch.bfloat16 for target in targets):
raise FinalLayoutContractError(
FinalLayoutContractCode.DTYPE_UNSUPPORTED,
"final-layout BF16 representation requires at least one BF16 parameter",
)
def build_format_metadata(
self,
*,
component_names: tuple[str, ...],
tensor_contract_digest: str,
tensor_count: int,
) -> CanonicalJson:
return CanonicalJson.from_value(
{
"component_names": list(component_names),
"format": FINAL_LAYOUT_BF16_REPRESENTATION,
"mixed_precision_policy": "bf16-with-preserved-fp32",
"tensor_contract_sha256": tensor_contract_digest,
"tensor_count": tensor_count,
"tensor_layout": "contiguous",
}
)
def validate_format_metadata(
self,
metadata: CanonicalJson,
*,
component_names: tuple[str, ...],
tensor_contract_digest: str,
tensor_count: int,
) -> None:
expected = self.build_format_metadata(
component_names=component_names,
tensor_contract_digest=tensor_contract_digest,
tensor_count=tensor_count,
)
if metadata != expected:
raise ValueError("lease format metadata differs from the BF16 artifact policy")
FINAL_LAYOUT_BF16_POLICY = FinalLayoutBF16Policy()
def _split_shards(
records: Sequence[RuntimeTensorTarget],
max_shard_bytes: int,
) -> tuple[tuple[RuntimeTensorTarget, ...], ...]:
if max_shard_bytes <= 0:
raise ValueError("final-layout BF16 shard size must be positive")
shards: list[tuple[RuntimeTensorTarget, ...]] = []
current: list[RuntimeTensorTarget] = []
current_bytes = 0
for record in records:
if current and current_bytes + record.nbytes > max_shard_bytes:
shards.append(tuple(current))
current = []
current_bytes = 0
current.append(record)
current_bytes += record.nbytes
if current:
shards.append(tuple(current))
return tuple(shards)
class FinalLayoutBF16Producer:
"""Publish finalized BF16-policy tensors through a store-scoped writer."""
def __init__(
self,
context: FinalLayoutIdentityContext,
pipeline: nn.Module,
dit_modules: Sequence[tuple[str, nn.Module]],
*,
max_shard_bytes: int = DEFAULT_SHARD_SIZE_BYTES,
) -> None:
if max_shard_bytes <= 0:
raise ValueError("final-layout BF16 shard size must be positive")
if context.spec != FINAL_LAYOUT_BF16_SPEC:
raise ValueError("producer requires the exact BF16 artifact specification")
self._context = context
self._pipeline = pipeline
self._dit_modules = tuple(dit_modules)
self._max_shard_bytes = max_shard_bytes
if tuple(name for name, _ in self._dit_modules) != context.dit_names:
raise ValueError("producer DiT components differ from the exact identity context")
if type(pipeline) is not context.pipeline_type or tuple(type(module) for _, module in self._dit_modules) != (
context.dit_types
):
raise ValueError("producer model implementation differs from the exact identity context")
self._spec = WeightProductionSpec(
producer_id=FINAL_LAYOUT_BF16_PRODUCER_ID,
outputs=(context.identity,),
source_mode=ProductionSourceMode.FINALIZED_MODEL,
coordination_scope=CoordinationScope.SINGLE_PROCESS,
lookup_phase=LookupPhase.POST_LOAD_ONLY,
)
@property
def spec(self) -> WeightProductionSpec:
return self._spec
def produce(self, writer: ArtifactWriter) -> ProductionMetadata:
try:
return self._produce(writer)
except FinalLayoutContractError as exc:
raise final_layout_producer_error(exc) from exc
def _produce(self, writer: ArtifactWriter) -> ProductionMetadata:
self._context.ensure_sources_unchanged()
bindings = validate_final_layout_model_contract(
self._dit_modules,
expected_schema=self._context.spec.model_contract_schema,
)
records = collect_final_layout_targets(
self._pipeline,
self._dit_modules,
policy=FINAL_LAYOUT_BF16_POLICY,
require_materialized=True,
)
contract_digest = validate_final_layout_identity(self._context, records)
for binding in bindings:
binding.validator()
shards = _split_shards(records, self._max_shard_bytes)
shard_count = len(shards)
for index, shard in enumerate(shards, start=1):
specs = tuple(
TensorWriteSpec(
name=record.name,
shape=tuple(record.tensor.shape),
dtype=record.tensor.dtype,
kind=record.kind,
role=record.role,
)
for record in shard
)
file_name = f"model-{index:05d}-of-{shard_count:05d}.safetensors"
with writer.open_tensor_file(file_name, specs) as output:
for record in shard:
output.write_tensor(record.name, record.tensor.detach())
self._context.ensure_sources_unchanged()
return ProductionMetadata(
producer_schema=FINAL_LAYOUT_BF16_MANIFEST_SCHEMA,
restorer_schema=FINAL_LAYOUT_TENSOR_RESTORER_SCHEMA,
format_metadata=FINAL_LAYOUT_BF16_POLICY.build_format_metadata(
component_names=self._context.dit_names,
tensor_contract_digest=contract_digest,
tensor_count=len(records),
),
)
__all__ = [
"DEFAULT_SHARD_SIZE_BYTES",
"FINAL_LAYOUT_BF16_MANIFEST_SCHEMA",
"FINAL_LAYOUT_BF16_POLICY",
"FINAL_LAYOUT_BF16_PRODUCER_ID",
"FINAL_LAYOUT_BF16_REPRESENTATION",
"FINAL_LAYOUT_BF16_SPEC",
"FINAL_LAYOUT_BF16_VERSION",
"FinalLayoutBF16Policy",
"FinalLayoutBF16Producer",
]
@@ -0,0 +1,196 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""Generic validation-first restorer for exact final-layout tensor artifacts."""
from __future__ import annotations
from collections.abc import Callable
from dataclasses import dataclass
import torch
from torch import nn
from vllm_omni.host_weight_runtime import HostWeightLease, TensorKind, WeightRestorePlan
from .identity_adapter import (
FinalLayoutIdentityContext,
validate_final_layout_identity,
)
from .tensor_layout import collect_final_layout_targets, validate_final_layout_model_contract
@dataclass(frozen=True)
class _Replacement:
name: str
parent: nn.Module
leaf_name: str
target: torch.Tensor
source: torch.Tensor
kind: TensorKind
def current_target(self) -> torch.Tensor | None:
if self.kind is TensorKind.PARAMETER:
return self.parent._parameters.get(self.leaf_name)
return self.parent._buffers.get(self.leaf_name)
class FinalLayoutTensorRestorePlan(WeightRestorePlan):
"""One-shot exact tensor rebinding validated before the first mutation."""
def __init__(
self,
lease: HostWeightLease,
replacements: tuple[_Replacement, ...],
validators: tuple[Callable[[], None], ...],
source_guard: Callable[[], None],
) -> None:
self._lease = lease
self._replacements = replacements
self._validators = validators
self._source_guard = source_guard
self._committed = False
def commit(self) -> None:
if self._committed:
raise RuntimeError("final-layout tensor restore plan was already committed")
if self._lease.closed:
raise RuntimeError("cannot commit a final-layout tensor restore from a closed lease")
self._source_guard()
for replacement in self._replacements:
current = replacement.current_target()
if current is not replacement.target:
raise RuntimeError(f"restore target {replacement.name!r} changed after preflight")
if (
tuple(current.shape) != tuple(replacement.source.shape)
or current.dtype != replacement.source.dtype
or tuple(current.stride()) != tuple(replacement.source.stride())
or not replacement.source.is_contiguous()
):
raise RuntimeError(f"restore target {replacement.name!r} changed layout after preflight")
# Mark first: assignment or model-validator failure makes the plan
# permanently non-retryable and the partially restored model disposable.
self._committed = True
for replacement in self._replacements:
if replacement.target.is_meta:
if replacement.kind is TensorKind.PARAMETER:
replacement.parent._parameters[replacement.leaf_name] = nn.Parameter(
replacement.source,
requires_grad=replacement.target.requires_grad,
)
else:
replacement.parent._buffers[replacement.leaf_name] = replacement.source
else:
replacement.target.data = replacement.source
for validator in self._validators:
validator()
class FinalLayoutTensorRestorer:
"""Validate and plan restoration for one exact policy-defined lease."""
def __init__(self, context: FinalLayoutIdentityContext) -> None:
if not context.dit_names or any(not name for name in context.dit_names):
raise ValueError("final-layout restorer requires DiT component names")
self._context = context
@property
def schema(self) -> str:
return self._context.spec.producer.restorer_schema
def plan_restore(self, model: object, lease: HostWeightLease) -> FinalLayoutTensorRestorePlan:
if not isinstance(model, nn.Module):
raise TypeError("final-layout restoration requires an nn.Module pipeline")
if type(model) is not self._context.pipeline_type:
raise ValueError("restore model implementation differs from the exact identity context")
if lease.closed:
raise ValueError("cannot restore from a closed HostWeightLease")
if lease.identity.canonical_bytes != self._context.identity.canonical_bytes:
raise ValueError("lease semantic identity differs from the exact restore request")
if lease.manifest.identity.canonical_bytes != self._context.identity.canonical_bytes:
raise ValueError("lease manifest identity differs from the exact restore request")
if lease.identity.producer.restorer_schema != self.schema or lease.manifest.restorer_schema != self.schema:
raise ValueError("lease restorer schema is incompatible with the exact restore request")
self._context.ensure_sources_unchanged()
try:
dit_modules = tuple((name, model.get_submodule(name)) for name in self._context.dit_names)
except AttributeError as exc:
raise ValueError("one or more lease-owned DiT modules do not exist in the pipeline") from exc
if tuple(type(module) for _, module in dit_modules) != self._context.dit_types:
raise ValueError("restore DiT implementation differs from the exact identity context")
bindings = validate_final_layout_model_contract(
dit_modules,
expected_schema=self._context.spec.model_contract_schema,
)
targets = collect_final_layout_targets(
model,
dit_modules,
policy=self._context.policy,
require_materialized=False,
)
contract_digest = validate_final_layout_identity(self._context, targets)
manifest_entries = {entry.name: entry for entry in lease.manifest.tensors}
target_names = {target.name for target in targets}
if set(manifest_entries) != target_names or target_names != self._context.tensor_names:
missing = sorted(target_names - set(manifest_entries))
unexpected = sorted(set(manifest_entries) - target_names)
raise ValueError(
"lease tensor coverage differs from the structurally initialized DiT: "
f"missing={missing[:5]}, unexpected={unexpected[:5]}"
)
self._context.policy.validate_format_metadata(
lease.manifest.format_metadata,
component_names=self._context.dit_names,
tensor_contract_digest=contract_digest,
tensor_count=len(manifest_entries),
)
replacements: list[_Replacement] = []
source_storages: dict[tuple[int, int], str] = {}
for target in targets:
entry = manifest_entries[target.name]
source = lease.tensors[target.name]
if entry.kind is not target.kind or entry.role != target.role:
raise ValueError(f"lease tensor kind or role differs for {target.name!r}")
if (
tuple(source.shape) != tuple(target.tensor.shape)
or source.dtype != target.tensor.dtype
or tuple(source.stride()) != tuple(target.tensor.stride())
or not source.is_contiguous()
or source.device.type != "cpu"
):
raise ValueError(f"lease tensor layout differs for {target.name!r}")
if source.numel():
storage = source.untyped_storage()
storage_id = (storage.data_ptr(), storage.nbytes())
owner = source_storages.setdefault(storage_id, target.name)
if owner != target.name:
raise ValueError(f"lease tensor {target.name!r} aliases {owner!r}")
parent_path, _, leaf_name = target.name.rpartition(".")
replacements.append(
_Replacement(
name=target.name,
parent=model.get_submodule(parent_path),
leaf_name=leaf_name,
target=target.tensor,
source=source,
kind=target.kind,
)
)
self._context.ensure_sources_unchanged()
return FinalLayoutTensorRestorePlan(
lease,
tuple(replacements),
tuple(binding.validator for binding in bindings),
self._context.ensure_sources_unchanged,
)
__all__ = [
"FinalLayoutTensorRestorePlan",
"FinalLayoutTensorRestorer",
]
@@ -0,0 +1,351 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""Canonical diffusion weight-source identity and one-load replacement guard."""
from __future__ import annotations
import hashlib
import os
import stat
from collections.abc import Sequence
from dataclasses import dataclass
from enum import Enum
from pathlib import Path
import regex as re
from vllm_omni.host_weight_runtime import CanonicalJson, WeightSourceIdentity
from vllm_omni.host_weight_runtime.identity import canonical_json
from .contracts import (
FinalLayoutContractCode,
FinalLayoutContractError,
ImplementationIdentity,
)
_HASH_CHUNK_BYTES = 8 * 1024**2
_IMMUTABLE_REVISION_RE = re.compile(r"(?:[0-9a-fA-F]{40}|[0-9a-fA-F]{64})\Z")
class WeightSourceKind(str, Enum):
LOCAL_PATH = "local_path"
HUGGING_FACE_HUB = "hugging_face_hub"
@dataclass(frozen=True)
class PreparedWeightSource:
"""Canonical source files and adapter ABI selected by the ordinary loader."""
model_or_path: str
subfolder: str | None
requested_revision: str | None
prefix: str
resolved_root: Path
weight_files: tuple[Path, ...]
use_safetensors: bool
checkpoint_adapter: ImplementationIdentity | None = None
source_kind: WeightSourceKind = WeightSourceKind.LOCAL_PATH
def __post_init__(self) -> None:
if not self.model_or_path or not isinstance(self.prefix, str):
raise ValueError("prepared source model and prefix must be valid strings")
if not isinstance(self.resolved_root, Path):
raise ValueError("prepared source root must use pathlib.Path")
if not isinstance(self.weight_files, tuple) or not self.weight_files:
raise ValueError("prepared source must contain an immutable tuple of weight files")
if any(not isinstance(path, Path) for path in self.weight_files):
raise ValueError("prepared source weight files must use pathlib.Path")
if not isinstance(self.use_safetensors, bool):
raise ValueError("prepared source safetensors mode must be a boolean")
if self.checkpoint_adapter is not None and not isinstance(
self.checkpoint_adapter,
ImplementationIdentity,
):
raise ValueError("checkpoint adapter must use ImplementationIdentity")
if not isinstance(self.source_kind, WeightSourceKind):
raise ValueError("prepared source kind must use WeightSourceKind")
@dataclass(frozen=True)
class _FileSnapshot:
path: Path
relative_name: str
size: int
device: int
inode: int
mtime_ns: int
ctime_ns: int
symlink_target: str | None
content_id: str
def semantic_dict(self) -> dict[str, object]:
return {
"relative_name": self.relative_name,
"size": self.size,
"content_id": self.content_id,
}
def unchanged(self) -> bool:
try:
current = self.path.stat()
symlink_target = os.readlink(self.path) if self.path.is_symlink() else None
except OSError:
return False
return (
current.st_size == self.size
and current.st_dev == self.device
and current.st_ino == self.inode
and current.st_mtime_ns == self.mtime_ns
and current.st_ctime_ns == self.ctime_ns
and symlink_target == self.symlink_target
)
@dataclass(frozen=True)
class _SourceSnapshot:
semantic: CanonicalJson
revision: str
files: tuple[_FileSnapshot, ...]
def unchanged(self) -> bool:
return all(file.unchanged() for file in self.files)
@dataclass(frozen=True)
class FinalLayoutSourceContext:
"""Resolved source identity plus process-local replacement observations."""
identity: WeightSourceIdentity
snapshots: tuple[_SourceSnapshot, ...]
def sources_unchanged(self) -> bool:
return all(snapshot.unchanged() for snapshot in self.snapshots)
def ensure_sources_unchanged(self) -> None:
if not self.sources_unchanged():
raise FinalLayoutContractError(
FinalLayoutContractCode.SOURCE_CHANGED,
"canonical source changed after final-layout identity resolution",
)
def _sha256_file(path: Path) -> str:
digest = hashlib.sha256()
with path.open("rb") as handle:
while chunk := handle.read(_HASH_CHUNK_BYTES):
digest.update(chunk)
return digest.hexdigest()
def _hf_blob_content_id(
path: Path,
*,
source: PreparedWeightSource,
source_root: Path,
) -> str | None:
if source.source_kind is not WeightSourceKind.HUGGING_FACE_HUB or not path.is_symlink():
return None
if Path(source.model_or_path).expanduser().exists():
return None
parts = source_root.parts
snapshot_indexes = [index for index, part in enumerate(parts[:-1]) if part == "snapshots"]
if not snapshot_indexes:
return None
snapshot_index = snapshot_indexes[-1]
revision = parts[snapshot_index + 1]
if _IMMUTABLE_REVISION_RE.fullmatch(revision) is None:
return None
repo_root = Path(*parts[:snapshot_index])
expected_repo_name = f"models--{source.model_or_path.replace('/', '--')}"
if repo_root.name != expected_repo_name:
return None
snapshot_root = repo_root / "snapshots" / revision
try:
path.relative_to(snapshot_root)
except ValueError:
return None
blobs_root = repo_root / "blobs"
try:
resolved_target = path.resolve(strict=True)
resolved_blobs_root = blobs_root.resolve(strict=True)
except OSError:
return None
if resolved_target.parent != resolved_blobs_root:
return None
blob_name = resolved_target.name
if _IMMUTABLE_REVISION_RE.fullmatch(blob_name) is None:
return None
return f"immutable-blob:{blob_name.lower()}"
def _snapshot_revision(path: Path) -> str | None:
parts = path.absolute().parts
for index, part in enumerate(parts[:-1]):
if part != "snapshots":
continue
candidate = parts[index + 1]
if _IMMUTABLE_REVISION_RE.fullmatch(candidate) is not None:
return candidate.lower()
return None
def _logical_model_id(value: str) -> str:
candidate = Path(value).expanduser()
try:
if candidate.exists():
return str(candidate.resolve())
except OSError:
pass
return value
def _snapshot_source(source: PreparedWeightSource) -> _SourceSnapshot:
root = source.resolved_root.expanduser().resolve()
if not root.is_dir():
raise ValueError(f"canonical source root is not a directory: {root}")
immutable_revision = _snapshot_revision(root)
if immutable_revision is None and source.requested_revision is not None:
requested = source.requested_revision.strip()
if _IMMUTABLE_REVISION_RE.fullmatch(requested) is not None:
immutable_revision = requested.lower()
files: list[_FileSnapshot] = []
for path in sorted(source.weight_files, key=lambda item: str(item)):
candidate = path.expanduser()
if not candidate.is_absolute():
candidate = root / candidate
candidate = candidate.absolute()
try:
relative_name = candidate.relative_to(root).as_posix()
except ValueError as exc:
raise ValueError(f"weight file {candidate} is outside its canonical source root {root}") from exc
try:
current = candidate.stat()
except OSError as exc:
raise ValueError(f"cannot stat canonical weight file {candidate}") from exc
if not stat.S_ISREG(current.st_mode):
raise ValueError(f"canonical weight source is not a regular file: {candidate}")
symlink_target = os.readlink(candidate) if candidate.is_symlink() else None
content_id = _hf_blob_content_id(
candidate,
source=source,
source_root=root,
)
if content_id is None:
content_id = f"sha256:{_sha256_file(candidate)}"
files.append(
_FileSnapshot(
path=candidate,
relative_name=relative_name,
size=current.st_size,
device=current.st_dev,
inode=current.st_ino,
mtime_ns=current.st_mtime_ns,
ctime_ns=current.st_ctime_ns,
symlink_target=symlink_target,
content_id=content_id,
)
)
file_semantics = [file.semantic_dict() for file in files]
file_fingerprint = hashlib.sha256(canonical_json(file_semantics)).hexdigest()
revision = immutable_revision or f"content-{file_fingerprint}"
semantic = CanonicalJson.from_value(
{
"checkpoint_adapter": (
source.checkpoint_adapter.to_dict() if source.checkpoint_adapter is not None else None
),
"files": file_semantics,
"model_or_path": _logical_model_id(source.model_or_path),
"prefix": source.prefix,
"resolved_revision": revision,
"source_kind": source.source_kind.value,
"subfolder": source.subfolder,
"use_safetensors": source.use_safetensors,
}
)
return _SourceSnapshot(semantic=semantic, revision=revision, files=tuple(files))
def _resolve_target_sources(
prepared_sources: Sequence[PreparedWeightSource],
target_names: frozenset[str],
) -> dict[str, PreparedWeightSource]:
"""Bind every target to one source using deterministic longest-prefix wins."""
if not target_names:
raise FinalLayoutContractError(
FinalLayoutContractCode.SOURCE_COVERAGE_INVALID,
"final-layout source identity requires at least one owned tensor",
)
bindings: dict[str, PreparedWeightSource] = {}
for target_name in sorted(target_names):
matches = [source for source in prepared_sources if target_name.startswith(source.prefix)]
if not matches:
raise FinalLayoutContractError(
FinalLayoutContractCode.SOURCE_COVERAGE_INVALID,
f"no canonical weight source covers final-layout tensor {target_name!r}",
)
longest_prefix = max(len(source.prefix) for source in matches)
winners = [source for source in matches if len(source.prefix) == longest_prefix]
if len(winners) != 1:
candidates = sorted(
f"{source.model_or_path}:{source.subfolder or ''}:{source.prefix}" for source in winners
)
raise FinalLayoutContractError(
FinalLayoutContractCode.SOURCE_COVERAGE_INVALID,
f"multiple equally specific canonical sources cover {target_name!r}: {candidates}",
)
bindings[target_name] = winners[0]
return bindings
def resolve_final_layout_source_identity(
prepared_sources: Sequence[PreparedWeightSource],
*,
model_id: str,
target_names: frozenset[str],
) -> FinalLayoutSourceContext:
"""Resolve exact source identity for files covering the owned tensors."""
bindings = _resolve_target_sources(prepared_sources, target_names)
selected_sources = frozenset(bindings.values())
snapshot_by_source = {source: _snapshot_source(source) for source in selected_sources}
snapshots = tuple(sorted(snapshot_by_source.values(), key=lambda snapshot: snapshot.semantic.encoded))
source_semantics = [snapshot.semantic.to_value() for snapshot in snapshots]
target_bindings = [
{
"source_fingerprint": hashlib.sha256(snapshot_by_source[source].semantic.encoded).hexdigest(),
"source_prefix": source.prefix,
"target_name": target_name,
}
for target_name, source in sorted(bindings.items())
]
source_document = {
"sources": source_semantics,
"target_bindings": target_bindings,
}
source_fingerprint = hashlib.sha256(canonical_json(source_document)).hexdigest()
unique_revisions = tuple(sorted({snapshot.revision for snapshot in snapshots}))
aggregate_revision = (
unique_revisions[0]
if len(unique_revisions) == 1
else f"aggregate-{hashlib.sha256(canonical_json(unique_revisions)).hexdigest()}"
)
return FinalLayoutSourceContext(
identity=WeightSourceIdentity(
model_id=_logical_model_id(model_id),
revision=aggregate_revision,
fingerprint=source_fingerprint,
metadata=CanonicalJson.from_value(source_document),
),
snapshots=snapshots,
)
__all__ = [
"FinalLayoutSourceContext",
"PreparedWeightSource",
"WeightSourceKind",
"resolve_final_layout_source_identity",
]
@@ -0,0 +1,287 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""Representation-neutral ownership discovery for final-layout tensors."""
from __future__ import annotations
import hashlib
from collections.abc import Callable, Iterator, Sequence
from dataclasses import dataclass
from itertools import chain
import torch
from torch import nn
from vllm_omni.diffusion.models.host_weight_contract import FinalLayoutModelContract
from vllm_omni.host_weight_runtime import TensorKind
from vllm_omni.host_weight_runtime.identity import canonical_json
from .contracts import (
FinalLayoutContractCode,
FinalLayoutContractError,
FinalLayoutTensorPolicy,
)
@dataclass(frozen=True)
class ModelRestoreBinding:
"""Model-declared restore ABI and its post-commit validator."""
name: str
contract: FinalLayoutModelContract
validator: Callable[[], None]
@dataclass(frozen=True)
class RuntimeTensorTarget:
"""One complete final-layout tensor owned by a DiT component."""
name: str
tensor: torch.Tensor
kind: TensorKind
role: str
@property
def nbytes(self) -> int:
return self.tensor.numel() * self.tensor.element_size()
def validate_final_layout_model_contract(
dit_modules: Sequence[tuple[str, nn.Module]],
*,
expected_schema: str,
) -> tuple[ModelRestoreBinding, ...]:
"""Require an explicit tensor-complete restore ABI from every DiT."""
if not dit_modules:
raise FinalLayoutContractError(
FinalLayoutContractCode.MODEL_CONTRACT_UNSUPPORTED,
"final-layout artifacts require at least one DiT component",
)
bindings: list[ModelRestoreBinding] = []
seen_names: set[str] = set()
for name, module in dit_modules:
if not name or name in seen_names:
raise FinalLayoutContractError(
FinalLayoutContractCode.OWNERSHIP_AMBIGUOUS,
f"DiT component name {name!r} is empty or duplicated",
)
seen_names.add(name)
contract = getattr(module, "host_weight_restore_contract", None)
if not isinstance(contract, FinalLayoutModelContract) or contract.schema != expected_schema:
raise FinalLayoutContractError(
FinalLayoutContractCode.MODEL_CONTRACT_UNSUPPORTED,
f"DiT component {name!r} does not declare restore schema {expected_schema!r}",
)
validator = getattr(module, "validate_restored_host_weights", None)
if not callable(validator):
raise FinalLayoutContractError(
FinalLayoutContractCode.MODEL_CONTRACT_UNSUPPORTED,
f"DiT component {name!r} does not implement validate_restored_host_weights()",
)
bindings.append(ModelRestoreBinding(name, contract, validator))
return tuple(bindings)
def _named_parameters_with_duplicates(module: nn.Module) -> Iterator[tuple[str, nn.Parameter]]:
try:
return module.named_parameters(remove_duplicate=False)
except TypeError: # pragma: no cover - compatibility with old torch
return module.named_parameters()
def _named_buffers_with_duplicates(module: nn.Module) -> Iterator[tuple[str, torch.Tensor]]:
try:
return module.named_buffers(remove_duplicate=False)
except TypeError: # pragma: no cover - compatibility with old torch
return module.named_buffers()
def _is_persistent_buffer(module: nn.Module, local_name: str) -> bool:
parent_path, _, leaf_name = local_name.rpartition(".")
owner = module.get_submodule(parent_path)
return leaf_name not in owner._non_persistent_buffers_set
def _resolve_pipeline_tensor(pipeline: nn.Module, runtime_name: str) -> torch.Tensor | None:
parent_path, _, leaf_name = runtime_name.rpartition(".")
parent = pipeline.get_submodule(parent_path)
tensor = parent._parameters.get(leaf_name)
if tensor is None:
tensor = parent._buffers.get(leaf_name)
return tensor
def collect_final_layout_targets(
pipeline: nn.Module,
dit_modules: Sequence[tuple[str, nn.Module]],
*,
policy: FinalLayoutTensorPolicy,
require_materialized: bool,
) -> tuple[RuntimeTensorTarget, ...]:
"""Return complete, alias-free targets validated by one representation policy."""
validate_final_layout_model_contract(
dit_modules,
expected_schema=policy.spec.model_contract_schema,
)
records: dict[str, RuntimeTensorTarget] = {}
for dit_name, dit_module in dit_modules:
for local_name, tensor in _named_parameters_with_duplicates(dit_module):
runtime_name = f"{dit_name}.{local_name}"
candidate = RuntimeTensorTarget(
runtime_name,
tensor,
TensorKind.PARAMETER,
policy.tensor_role(runtime_name, tensor, TensorKind.PARAMETER),
)
existing = records.get(runtime_name)
if existing is not None and existing.tensor is not tensor:
raise FinalLayoutContractError(
FinalLayoutContractCode.OWNERSHIP_AMBIGUOUS,
f"multiple DiT tensors resolve to {runtime_name!r}",
)
records[runtime_name] = candidate
for local_name, tensor in _named_buffers_with_duplicates(dit_module):
if not _is_persistent_buffer(dit_module, local_name):
continue
runtime_name = f"{dit_name}.{local_name}"
candidate = RuntimeTensorTarget(
runtime_name,
tensor,
TensorKind.BUFFER,
policy.tensor_role(runtime_name, tensor, TensorKind.BUFFER),
)
existing = records.get(runtime_name)
if existing is not None and existing.tensor is not tensor:
raise FinalLayoutContractError(
FinalLayoutContractCode.OWNERSHIP_AMBIGUOUS,
f"multiple DiT tensors resolve to {runtime_name!r}",
)
records[runtime_name] = candidate
if not records:
raise FinalLayoutContractError(
FinalLayoutContractCode.OWNERSHIP_AMBIGUOUS,
"no DiT parameters or persistent buffers were discovered",
)
object_owners: dict[int, str] = {}
storage_owners: dict[tuple[int, int], str] = {}
for record in records.values():
tensor = record.tensor
if not record.role:
raise FinalLayoutContractError(
FinalLayoutContractCode.TENSOR_UNSUPPORTED,
f"{record.name!r} has an empty semantic role",
)
try:
pipeline_tensor = _resolve_pipeline_tensor(pipeline, record.name)
except AttributeError as exc:
raise FinalLayoutContractError(
FinalLayoutContractCode.OWNERSHIP_AMBIGUOUS,
f"DiT tensor {record.name!r} is not owned by the pipeline",
) from exc
if pipeline_tensor is not tensor:
raise FinalLayoutContractError(
FinalLayoutContractCode.OWNERSHIP_AMBIGUOUS,
f"DiT tensor {record.name!r} does not resolve to the discovered object",
)
object_owner = object_owners.setdefault(id(tensor), record.name)
if object_owner != record.name:
raise FinalLayoutContractError(
FinalLayoutContractCode.TENSOR_UNSUPPORTED,
f"{record.name!r} aliases tensor object {object_owner!r}",
)
if hasattr(tensor, "to_local"):
raise FinalLayoutContractError(
FinalLayoutContractCode.TENSOR_UNSUPPORTED,
f"{record.name!r} is a distributed tensor",
)
if tensor.layout != torch.strided:
raise FinalLayoutContractError(
FinalLayoutContractCode.TENSOR_UNSUPPORTED,
f"{record.name!r} uses unsupported layout {tensor.layout}",
)
if tensor.device.type not in {"cpu", "meta"}:
raise FinalLayoutContractError(
FinalLayoutContractCode.TENSOR_UNSUPPORTED,
f"{record.name!r} must be a CPU or meta tensor, got {tensor.device}",
)
if not tensor.is_contiguous():
raise FinalLayoutContractError(
FinalLayoutContractCode.TENSOR_UNSUPPORTED,
f"{record.name!r} is non-contiguous with stride {tensor.stride()}",
)
policy.validate_target(record)
if not require_materialized:
continue
if tensor.device.type != "cpu" or tensor.is_meta:
raise FinalLayoutContractError(
FinalLayoutContractCode.TENSOR_UNSUPPORTED,
f"{record.name!r} must be a materialized CPU tensor, got {tensor.device}",
)
storage = tensor.untyped_storage()
if tensor.storage_offset() != 0 or storage.nbytes() != record.nbytes:
raise FinalLayoutContractError(
FinalLayoutContractCode.TENSOR_UNSUPPORTED,
f"{record.name!r} is a view into a larger storage",
)
if record.nbytes:
storage_id = (storage.data_ptr(), storage.nbytes())
storage_owner = storage_owners.setdefault(storage_id, record.name)
if storage_owner != record.name:
raise FinalLayoutContractError(
FinalLayoutContractCode.TENSOR_UNSUPPORTED,
f"{record.name!r} shares storage with {storage_owner!r}",
)
for pipeline_name, tensor in chain(
_named_parameters_with_duplicates(pipeline),
_named_buffers_with_duplicates(pipeline),
):
owner = object_owners.get(id(tensor))
if owner is not None and owner != pipeline_name:
raise FinalLayoutContractError(
FinalLayoutContractCode.TENSOR_UNSUPPORTED,
f"cached tensor {owner!r} aliases pipeline tensor {pipeline_name!r}",
)
if not require_materialized or tensor.device.type != "cpu" or tensor.is_meta or tensor.numel() == 0:
continue
storage = tensor.untyped_storage()
pipeline_storage_owner = storage_owners.get((storage.data_ptr(), storage.nbytes()))
if pipeline_storage_owner is not None and pipeline_storage_owner != pipeline_name:
raise FinalLayoutContractError(
FinalLayoutContractCode.TENSOR_UNSUPPORTED,
f"cached tensor {pipeline_storage_owner!r} shares storage with pipeline tensor {pipeline_name!r}",
)
ordered = tuple(records[name] for name in sorted(records))
policy.validate_collection(ordered)
return ordered
def tensor_contract_sha256(records: Sequence[RuntimeTensorTarget]) -> str:
"""Hash exact structural ownership and representation-specific tensor roles."""
contract = [
{
"dtype": str(record.tensor.dtype),
"kind": record.kind.value,
"name": record.name,
"role": record.role,
"shape": list(record.tensor.shape),
"stride": list(record.tensor.stride()),
}
for record in records
]
return hashlib.sha256(canonical_json(contract)).hexdigest()
__all__ = [
"ModelRestoreBinding",
"RuntimeTensorTarget",
"collect_final_layout_targets",
"tensor_contract_sha256",
"validate_final_layout_model_contract",
]
@@ -0,0 +1,40 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""Model-owned declaration for complete final-layout tensor restoration."""
from __future__ import annotations
from dataclasses import dataclass
FINAL_LAYOUT_TENSOR_MODEL_CONTRACT_SCHEMA = "vllm-omni.diffusion.final-layout-tensors-v1"
@dataclass(frozen=True)
class FinalLayoutModelContract:
"""Versioned model ABI for reconstruction from exact final-layout tensors."""
implementation_id: str
version: str
schema: str = FINAL_LAYOUT_TENSOR_MODEL_CONTRACT_SCHEMA
def __post_init__(self) -> None:
for name, value in (
("schema", self.schema),
("implementation_id", self.implementation_id),
("version", self.version),
):
if not isinstance(value, str) or not value:
raise ValueError(f"final-layout model contract {name} must not be empty")
def to_dict(self) -> dict[str, str]:
return {
"schema": self.schema,
"implementation_id": self.implementation_id,
"version": self.version,
}
__all__ = [
"FINAL_LAYOUT_TENSOR_MODEL_CONTRACT_SCHEMA",
"FinalLayoutModelContract",
]
@@ -1,4 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""MiniMax H3 packed-token audio/video DiT for vLLM-Omni.
vLLM tensor parallel linears and the unified attention layer provide TP and
@@ -43,6 +44,7 @@ from vllm_omni.diffusion.layers.activation import SiluAndMul
from vllm_omni.diffusion.layers.fused_qk_norm_rope import fused_qk_norm_rope
from vllm_omni.diffusion.layers.norm import RMSNorm
from vllm_omni.diffusion.layers.rope import RotaryEmbedding
from vllm_omni.diffusion.models.host_weight_contract import FinalLayoutModelContract
if TYPE_CHECKING:
from vllm.model_executor.layers.quantization.base_config import (
@@ -889,6 +891,15 @@ class MiniMaxH3SPGather(nn.Module):
class MiniMaxH3DiTModel(nn.Module):
# Loading is tensor-complete: constructor state plus final-layout
# parameters and persistent buffers is sufficient to reconstruct a ready
# inference model. The model-specific validator below checks the preserved
# FP32 portion after a lease-backed restore commits.
host_weight_restore_contract = FinalLayoutModelContract(
implementation_id="minimax-h3-dit",
version="1",
)
_cache_dit_adapter_config = CacheDiTAdapterConfig(
block_forward_patterns={"blocks": ForwardPattern.Pattern_3},
# H3 is CFG-distilled and performs one transformer forward per step.
@@ -1064,6 +1075,10 @@ class MiniMaxH3DiTModel(nn.Module):
if name in MINIMAX_H3_FP32_BUFFER_NAMES and buffer.dtype != _FP32_DTYPE:
raise ValueError(f"{name} must stay fp32 after load, got {buffer.dtype}.")
def validate_restored_host_weights(self) -> None:
"""Validate mixed-precision invariants after lease-backed restore."""
self.post_load_weights()
def load_weights(
self,
weights: Iterable[tuple[str, torch.Tensor]],