[Model] Derive MiniMax-H3 DLO component cache budget from staged bytes

Replace BoundedAllocatorCache's device-capacity fraction bound (25% of
total memory) with a budget derived from the model topology: each attached
PinnedModuleStager reports its unique staged storage bytes once, and the
retention ceiling is that total times a fixed 2.0 headroom multiplier
covering allocator fragmentation and transient storages freed at component
boundaries. The 5% physically-free floor is unchanged.

The budget therefore scales with the staged components instead of device
capacity, so no user-chosen fraction is needed and behavior is identical
across differently sized devices with the same model. Re-attaching the
same cache does not double count; a second stager grows the budget by its
own bytes.

Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
This commit is contained in:
hsliuustc0106 committed 2026-08-26 09:49:55 +08:00
1 parent 780e8d40aa
commit ba5943dddb
3 files changed
+72 -12

No files matched your search

@@ -177,8 +177,12 @@ component can be reused by the next. The DiT prefetch path and its two shared
device buffers are unchanged.
After a component is offloaded, cached-but-unallocated memory is retained only
while it is at most 25% of device capacity and at least 5% of device capacity
remains physically free. Crossing either bound releases the allocator cache.
while it stays within a budget derived from the model topology — the staged
component byte total reported by the attached `PinnedModuleStager` instances
times a fixed headroom multiplier — and at least 5% of device capacity
remains physically free. Crossing either bound releases the allocator cache,
so the budget scales with the staged components rather than with device
capacity and needs no user-chosen fraction.
Missing allocator telemetry also releases it conservatively. Component or
staging failure forces a release, and an out-of-memory allocation gets one
retry after release. This is a component-local policy rather than a global