Files
vllm-omni/tests/diffusion
hsliuustc0106 ba5943dddb [Model] Derive MiniMax-H3 DLO component cache budget from staged bytes
Replace BoundedAllocatorCache's device-capacity fraction bound (25% of
total memory) with a budget derived from the model topology: each attached
PinnedModuleStager reports its unique staged storage bytes once, and the
retention ceiling is that total times a fixed 2.0 headroom multiplier
covering allocator fragmentation and transient storages freed at component
boundaries. The 5% physically-free floor is unchanged.

The budget therefore scales with the staged components instead of device
capacity, so no user-chosen fraction is needed and behavior is identical
across differently sized devices with the same model. Re-attaching the
same cache does not double count; a second stager grows the budget by its
own bytes.

Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
2026-08-26 09:49:55 +08:00
..
2026-08-25 09:10:17 +08:00