[Misc] Add standalone safetensors retention diagnostic (#7145)

Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
This commit is contained in:
Hongsheng Liu authored and GitHub committed 2026-09-10 17:57:20 +08:00
1 parent ce27ee6038
commit f11bf1a05d
3 files changed
+182

No files matched your search

@@ -42,6 +42,11 @@ V1 does not include:
## Motivation and use cases
For investigating CPU memory retained by dependency tensor materialization,
see the [standalone safetensors diagnostic](https://github.com/vllm-project/vllm-omni/blob/main/benchmarks/host_weight_runtime/README.md).
It distinguishes repeated dependency calls from reuse of cached views and does
not establish per-request HWR leakage.
Model loading can create the same final host representation repeatedly. This
is especially expensive when loading performs checkpoint decoding, tensor
renaming, TP slicing, quantization, packing, or scale construction before GPU