[Misc] Add standalone safetensors retention diagnostic (#7145)
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
This commit is contained in:
1 parent
ce27ee6038
commit
f11bf1a05d
3 files changed
+182
No files matched your search
@@ -0,0 +1,57 @@
|
||||
# Host weight dependency memory diagnostic
|
||||
|
||||
`safetensors_retention.py` isolates repeated CPU `get_tensor()` calls from
|
||||
access to one cached tensor view. It imports neither vLLM nor HWR and uses a
|
||||
synthetic float32 payload controlled by `--tensor-elements` (default: 64).
|
||||
Tensor size can affect retention: a negative result for 16 elements does not
|
||||
rule out growth for 64 elements. The selected size is recorded in the JSON
|
||||
arguments and used for both file creation and the correctness check.
|
||||
HWR itself calls `get_tensor()` when
|
||||
acquiring a lease, then exposes cached views; this probe does not measure HWR
|
||||
requests or repeated lease acquisition.
|
||||
|
||||
Run one mode per fresh process in the same Python environment and CPU affinity:
|
||||
|
||||
```bash
|
||||
git rev-parse HEAD > revision.txt
|
||||
git status --short > working-tree.txt
|
||||
git diff > working-tree.patch
|
||||
# Select an allowed CPU from your process affinity; 0 is only an example.
|
||||
CUDA_VISIBLE_DEVICES='' taskset -c 0 timeout 120s python \
|
||||
benchmarks/host_weight_runtime/safetensors_retention.py \
|
||||
--mode get_tensor --iterations 10000 --sample-every 5000 > feasibility.json
|
||||
```
|
||||
|
||||
For a size comparison, run 16 and 64 elements twice each in fresh processes,
|
||||
with cached reuse as a control at both sizes:
|
||||
|
||||
```bash
|
||||
for repetition in 1 2; do
|
||||
for elements in 16 64; do
|
||||
for mode in get_tensor reuse; do
|
||||
CUDA_VISIBLE_DEVICES='' taskset -c 0 timeout 120s python \
|
||||
benchmarks/host_weight_runtime/safetensors_retention.py \
|
||||
--mode "$mode" --tensor-elements "$elements" \
|
||||
> "$mode-$elements-$repetition.json"
|
||||
done
|
||||
done
|
||||
done
|
||||
```
|
||||
|
||||
The diagnostic sets Torch to one CPU thread, warms up before the baseline,
|
||||
and reports preparation separately from timed loop work. Checkpoints run GC
|
||||
and report Linux private/anonymous memory, RSS, and open descriptors. It checks
|
||||
that transient tensor objects and the final cached view are released. Linux
|
||||
`/proc/self/smaps_rollup` must be readable; invalid arguments and missing proc
|
||||
support fail explicitly. Keep raw JSON and the repository snapshot with results.
|
||||
|
||||
Compare the final loop sample with the warmed baseline, then inspect the closed
|
||||
sample. Private-memory growth after Python tensor objects disappear is evidence
|
||||
of retention in the dependency/allocator path, not proof of a leak's root cause.
|
||||
RSS can include file-backed pages; it is not a private-memory metric. Samples
|
||||
also include small diagnostic bookkeeping allocations. Do not assert a portable
|
||||
memory threshold or turn this synthetic loop into a per-request service estimate.
|
||||
|
||||
Do not drop shared caches, trim allocators, or add serving-time GC to make the
|
||||
numbers look smaller. A dependency replacement or version change requires its
|
||||
own reproducible comparison and correctness/lifecycle validation.
|
||||
Reference in new issue
Block a user