[Model] Derive the retention budget from observed boundary peaks

The staged-bytes seed left the transient half of the working set to a
guessed multiplier, and that half scales with the running workload rather
than the model topology. Track the largest cached-but-unallocated value
observed at any component boundary instead: the budget is the larger of
the seed and the observed peak times a 1.25 margin, still capped at a
quarter of device capacity. A workload that outgrows the seed pays one
release while the observation lands, then retains steadily. No warmup
flag is needed and no user-chosen fraction exists.

Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>

docs/design/feature/offloader/distributed_layerwise_offload.md

@@ -177,17 +177,19 @@ component can be reused by the next. The DiT prefetch path and its two shared
 device buffers are unchanged.
 
 After a component is offloaded, cached-but-unallocated memory is retained only
-while it stays within a budget derived from the model topology — the staged
-component byte total reported by the attached `PinnedModuleStager` instances
-times a fixed headroom multiplier — and at least 5% of device capacity
-remains physically free. The derived budget is capped at a quarter of device
-capacity so a large staged total cannot monopolize a small device, and a
-detached stager returns its bytes to the budget. Crossing either bound
-releases the allocator cache, so the budget scales with the staged components
-rather than with device capacity and needs no user-chosen fraction. Each
-release decision logs the staged byte total, the effective budget and cap,
-cached-but-unallocated bytes, and free memory, so a target run can validate
-the headroom multiplier directly.
+while it stays within a budget derived from the workload itself and at least
+5% of device capacity remains physically free. The budget starts from the
+model topology — the staged component byte total reported by the attached
+`PinnedModuleStager` instances times a seed multiplier — and then tracks the
+largest cached value observed at any component boundary: if a workload's
+transient footprint outgrows the seed, the first over-budget boundary
+releases once, the observation raises the budget to the observed peak times
+a margin, and later boundaries retain. The budget is capped at a quarter of
+device capacity so neither a large staged total nor a spurious observation
+can monopolize a small device, and a detached stager returns its bytes to the
+seed. No user-chosen fraction and no warmup flag are needed. Each decision
+logs the staged byte total, observed peak, effective budget and cap,
+cached-but-unallocated bytes, and free memory.
 Missing allocator telemetry also releases it conservatively. Component or
 staging failure forces a release, and an out-of-memory allocation gets one
 retry after release. This is a component-local policy rather than a global