[Model] Prime DLO cache retention with a startup warmup

Models whose request geometry opts out of the generic dummy run (H3: t2va
needs an explicit aspect ratio and a bounded duration) can declare a
dlo_dummy_run_recipe. Under distributed layerwise offload the engine's
startup warmup runs it, so the learned cache-retention budget is primed
before real traffic instead of on the first request.

H3 declares a maximum-duration t2va generation: duration is the only
workload variable at the fixed 768 short edge, so the learned peak
dominates every production request and no real request re-learns. Validated
on L20X: all cold-start releases happen during the startup warmup, zero
during requests; first request 15.91 s vs 16.7 s unwarmed, steady 14.9 s;
startup gains one max-length generation (~10-20 s). Non-DLO deployments
keep the opted-out generic warmup unchanged.

Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
This commit is contained in:
hsliuustc0106 committed 2026-08-26 15:06:08 +08:00
1 parent c9bb65902e
commit f4394f1d3d
6 files changed
+201 -3

No files matched your search

@@ -184,7 +184,10 @@ the weight-reuse payload plus the transient footprint whatever the
resolution. It starts at zero: a cold start pays one release per observation
step while the peak climbs, after which boundaries retain steadily, and a
workload change whose cached footprint grows by more than the margin pays one
further release while the new peak lands. The budget is capped at a quarter
further release while the new peak lands. Models whose request geometry opts
out of the generic dummy run can declare `dlo_dummy_run_recipe`; MiniMax-H3
declares one minimal t2va generation, so under DLO the engine's startup
warmup runs it and the cold-start flushes land before real traffic. The budget is capped at a quarter
of device capacity so a spurious observation cannot monopolize a small
device, and detaching the last attached stager resets the learned peak so one
workload never permanently influences another. No user-chosen fraction, no