[Model] Prime DLO cache retention with a startup warmup

Models whose request geometry opts out of the generic dummy run (H3: t2va
needs an explicit aspect ratio and a bounded duration) can declare a
dlo_dummy_run_recipe. Under distributed layerwise offload the engine's
startup warmup runs it, so the learned cache-retention budget is primed
before real traffic instead of on the first request.

H3 declares a maximum-duration t2va generation: duration is the only
workload variable at the fixed 768 short edge, so the learned peak
dominates every production request and no real request re-learns. Validated
on L20X: all cold-start releases happen during the startup warmup, zero
during requests; first request 15.91 s vs 16.7 s unwarmed, steady 14.9 s;
startup gains one max-length generation (~10-20 s). Non-DLO deployments
keep the opted-out generic warmup unchanged.

Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
This commit is contained in:
hsliuustc0106 committed 2026-08-26 15:06:08 +08:00
1 parent c9bb65902e
commit f4394f1d3d
6 files changed
+201 -3

No files matched your search

@@ -67,6 +67,16 @@ omni = Omni(
| `--host-weight-runtime-root PATH` | Writable node-local HWR store shared by workers in one storage domain; required for `preferred` and `required` | unset |
| `--dlo-host-registration-limit-gib N` | Optional per-worker ceiling for registering an HWR mmap; zero adds no ceiling | `0` |
## Startup warmup
Under DLO, models that need model-specific sampling arguments for a warmup
generation (currently MiniMax-H3: `t2va`, an explicit aspect ratio) run one
at engine startup. It runs at the maximum output duration so the component
allocator-cache retention policy learns a footprint that covers every
production request: the cold-start flushes happen during startup and no real
request pays them. The tradeoff is one maximum-length generation added to
boot time; on deployments without DLO the behavior is unchanged.
## Host-weight loading
The diffusion loader chooses host storage before DLO is enabled. It first