[Model] Prime DLO cache retention with a startup warmup

Models whose request geometry opts out of the generic dummy run (H3: t2va
needs an explicit aspect ratio and a bounded duration) can declare a
dlo_dummy_run_recipe. Under distributed layerwise offload the engine's
startup warmup runs it, so the learned cache-retention budget is primed
before real traffic instead of on the first request.

H3 declares a maximum-duration t2va generation: duration is the only
workload variable at the fixed 768 short edge, so the learned peak
dominates every production request and no real request re-learns. Validated
on L20X: all cold-start releases happen during the startup warmup, zero
during requests; first request 15.91 s vs 16.7 s unwarmed, steady 14.9 s;
startup gains one max-length generation (~10-20 s). Non-DLO deployments
keep the opted-out generic warmup unchanged.

Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
This commit is contained in:
hsliuustc0106 committed 2026-08-26 15:06:08 +08:00
1 parent c9bb65902e
commit f4394f1d3d
6 files changed
+201 -3

No files matched your search

@@ -704,6 +704,20 @@ class MiniMaxH3Pipeline(
"post_decode",
]
dummy_run_num_frames: ClassVar[int] = 0
# Startup warmup used only when distributed layerwise offload is enabled:
# the generic dummy run cannot satisfy t2va's required sampling arguments
# (explicit aspect ratio, bounded duration). Duration is the only workload
# variable at the fixed 768 short edge, so warming at the maximum makes the
# learned cache-retention peak dominate every production request and moves
# all cold-start flushes ahead of real traffic, at the cost of one
# max-length generation in boot time.
dlo_dummy_run_recipe: ClassVar[Mapping[str, Any] | None] = {
"task": "t2va",
"duration": MINIMAX_H3_MAX_OUTPUT_SECONDS,
"aspect_ratio": "16:9",
"num_inference_steps": 2,
"guidance_scale": 1.0,
}
# Only distilled releases pin a schedule, so the default keeps the legacy
# uniform path available to partially constructed pipelines.
_base_schedule_by_partition: ClassVar[Mapping[str, DMD2SigmaSchedule | None]] = {}