[Model] Make the dummy-run recipe feature-neutral

Per review, the warmup recipe is a property of the model's request
geometry, not of distributed layerwise offload: rename
dlo_dummy_run_recipe to dummy_run_recipe and drop the DLO gate in
_make_dummy_request. A model whose geometry needs recipe-specific sampling
arguments warms up at startup whatever features are enabled — under DLO
the generation also primes the component cache-retention budget; without
it, the same warmup covers any other first-request cost. Validated both
ways on L20X: DLO boot unchanged (releases stay in the startup window) and
a non-DLO MiniMax-H3 boot now runs the warmup and serves normally.

Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
This commit is contained in:
hsliuustc0106 committed 2026-08-26 15:48:07 +08:00
1 parent 33297bdb9d
commit e1c4b6153a
6 files changed
+53 -49

No files matched your search

@@ -185,9 +185,10 @@ resolution. It starts at zero: a cold start pays one release per observation
step while the peak climbs, after which boundaries retain steadily, and a
workload change whose cached footprint grows by more than the margin pays one
further release while the new peak lands. Models whose request geometry opts
out of the generic dummy run can declare `dlo_dummy_run_recipe`; MiniMax-H3
declares one minimal t2va generation, so under DLO the engine's startup
warmup runs it and the cold-start flushes land before real traffic. The budget is capped at a quarter
out of the generic dummy run declare a feature-neutral `dummy_run_recipe`
instead; MiniMax-H3 declares a maximum-duration t2va generation, so the
engine's startup warmup runs it and, under DLO, the cold-start flushes land
before real traffic. The budget is capped at a quarter
of device capacity so a spurious observation cannot monopolize a small
device, and detaching the last attached stager resets the learned peak so one
workload never permanently influences another. No user-chosen fraction, no
@@ -69,13 +69,13 @@ omni = Omni(
## Startup warmup
Under DLO, models that need model-specific sampling arguments for a warmup
generation (currently MiniMax-H3: `t2va`, an explicit aspect ratio) run one
at engine startup. It runs at the maximum output duration so the component
allocator-cache retention policy learns a footprint that covers every
production request: the cold-start flushes happen during startup and no real
request pays them. The tradeoff is one maximum-length generation added to
boot time; on deployments without DLO the behavior is unchanged.
Models whose request geometry needs model-specific sampling arguments for a
warmup generation (currently MiniMax-H3: `t2va`, an explicit aspect ratio)
run one at engine startup whatever features are enabled. It runs at the
maximum output duration so it covers every production request; under DLO
this means the component allocator-cache retention policy learns its
footprint during startup and no real request pays the cold-start flushes.
The tradeoff is one maximum-length generation added to boot time.
## Host-weight loading