[Model] Make the dummy-run recipe feature-neutral
Per review, the warmup recipe is a property of the model's request geometry, not of distributed layerwise offload: rename dlo_dummy_run_recipe to dummy_run_recipe and drop the DLO gate in _make_dummy_request. A model whose geometry needs recipe-specific sampling arguments warms up at startup whatever features are enabled — under DLO the generation also primes the component cache-retention budget; without it, the same warmup covers any other first-request cost. Validated both ways on L20X: DLO boot unchanged (releases stay in the startup window) and a non-DLO MiniMax-H3 boot now runs the warmup and serves normally. Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
This commit is contained in:
1 parent
33297bdb9d
commit
e1c4b6153a
6 files changed
+53
-49
No files matched your search
@@ -185,9 +185,10 @@ resolution. It starts at zero: a cold start pays one release per observation
|
||||
step while the peak climbs, after which boundaries retain steadily, and a
|
||||
workload change whose cached footprint grows by more than the margin pays one
|
||||
further release while the new peak lands. Models whose request geometry opts
|
||||
out of the generic dummy run can declare `dlo_dummy_run_recipe`; MiniMax-H3
|
||||
declares one minimal t2va generation, so under DLO the engine's startup
|
||||
warmup runs it and the cold-start flushes land before real traffic. The budget is capped at a quarter
|
||||
out of the generic dummy run declare a feature-neutral `dummy_run_recipe`
|
||||
instead; MiniMax-H3 declares a maximum-duration t2va generation, so the
|
||||
engine's startup warmup runs it and, under DLO, the cold-start flushes land
|
||||
before real traffic. The budget is capped at a quarter
|
||||
of device capacity so a spurious observation cannot monopolize a small
|
||||
device, and detaching the last attached stager resets the learned peak so one
|
||||
workload never permanently influences another. No user-chosen fraction, no
|
||||
|
||||
Reference in new issue
Block a user