[Model] Make the dummy-run recipe feature-neutral
Per review, the warmup recipe is a property of the model's request geometry, not of distributed layerwise offload: rename dlo_dummy_run_recipe to dummy_run_recipe and drop the DLO gate in _make_dummy_request. A model whose geometry needs recipe-specific sampling arguments warms up at startup whatever features are enabled — under DLO the generation also primes the component cache-retention budget; without it, the same warmup covers any other first-request cost. Validated both ways on L20X: DLO boot unchanged (releases stay in the startup window) and a non-DLO MiniMax-H3 boot now runs the warmup and serves normally. Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
This commit is contained in:
1 parent
33297bdb9d
commit
e1c4b6153a
6 files changed
+53
-49
No files matched your search
@@ -69,13 +69,13 @@ omni = Omni(
|
||||
|
||||
## Startup warmup
|
||||
|
||||
Under DLO, models that need model-specific sampling arguments for a warmup
|
||||
generation (currently MiniMax-H3: `t2va`, an explicit aspect ratio) run one
|
||||
at engine startup. It runs at the maximum output duration so the component
|
||||
allocator-cache retention policy learns a footprint that covers every
|
||||
production request: the cold-start flushes happen during startup and no real
|
||||
request pays them. The tradeoff is one maximum-length generation added to
|
||||
boot time; on deployments without DLO the behavior is unchanged.
|
||||
Models whose request geometry needs model-specific sampling arguments for a
|
||||
warmup generation (currently MiniMax-H3: `t2va`, an explicit aspect ratio)
|
||||
run one at engine startup whatever features are enabled. It runs at the
|
||||
maximum output duration so it covers every production request; under DLO
|
||||
this means the component allocator-cache retention policy learns its
|
||||
footprint during startup and no real request pays the cold-start flushes.
|
||||
The tradeoff is one maximum-length generation added to boot time.
|
||||
|
||||
## Host-weight loading
|
||||
|
||||
|
||||
Reference in new issue
Block a user