[Edge][WP2] Add initialization timeline and cold-start profiler
* vllm_omni/engine/init_timeline.py: InitTimeline phase collector; engine phases (stages_init, g3_device_lock, g2_spawn/ready) and worker phases (proc_entry, engine_core_init, init_device, load_model/load_weights, talker_mtp_init, attn_prewarm, cudagraph_capture, talker_mtp_capture, code2wav_cudagraph, predictor_setup_compile/warmup/cudagraph) recorded via worker_phase/timed_phase; workers append JSONL under VLLM_OMNI_INIT_TIMELINE_DIR (inherited at spawn) and the engine merges them. Enabled by VLLM_OMNI_INIT_TIMELINE=<json> or log_stats; table logged at INFO, partial table on orchestrator startup timeout, suggested timeouts logged. * omni_base.py: named deploy profiles default to stage_init_timeout=900 / init_timeout=1500 unless given explicitly. * gpu_model_runner._init_talker_mtp tolerates max_cudagraph_capture_size=None; gpu_ar_model_runner.shutdown skips CUDA graph teardown on CPU. * benchmarks/tts/init_profile.py: runs stream_latency_bench --init-only per configuration (default, cold_cache, eager_stage1, eager_both, parallel_stage_init, edge, ...) and tabulates phase durations. * stream_latency_bench: --parallel-stage-init / --deploy-profile flags; the resource poller no longer shadows Thread._stop and skips GPU sampling on CPU. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
1 parent
f804bbd679
commit
073d31b9cd
14 files changed
+885
-24
No files matched your search
@@ -177,8 +177,17 @@ class OmniBase(PDDisaggregationMixin):
|
||||
"You should instead, pass the keyword arguments used to initialize the engine args "
|
||||
"directly to this object's initializer."
|
||||
)
|
||||
stage_init_timeout = kwargs.pop("stage_init_timeout", 300)
|
||||
init_timeout = kwargs.pop("init_timeout", 600)
|
||||
# Tri-state: ``None`` means "not specified". Named deploy profiles
|
||||
# (e.g. ``deploy_profile="edge"``) initialize both stages on one device
|
||||
# and compile/capture at start, so they get larger defaults; the
|
||||
# init timeline (VLLM_OMNI_INIT_TIMELINE) reports what a run needed.
|
||||
stage_init_timeout = kwargs.pop("stage_init_timeout", None)
|
||||
init_timeout = kwargs.pop("init_timeout", None)
|
||||
if kwargs.get("deploy_profile"):
|
||||
stage_init_timeout = 900 if stage_init_timeout is None else stage_init_timeout
|
||||
init_timeout = 1500 if init_timeout is None else init_timeout
|
||||
stage_init_timeout = 300 if stage_init_timeout is None else stage_init_timeout
|
||||
init_timeout = 600 if init_timeout is None else init_timeout
|
||||
log_stats = kwargs.pop("log_stats", False)
|
||||
self._enable_ar_profiler = kwargs.pop("enable_ar_profiler", False)
|
||||
# NOTE: read-only lookup — must NOT pop. Popping here drops the key
|
||||
|
||||
Reference in new issue
Block a user