3 Commits

Include renames

Author	SHA1	Message	Date
tzhouamandClaude Fable 5.1	37c0ea9a6b	[Edge][WP8] Per-step overhead attribution, edge scheduling helpers, CPU thread sweep ... - vllm_omni/edge/step_stats.py: process-local counters enabled by VLLM_OMNI_STEP_STATS_DIR (scheduler schedule/update_from_output, engine-core step, chunk hop enqueue->put / put->get / get, orchestrator dispatch), dumped per process and merged into an attribution table (share of the 80 ms frame). Hooks: stage_engine_core_proc (instrument after construction, dump at exit), chunk_transfer_adapter (put/get timing, put->get via the shm lock-file mtime), orchestrator dispatch loop. Bench: stream_latency_bench.py --step-stats. - vllm_omni/edge/scheduling.py: playback slack, prefill-throttle decision and the no-preemption KV admission rule (exact bytes/token from the talker KV geometry); adapt.derive_overrides now sizes max_num_seqs with it and auto_profile passes the geometry read from the model config. - Engine re-reads parallel_stage_init after apply_deploy_profile so the edge profile's orchestrator default takes effect (was read before the profile). - benchmarks/tts/cpu_thread_sweep.py: CPU-only 8/16/32-thread sweep (+inductor) with per-step attribution. - Tests: tests/core/test_edge_scheduling.py, tests/edge/test_step_stats.py, tests/benchmarks/test_cpu_thread_sweep.py, profile default test. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>	2026-09-09 14:55:55 +08:00
tzhouamandClaude Fable 5.1	073d31b9cd	[Edge][WP2] Add initialization timeline and cold-start profiler ... * vllm_omni/engine/init_timeline.py: InitTimeline phase collector; engine phases (stages_init, g3_device_lock, g2_spawn/ready) and worker phases (proc_entry, engine_core_init, init_device, load_model/load_weights, talker_mtp_init, attn_prewarm, cudagraph_capture, talker_mtp_capture, code2wav_cudagraph, predictor_setup_compile/warmup/cudagraph) recorded via worker_phase/timed_phase; workers append JSONL under VLLM_OMNI_INIT_TIMELINE_DIR (inherited at spawn) and the engine merges them. Enabled by VLLM_OMNI_INIT_TIMELINE=<json> or log_stats; table logged at INFO, partial table on orchestrator startup timeout, suggested timeouts logged. * omni_base.py: named deploy profiles default to stage_init_timeout=900 / init_timeout=1500 unless given explicitly. * gpu_model_runner._init_talker_mtp tolerates max_cudagraph_capture_size=None; gpu_ar_model_runner.shutdown skips CUDA graph teardown on CPU. * benchmarks/tts/init_profile.py: runs stream_latency_bench --init-only per configuration (default, cold_cache, eager_stage1, eager_both, parallel_stage_init, edge, ...) and tabulates phase durations. * stream_latency_bench: --parallel-stage-init / --deploy-profile flags; the resource poller no longer shadows Thread._stop and skips GPU sampling on CPU. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>	2026-09-09 11:01:39 +08:00
tzhouamandClaude Fable 5.1	b70207f57f	[Edge][WP0] Add offline streaming-latency benchmark for chunked TTS ... Adds benchmarks/tts/stream_latency_bench.py (AsyncOmni driver reusing the Qwen3-TTS example's prompt builder; --model override, warm-up/repeat, --init-only, per-chunk arrivals, peak RSS and per-process GPU memory) and benchmarks/tts/stream_latency_metrics.py (TTFA, underrun-free playback start, stall at TTFA, RTF streamed/total, chunk gaps, percentiles) with CPU unit tests. Results go under benchmarks/tts/edge_results/stream_latency/. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>	2026-09-09 10:24:53 +08:00