0 Fork 0

Files

T

tzhouamandClaude Fable 5.1 37c0ea9a6b [Edge][WP8] Per-step overhead attribution, edge scheduling helpers, CPU thread sweep

- vllm_omni/edge/step_stats.py: process-local counters enabled by
  VLLM_OMNI_STEP_STATS_DIR (scheduler schedule/update_from_output, engine-core
  step, chunk hop enqueue->put / put->get / get, orchestrator dispatch), dumped
  per process and merged into an attribution table (share of the 80 ms frame).
  Hooks: stage_engine_core_proc (instrument after construction, dump at exit),
  chunk_transfer_adapter (put/get timing, put->get via the shm lock-file mtime),
  orchestrator dispatch loop. Bench: stream_latency_bench.py --step-stats.
- vllm_omni/edge/scheduling.py: playback slack, prefill-throttle decision and
  the no-preemption KV admission rule (exact bytes/token from the talker KV
  geometry); adapt.derive_overrides now sizes max_num_seqs with it and
  auto_profile passes the geometry read from the model config.
- Engine re-reads parallel_stage_init after apply_deploy_profile so the edge
  profile's orchestrator default takes effect (was read before the profile).
- benchmarks/tts/cpu_thread_sweep.py: CPU-only 8/16/32-thread sweep (+inductor)
  with per-step attribution.
- Tests: tests/core/test_edge_scheduling.py, tests/edge/test_step_stats.py,
  tests/benchmarks/test_cpu_thread_sweep.py, profile default test.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

2026-09-09 14:55:55 +08:00

1.4 KiB

Raw Blame History

Edge benchmark results

Committed JSON produced by the edge branch benchmarks (kept small; WAVs are never committed):

stream_latency/<model>/<deploy>/<timestamp>[_tag].json — stream_latency_bench.py output (schema 1): meta (model, deploy config, git SHA, GPU), init (init_s, optional init timeline), resources (peak RSS, peak GPU MiB, sampling mode), requests[] (per-chunk arrivals, TTFA, playback-start, stall at TTFA, RTF streamed / total) and summary (p50/p90/max).
init_profile/<timestamp>/summary.json — init_profile.py comparison of initialization phases per configuration.
hw_emulation/<timestamp>/summary.json — hw_emulation.py matrix (emulated hardware class x TTFA / playback-start / RTF / RSS; per-case bench JSON, hw_profile.json and init timeline next to it).
cpu_sweep/<timestamp>/summary.json — cpu_thread_sweep.py (CPU-only 8/16/32 threads + inductor variant, with per-step attribution).
stream_latency/step_stats/<timestamp>/*.json — raw per-process counters from --step-stats (merged into the bench JSON under step_stats).
codec_stream/ — codec_stream_e2e.py parity (server PCM vs client-decoded codec tokens, SNR) and client latency.
stream_latency/timelines/ — init timelines (VLLM_OMNI_INIT_TIMELINE) of the committed runs.

Metric definitions live in benchmarks/tts/stream_latency_metrics.py.

1.4 KiB Raw Blame History

Edge benchmark results

1.4 KiB

Raw Blame History