[Edge][WP8] Per-step overhead attribution, edge scheduling helpers, CPU thread sweep
- vllm_omni/edge/step_stats.py: process-local counters enabled by VLLM_OMNI_STEP_STATS_DIR (scheduler schedule/update_from_output, engine-core step, chunk hop enqueue->put / put->get / get, orchestrator dispatch), dumped per process and merged into an attribution table (share of the 80 ms frame). Hooks: stage_engine_core_proc (instrument after construction, dump at exit), chunk_transfer_adapter (put/get timing, put->get via the shm lock-file mtime), orchestrator dispatch loop. Bench: stream_latency_bench.py --step-stats. - vllm_omni/edge/scheduling.py: playback slack, prefill-throttle decision and the no-preemption KV admission rule (exact bytes/token from the talker KV geometry); adapt.derive_overrides now sizes max_num_seqs with it and auto_profile passes the geometry read from the model config. - Engine re-reads parallel_stage_init after apply_deploy_profile so the edge profile's orchestrator default takes effect (was read before the profile). - benchmarks/tts/cpu_thread_sweep.py: CPU-only 8/16/32-thread sweep (+inductor) with per-step attribution. - Tests: tests/core/test_edge_scheduling.py, tests/edge/test_step_stats.py, tests/benchmarks/test_cpu_thread_sweep.py, profile default test. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
1 parent
74acd66307
commit
37c0ea9a6b
16 files changed
+926
-16
No files matched your search
@@ -81,3 +81,20 @@ def test_modify_stage_config_keeps_relative_base_config_resolvable():
|
||||
cfg = load_deploy_config(Path(tmp_yaml))
|
||||
assert cfg.stages[0].max_num_seqs == 1
|
||||
assert cfg.stages[1].max_num_seqs == 4
|
||||
|
||||
|
||||
@pytest.mark.core_model
|
||||
@pytest.mark.cpu
|
||||
def test_edge_profile_carries_parallel_stage_init():
|
||||
from vllm_omni.entrypoints.utils import apply_deploy_profile, profile_orchestrator_defaults
|
||||
|
||||
args = apply_deploy_profile("Qwen/Qwen3-TTS-12Hz-0.6B-CustomVoice", {"deploy_profile": "edge"})
|
||||
assert args["parallel_stage_init"] is True
|
||||
assert profile_orchestrator_defaults(args["deploy_config"]) == {"parallel_stage_init": True}
|
||||
# explicit values win over the profile default
|
||||
args = apply_deploy_profile(
|
||||
"Qwen/Qwen3-TTS-12Hz-0.6B-CustomVoice", {"deploy_profile": "edge", "parallel_stage_init": True}
|
||||
)
|
||||
assert args["parallel_stage_init"] is True
|
||||
# unknown block keys are ignored
|
||||
assert "init_timeout" not in args
|
||||
Reference in new issue
Block a user