tzhouamandClaude Fable 5.1
580a201631
vllm_omni/edge/decoder_export.py exports a stateless windowed decoder (context 72 frames + chunk, one program per ramp chunk size) via torch.export (.pt2, verified here on CPU for chunks 2 and 25 with the real 0.6B decoder: exact match vs eager, 460 MB fp32 per program) or ExecuTorch XNNPACK (.pte; written against the executorch 1.x API, not executed: package not installed, install is permission gated; QNN partitioner export-only/unverified). The WP5 client gains --decoder-backend executorch --pte-manifest for phone/NPU targets. benchmarks/tts/edge_results/summarize.py renders all committed JSON as tables. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Edge benchmark results
Committed JSON produced by the edge branch benchmarks (kept small; WAVs are never committed):
stream_latency/<model>/<deploy>/<timestamp>[_tag].json—stream_latency_bench.pyoutput (schema 1):meta(model, deploy config, git SHA, GPU),init(init_s, optional init timeline),resources(peak RSS, peak GPU MiB, sampling mode),requests[](per-chunk arrivals, TTFA, playback-start, stall at TTFA, RTF streamed / total) andsummary(p50/p90/max).init_profile/<timestamp>/summary.json—init_profile.pycomparison of initialization phases per configuration.hw_emulation/<timestamp>/summary.json—hw_emulation.pymatrix (emulated hardware class x TTFA / playback-start / RTF / RSS; per-case bench JSON,hw_profile.jsonand init timeline next to it).cpu_sweep/<timestamp>/summary.json—cpu_thread_sweep.py(CPU-only 8/16/32 threads + inductor variant, with per-step attribution).stream_latency/step_stats/<timestamp>/*.json— raw per-process counters from--step-stats(merged into the bench JSON understep_stats).codec_stream/—codec_stream_e2e.pyparity (server PCM vs client-decoded codec tokens, SNR) and client latency.stream_latency/timelines/— init timelines (VLLM_OMNI_INIT_TIMELINE) of the committed runs.
Metric definitions live in benchmarks/tts/stream_latency_metrics.py.