Files
vllm-omni/tests
tzhouamandClaude Fable 5.1 94485152d2 [Edge][CPU] Predictor fast-path modes: torch.ao int8 (default), torchao weight-only + compiled step, fp32 control; parity metrics
torchao's dynamic-activation int8 is numerically wrong under torch.compile on
CPU (cosine 0.02 vs fp32; caused runaway generations), so it is not offered;
weight-only + compile is faithful but slower (78 ms vs 36 ms per frame for the
15 sub-steps), so torch.ao dynamic int8 stays the default. Parity mode now also
records top-5 overlap and KL vs the bf16 model; compiled-step equivalence test.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-10 02:44:29 +08:00
..