An independent review (gpt-6-astra via the Codex hook) found that two modules
still ship claims this study had already withdrawn or that the vLLM source
contradicts. Documentation in executable files is how a retracted number gets
quoted again, so these are corrected at the source:
kv_budget.py claimed right-sizing the cache saved 8329 -> 5254 MB. That was
read from VmRSS (which counts the mmap'd checkpoint and moved 1-2 GB between
identical runs) and the two arms were different model files. On RssAnon the
saving is 188 MB. The retraction is now stated in the docstring, together with
why the number is small -- the benchmark generated 8 tokens, so no cache was
filled -- and the 1323 MiB flat-vs-hybrid difference at 32k that makes this
worth re-measuring on a benchmark that does fill it.
weight_path.py:
- bits_per_weight 4.16 -> 4.25 in all three 4-bit paths. 4 bits plus a bf16
scale and zero per 128-weight group is 4 + 32/128; 4.16 matched no
arithmetic and made the llama.cpp fidelity comparison look budget-matched
when it is not (Q4_K is 4.5 bits over groups of 32, Q4_K_M promotes some
tensors further).
- fused_cpu_norms True -> False in all three. Fused CPU RMSNorm is 3.6-4.5x
faster in isolation and slower in situ; VLLM_OMNI_CPU_FUSED_NORMS already
defaults to "0" for that reason, so the metadata contradicted the default.
- "tinygemm is the only option without AMX" is false: vLLM 0.28 selects
ops.cpu_gemm_wna16 whenever use_w4a8 is off (mixed_precision/cpu.py:202).
Unmeasured here, so the selector is unchanged, but the claim is not.
- W4A8's symmetric grid is our exporter's "sym": true, not a kernel limit --
_process_gptq_weights_w4a8 uses checkpoint zero points when
config.zero_points is set (same file, :98). An asymmetric export is a config
change and the cheapest untried fidelity experiment.
- kv_budget's flat-accounting observation is flagged for re-checking, since
CpuPlatform.support_hybrid_kv_cache() returns True (platforms/cpu.py:545).
Both fields now carry docstrings saying what they mean, so they cannot drift
back silently. tests/edge: 18 passed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
