This module carried a retraction saying the measured saving from right-sizing
the cache was 188 MB. That retraction was itself measured through a path that
ignored the knob: CpuPlatform.check_and_update_config overwrote
kv_cache_memory_bytes from VLLM_CPU_KVCACHE_SPACE unconditionally, so the
"32x shrink" never happened and both arms ran the same cache.
With the budget actually honoured, 3 interleaved passes per arm, both filling
a 1536-token context:
1 GiB budget 3264 MB resident
192 MiB budget 2421 MB resident -> 843 MB
And the reasoning behind "the pages are never touched" is wrong too. mincore(2)
reports the cache fully in core: 1024.1 MB resident of 1023.8 allocated, and
191.1 of 190.8 when sized down. It is not address space, it is memory.
843 MB is the largest single lever measured on this model -- more than the
single weight layout (615 MB), the blocked repack (489 MB peak), or the RoPE
table (618 MB).
The general lesson, recorded in the report: a lever tested through a path that
silently ignores it measures nothing, and "no effect" is the most dangerous
result to accept without first checking the knob was connected.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L1ax5wYLA6XbqEL1i8L9gZ
