[Edge][CPU] Return the weight-loading churn to the OS: 7338 -> 3510 MB

Loading a quantized checkpoint allocates far more than the weights occupy.
`_process_gptq_weights_w4a8` unpacks 4-bit codes to int32 -- four bytes per
weight -- permutes, repacks, and drops the intermediates. For a 1.7 B model
that is gigabytes of short-lived allocation, correctly freed, and still
resident afterwards because a caching allocator keeps the arenas.

Measured on Spark-X2.5-1.7B, anonymous RSS:

    path               after load   after release   freed
    W4A8 (GPTQ)          7338 MB       3510 MB     3828 MB   (-52%)
    W4A16 (tinygemm)     5023 MB       3411 MB     1612 MB

The second column is the more useful finding: after the release both paths
land in the same place, so W4A8's apparent 2.3 GB penalty over W4A16 was never
a footprint difference -- it was the repack. W4A8 is therefore faster *and*
level on memory, and fidelity is the only remaining reason to pick W4A16.

Hooked to CPUWorker.load_model and compile_or_warm_up_model, since those churn
separately; `VLLM_OMNI_CPU_RELEASE_MEMORY=0` disables it. CPU only: on GPU the
weights are in device memory and host churn constrains nothing.

`anon_rss_bytes()` ships alongside because the metric matters as much as the
fix. VmRSS counts the mmap'd checkpoint's file pages and moved 1.2-2.0 GB
between *identical* runs on this host; RssAnon repeats to within 0.5 MB. An
earlier claim in this tree that KV sizing saved 3.1 GB came from reading VmRSS
across a comparison that also changed the model file; re-measured on RssAnon,
shrinking the KV cache 32x is worth 188 MB, because its pages are never
touched. The analysis note is corrected accordingly.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

vllm_omni/patch.py

@@ -694,3 +694,56 @@ def _patch_cpu_explicit_kv_cache_memory() -> None:
 
 
 _patch_cpu_explicit_kv_cache_memory()
+
+
+def _patch_cpu_release_memory_after_load() -> None:
+    """Return the weight-loading churn to the OS on CPU.
+
+    Loading a quantized checkpoint allocates far more than the weights occupy
+    -- the GPTQ path unpacks 4-bit codes to int32, four bytes per weight, to
+    permute and repack them -- and the intermediates, though freed, stay
+    resident because a caching allocator keeps the arenas. On Spark-X2.5-1.7B
+    that is 3828 MiB held after a W4A8 load and 1612 MiB after W4A16; released,
+    both land at ~3.5 GiB, so W4A8's apparent memory penalty over W4A16 was
+    entirely this and not a real difference in footprint.
+
+    Hooked in two places because they churn separately: after weights are
+    loaded and processed, and after the compile/warm-up pass.
+    """
+    from vllm.platforms import current_platform
+
+    if not current_platform.is_cpu():
+        return
+    if os.environ.get("VLLM_OMNI_CPU_RELEASE_MEMORY", "1") != "1":
+        return
+
+    from vllm.v1.worker.cpu_worker import CPUWorker
+
+    if getattr(CPUWorker, "_omni_release_memory", False):
+        return
+
+    from vllm_omni.edge.memory import release_after_load
+
+    base_load = CPUWorker.load_model
+    base_warm = CPUWorker.compile_or_warm_up_model
+
+    def load_model(self, *args, **kwargs):
+        out = base_load(self, *args, **kwargs)
+        release_after_load("weight load")
+        return out
+
+    def compile_or_warm_up_model(self, *args, **kwargs):
+        out = base_warm(self, *args, **kwargs)
+        release_after_load("warm-up")
+        return out
+
+    CPUWorker.load_model = load_model
+    CPUWorker.compile_or_warm_up_model = compile_or_warm_up_model
+    CPUWorker._omni_release_memory = True
+    _PATCH_LOGGER.info(
+        "CPU weight-load memory release installed (set "
+        "VLLM_OMNI_CPU_RELEASE_MEMORY=0 to disable)."
+    )
+
+
+_patch_cpu_release_memory_after_load()