# Issue 2 probe: Qwen-Image conditioning KV on RTX 3090 (24 GB) and RTX 5080 (16 GB) Probed 2026-10-07/08 on `cachyos-alexander` (RTX 3090, sm_86) and `cachyos-9900x-o11d` (RTX 5080, sm_120), both at upstream main `8484d85d1`, vLLM-Omni 0.31.0 recipe (Python 3.13 venv, torch 2.13.0+cu130, transformers 5.14.1, diffusers git `122b1e11`), served from the digest-pinned snapshot `Qwen/Qwen-Image-2.1@790c9263` with `HF_HUB_OFFLINE=1`. Eager mode, one request at a time, fixed seeds, 20 steps, `true_cfg_scale 1.0`, no negative prompt — mirroring the sealed Diffusers attempt `2026-10-07T170729Z-q3-4r` (profile `image-q3-int8-dit-20-steps-nocache-4ref`) fixture-for-fixture: same frozen references-suite-2 images, same rendered brief-prose/1 prompts, same seeds 2301–2304, same per-fixture resolutions. ## Verdicts **RTX 3090 (24 GB): not-built.** No host-DRAM spill path for the conditioning KV exists anywhere in the pipeline (verified by code read and by exhausting every reachable knob). At BF16 the served reference ceiling is two condition images at `--enable-cpu-offload` — with the conditioning cache ON, at 2.2× the Diffusers nocache baseline's speed. Four references OOM inside the DiT forward under every BF16 flag combination, and the binding constraint is the resident BF16 working set plus a fixed ~490 MiB image-MLP activation, not the conditioning KV: the per-request cache contributes only roughly 0.25–0.3 GiB per reference (see "KV size" below). FP8 weight quantization (`--quantization fp8`) does free the ~6 GiB the arithmetic calls for — all of one, two, three, and four references then fit and run (4-ref: 44.1–44.4 s, server peak 23,522 MB, nvidia-smi series max 23,983 MiB) — **but on this commit and card the FP8 path produces structurally broken output** (~96.5–97% transparent RGBA, grey noise RGB, PSNR 7–9 dB), for the edit path at every reference count and for T2I alike (see "FP8 quality collapse"). So the verdict stands: no usable path to four references, and none that spills the KV. **RTX 5080 (16 GB): not-built, stronger — the server cannot initialize.** Six init attempts (default, cpu-offload, cpu+layerwise offload, fp8, fp8+expandable-segments, int8) all OOM at the same late-init 96 MiB allocation with the card full. The failure is in `vllm_omni/diffusion/models/utils.py:159-168` (`init_parameters`): the unquantized parameter skeleton is allocated on-device at construction, before any quantization or offload path engages, so no runtime flag can shrink the load-time peak. The reference path is unreachable on this card with this stack; its one-reference Diffusers ceiling (B2g) relied on a torchao-int8 + group-offload load path that never materializes the full BF16 model on the GPU. The conditioning-KV question is moot on this card. ## Where the cache lives (issue scope item 2) - Created per request at `vllm_omni/diffusion/models/qwen_image_21/pipeline_qwen_image_21.py:1204-1207` (`state.extra["kv_cache"]`, one dict per DiT block — 32 — keyed by CFG branch `cond`/`uncond`), valid because `causal_condition=True` makes text + condition-image K/V timestep-independent. - Written at prefill (first denoise step) and read at decode in `qwen_image_21/.../qwen_image_21_transformer.py:538-598` (`branch_cache["key"] = key[:, :cache_write_len].clone()` and the decode-side `torch.cat` prepend). - Batched mixed-phase assembly/scatter at `pipeline_qwen_image_21.py:1218-1265` (`_assemble_kv_cache`) and `:1303-1325` (`_scatter_kv_cache`); decode-graph static copies in `qwen_image_21/decode_graph.py` (inactive here — eager). - **Allocator**: plain `torch.Tensor` `.clone()`s on the GPU. Not vLLM's paged KVCacheManager — the model deliberately runs `DENSE_LEGACY` (`recipes/Qwen/Qwen-Image-2.1.md:318-329` documents the rejection of `PAGED_SCHEDULER`). Optional FP8-E4M3 storage (K+V or V-only) via `extras.prefix_kv_cache_dtype`. - **Lifetime**: per-request only; freed by refcount when the runner drops the request state (`diffusion/worker/diffusion_model_runner.py:1034ff`). No cross-request retention, no LRU, no dedup — unlike Diffusers' helper cache, nothing persists between requests. - **Flags that reach it**: `extras.prefix_kv_cache_dtype` (`fp8` / `fp8_v`; deploy-YAML stage extras or offline `Omni(...)` kwarg — see the propagation gap below), `--enforce-eager` / `--enable-cuda-graph-decode` (static-buffer copy), `cuda_graph_max_decode_graphs`. `--swap-space` and `--cpu-offload-gb` **do not exist** in this fork; `--gpu-memory-utilization` sizes only the unrelated paged-scheduler path. The weight-offload flags (`--enable-cpu-offload`, `--enable-layerwise-offload`) never move KV tensors. - **Spill path**: none. A host-DRAM contribution would go at the prefill write / decode read (`qwen_image_21_transformer.py:541-598` — allocate on pinned CPU, prefetch before the decode `torch.cat`) or at the batch boundary helpers (`pipeline_qwen_image_21.py:1218-1325`), preserving the request-owns-independent-storage invariant that the decode-graph weakref identity check at `decode_graph.py:350-363` depends on. ## KV size (empirical) Between one and two references the per-request peak moves by only ~200–450 MiB (cpuoff: 23,444→23,638 MB server-reported; cpuoff+ expandable: 21,458→21,896 MB), i.e. ≈0.25–0.3 GiB per condition image — roughly 64 KiB/prefix-token — versus the 2.8 GiB per reference the helper-system measured Diffusers retaining (multi-model-helper-server #160). The cache is per-request and lean; it is not what blocks four references. ## Run matrix — RTX 3090 All runs: `vllm serve --omni --port 8091 --enforce-eager --served-model-name Qwen/Qwen-Image-2.1 [+ flags]`, readiness via `/health`, VRAM via 1 Hz `nvidia-smi` series; per-run artifacts under `~/Projects/vllm-omni-val/issue2/runs/