anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

probe: Qwen-Image conditioning-KV spill probe on RTX 3090 / RTX 5080 (issue 2) #4

Open
anodyine wants to merge 2 commits from ISSUE-2 into main
pull from: ISSUE-2
merge into: anodyine:main
Conversation 4 Commits 2 Files Changed 11
+2498
2 Commits
Author SHA1 Message Date
admin 4f5f99d8c6 probe(review): fp8 serve row + 3-ref fixture + fp8 quality-collapse diagnosis
Build Wheel / build (3.12) (pull_request) Failing after 50s
Details
Build Wheel / build (3.11) (pull_request) Failing after 50s
Details
pre-commit / pre-commit (pull_request) Failing after 54s
Details 
Review follow-up on PR #4: run --quantization fp8 through the served
path on the 3090 (the flag issue 1 validated for capacity), add the
untested 3-reference fixture, and measure all counts.

Capacity: confirmed — all of 1/2/3/4 references fit and run (4-ref
44.1-44.4s, server peak 23,522MB, nvidia-smi max 23,983MiB; Marlin
W8A16 is faster than BF16, 3.3x the Diffusers nocache baseline).

Quality: every fp8 output is structurally broken on this commit and
card (~97% transparent RGBA, grey RGB, PSNR 7-9dB) — on the edit path
at every reference count and on T2I alike (issue 1's fp8 artifacts
reinspected: same collapse; that run validated rc+image, not output
structure). img_mlp exclusion does not fix it; transformer-only fp8
also collapses, pinning the culprit on the Marlin-quantized DiT.
3090 verdict stays not-built; upstream fix order is fp8 output
correctness before any capacity role for it.
 2026-10-08 06:27:36 -04:00
admin 67eb7fdb48 probe: Qwen-Image conditioning-KV spill probe on RTX 3090 / RTX 5080 (issue 2)
Build Wheel / build (3.11) (pull_request) Failing after 33s
Details
Build Wheel / build (3.12) (pull_request) Failing after 40s
Details
pre-commit / pre-commit (pull_request) Failing after 41s
Details 
Evidence-only change: probe kit (serve wrapper, API client, PSNR
comparator, offline fp8_v driver), per-host run summaries, and the
findings report. No code changes to the stack.

Verdicts: not-built on both cards. On the 3090 the served reference
ceiling is two condition images with the conditioning cache on
(2.2x the Diffusers nocache baseline, pair PSNR mean 27.57 dB);
four references OOM on the resident BF16 working set, not the KV
(~0.3 GiB/reference, per-request). On the 5080 the server cannot
initialize under any flag: init_parameters materializes the
unquantized skeleton on-device before quantization/offload engage.
Also documents that the recipe's deploy-YAML extras route for
prefix_kv_cache_dtype does not reach the served transformer on
8484d85d (offline Omni engine kwarg does).
 2026-10-08 00:58:29 -04:00