Files
vllm-omni/probe/issue-5/FINDINGS.md
T
admin bf65a41ceb
Build Wheel / build (3.11) (pull_request) Failing after 42s
Build Wheel / build (3.12) (pull_request) Failing after 42s
pre-commit / pre-commit (pull_request) Failing after 37s
probe(evidence): int8-wo capacity matrix, structure gate, PSNR, Marlin 8-bit NaN isolation (issue 5)
2026-10-08 08:00:51 -04:00

184 lines
11 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Issue 5 probe: int8 weight-only compressed-tensors DiT on the RTX 3090 (24 GB)
Probed 2026-10-08 on `cachyos-alexander` (RTX 3090, sm_86, driver 615.71.09,
CUDA 13.4), at upstream main `8484d85d1` plus this branch's two loader/factory
fixes, vLLM-Omni 0.31.0 recipe (Python 3.13 venv, torch 2.13.0+cu130,
transformers 5.14.1, diffusers git `122b1e11`, compressed-tensors 0.17.0).
The DiT was quantized **offline** from the sealed snapshot
`Qwen/Qwen-Image-2.1@790c9263` to int8 weight-only (W8A16), per-output-channel
symmetric, stored as a compressed-tensors pack-quantized checkpoint; encoder
and VAE stay BF16 on disk via the artifact's per-component config wrapper.
The probe mirrors the sealed Diffusers attempt `2026-10-07T170729Z-q3-4r`
fixture-for-fixture (same frozen references-suite-2 images, same rendered
prompts, seeds 2301–2304, 20 steps, `true_cfg_scale 1.0`).
## Verdict
**Capacity: delivered. Quality: collapsed in the serve kernel — the int8
sidestep does not rescue 3–4 references on this card with this stack, and the
blocker is now sharply isolated: the 8-bit Marlin W8A16 path itself.**
- Every step of the artifact chain is verified correct: the quantizer's
round-trip reproduces the sealed BF16 weights at ~1.1% rel-L2
(`evidence/rtx3090/artifact-verify.txt`), and the serve path loads the
packed tensors **bitwise-identically** — including the fused q/k/v stacked
shards — with finite scales before and after the Marlin repack.
- Serving works mechanically: init ~40 s, all 16 capacity-matrix requests
(1/2/3/4 references × seeds 2301–2304) complete at BF16-comparable speed
(4-ref 46.4 s; BF16 OOMs at 4 refs, online-FP8 was 44.3 s), peak 23,935 MiB.
- But the output is structurally broken on **every** counted run (gate FAIL
16/16: opaque 0.0–0.20 vs 1.0 at BF16; PSNR vs the sealed baseline
7.1–7.5 dB — the same collapsed band as online FP8). The images are
horizontal-stripe noise.
- A one-layer offline isolation (`marlin_isolate.py`) pins it: vLLM parallel
linears built exactly as the qwen_image_21 transformer builds them, loaded
with the artifact's verified tensors, emit **NaN** for all four layer
geometries (fused QKV, to_out, img_mlp proj/out), for both bf16 and fp16
activations — and do the same for **synthetic random int8 weights**, so the
checkpoint and its quantization are exonerated. The 8-bit Marlin W8A16 path
on this stack (sm_86, vllm 0.31.0, torch 2.13.0+cu130) is broken even though
`query_marlin_supported_quant_types` accepts `uint8b128` on the card, and
`choose_mp_linear_kernel` offers no non-Marlin W8A16 fallback on sm_86
(Machete is sm90+, ExLlama int4-only).
The torchao-int8-in-Diffusers prior (multi-model-helper-server #160, blinded
review within label noise of BF16) stands unchallenged: int8 quantization is
fine; the serve-path kernel is not. The online-FP8 collapse from PR #4 and
this int8 collapse now share one root suspect — the Marlin mixed-precision
path on this fork/card — with the fp8 case previously isolated to the
Marlin-quantized DiT and this run extending it to int8 with a fully verified
checkpoint on the far side of the kernel.
## Two serve-path fixes this issue required (in this PR)
1. **Quant factory could not build compressed-tensors at all.**
`_normalize_method_name` canonicalizes `compressed-tensors` to
`compressed_tensors`, which then missed vLLM's hyphenated registry — so
both `--quantization compressed-tensors` and checkpoint auto-detection
raised "Unknown quantization method". The registry lookup now resolves
both spellings, and registry configs whose `__init__` does not take the
serialized schema fall back to `from_config` (`vllm_omni/quantization/factory.py`,
tests in `tests/quantization/test_quant_factory.py`).
2. **The diffusion loader misclassified pre-quantized checkpoints as online
quantization under CPU offload.** With `--enable-cpu-offload` it loaded
every component on the GPU before offloading (encoder 17 GiB + VAE + the
packed DiT skeleton > 24 GiB → init OOM). Serialized formats are now
detected (`is_prequantized_config`, component-wrapper aware) and load
directly on the host, with each quantized module round-tripped through the
accelerator only for the post-load Marlin repack
(`vllm_omni/diffusion/model_loader/diffusers_loader.py`). With these two
fixes the artifact serves: init healthy in ~40 s, logs
`Per-component quantization: {'transformer': 'compressed-tensors'}`.
## Artifact (outside Git, under the helper state root)
`~/.local/state/multi-model-helper-server/state/image-service/artifacts/qwen-image-2.1-dit-int8wo/`
— quantized `transformer/` (6.76 GiB packed int8 + BF16 boundary layers;
224 tensors = 32 blocks × {to_q,to_k,to_v,to_out.0,proj,gate_layer,out};
6.98 B params quantized, 0.14 B BF16), `model_index.json`/`scheduler`/
`text_encoder`/`vae`/`processor` symlinked to the sealed snapshot, and
`MANIFEST.json` with inputs (snapshot path, revision, tree digest), versions,
per-file sha256, and quantization stats (mean rel-L2 1.10%, max 1.73%).
Built by `quantize_dit_int8wo.py` using compressed-tensors' own
`pack_to_int32` (uint8 biased by 128, 4 per int32) and `QuantizationConfig`
serialization, so the storage convention matches llm-compressor bit-for-bit.
`transformer/config.json` carries the quantization metadata wrapped
per-component (`{"transformer": {...}}`) so auto-detection quantizes the DiT
only — the encoder and VAE resolve to None and stay BF16
(the plain global form would hand the encoder a compressed-tensors config
it has no scales for).
## Run matrix — RTX 3090
Single run `int8wo`: `vllm serve <artifact> --omni --port 8091
--served-model-name Qwen/Qwen-Image-2.1 --enforce-eager
--enable-cpu-offload`, readiness via `/health`, VRAM via the 1 Hz
`nvidia-smi` series (issue 2's instrument). Two precursor runs OOM'd at init
for reasons fixed on this branch (see Verdict item 2); their logs survive
under `runs/int8wo-expandable/` on the host (the first attempt's log was
overwritten by the successful rerun in the same label).
| Refs | Fixture | Wall (mean, s) | Peak (MiB) | Gate | PSNR vs Diffusers (dB) |
| --- | --- | --- | --- | --- | --- |
| 1 | edit-expression-ysolde | 31.0 | 23,935 (run max) | FAIL 4/4 | 7.14 |
| 2 | edit-compose-pair | 36.6 | ” | FAIL 4/4 | 7.46 |
| 3 | edit-compose-three | 41.4 | ” | FAIL 4/4 | no sealed row (derived fixture) |
| 4 | edit-compose-four | 46.4 | ” | FAIL 4/4 | 7.11 |
BF16 comparison rows (issue 2): 1-ref 34.2 s, 2-ref 36.7 s, 4-ref OOM.
Online-fp8 (broken output, capacity reference): 29.9 / 34.1 / 38.9 / 44.3 s.
So the int8 artifact matches the fp8 capacity/speed profile exactly —
including its broken-output signature (PSNR band 7.0–7.5 dB) — while the
checkpoint side is proven numerically sound, moving the defect wholly into
the kernel.
## Structure gate and quality read
- Gate rule (issue 1's lesson): `opaque_fraction ≥ 0.99` and `mean(rgb_std)
≥ 10`; BF16 outputs measure 1.0 opaque, 26–29 dB. Every int8-artifact run
fails (opaque 0.0–0.20, alpha mean 114–137; rgb_std spans 3–93 — several
outputs have healthy RGB spread yet near-total transparency, and one
(seed 2302) is near-flat grey like the fp8 signature). Outputs are
horizontal-stripe noise on visual inspection.
- PSNR vs the sealed 4-reference Diffusers baseline (RGBA-on-RGBA,
#1's comparator): 7.14 / 7.46 / 7.11 dB mean for 1/2/4 refs — the
collapsed band, not the 26–29 dB BF16-row parity the issue hoped for.
## Kernel isolation (the issue's "different kernels" prior, answered)
`marlin_isolate.py` builds vLLM `QKVParallelLinear` / `RowParallelLinear` /
`ColumnParallelLinear` exactly as `qwen_image_21_transformer.py` does, loads
the artifact tensors through the production weight-loader machinery
(bitwise-verified after load, fused q/k/v shards included), runs
`process_weights_after_loading` (the Marlin repack) on the GPU, and compares
`layer(x)` against `float32 x @ dequant(W)ᵀ` computed from the artifact's own
unpacked weights:
| Layer | bf16 acts | fp16 acts | packed bitwise | scales post-repack |
| --- | --- | --- | --- | --- |
| attn.to_qkv (fused) | NaN | NaN | equal | finite (6.6e-4…1.2e-2) |
| attn.to_out.0 (row) | NaN | NaN | equal | finite |
| img_mlp.proj (column) | NaN | NaN | equal | finite |
| img_mlp.out (row, wide-in) | NaN | NaN | equal | finite |
Synthetic random int8 weights reproduce the NaN identically, and the runtime
capability query accepts `uint8b128` on this card — so this is a defect of
the 8-bit Marlin W8A16 path on this stack (repack or kernel; sm_86,
vllm 0.31.0, torch 2.13.0+cu130), not of the checkpoint, the quantizer, the
loader, or the fused-QKV stacking. No alternative W8A16 kernel exists on
sm_86 in this build to switch to.
## Reproducibility
Scripts in this directory: `quantize_dit_int8wo.py` (artifact + manifest),
`verify_artifact.py` (round-trip check), `serve.sh` / `probe.py` (adapted
issue-2 kit; probe records the structure gate inline per request), `gate.py`
(gate + capacity summarizer), `psnr.py`, `marlin_isolate.py` (kernel
isolation, `--synthetic` for the stack-only variant). All were pushed
verbatim to `~/Projects/vllm-omni-val/issue5/` on the host and run from the
validation clone checked out on this branch. Per-run serve/vram logs and all
output PNGs live under `~/Projects/vllm-omni-val/issue5/runs/int8wo/`;
condensed evidence in `evidence/rtx3090/`.
## Helper-system follow-through (input to multi-model-helper-server #27 — not implemented here)
- The 3090's 3–4 reference serving path remains blocked, now with a single
named suspect: the 8-bit Marlin W8A16 kernel path on sm_86 (this fork's
vllm 0.31.0 / torch 2.13.0 stack). Both quantized routes (online FP8,
offline int8 compressed-tensors with a verified checkpoint) collapse at the
same place. The unlock order for #27 candidacy is now: upstream Marlin
8-bit fix (or a non-Marlin W8A16 kernel on sm_86), then serving — capacity
and speed are already proven sufficient (4-ref at 46.4 s, 2.9× the sealed
Diffusers nocache baseline).
- The two vllm-omni loader/factory fixes on this branch are prerequisites for
any compressed-tensors artifact serving under CPU offload and are
independent of the kernel defect.
## Out-of-scope confirmation
No upstream PRs; no W8A8/int4/NVFP4 artifacts built; no helper-system
deployment evidence. Root-causing the Marlin 8-bit defect remains the
separate lane (PR #4 carries the fp8 isolation; this branch adds the int8 +
synthetic + bitwise-load isolation). The only code changes are the two
serve-path fixes required to make the issue's plan steps executable at all.