diff --git a/probe/issue-5/FINDINGS.md b/probe/issue-5/FINDINGS.md
new file mode 100644
index 000000000..a9f268f24
--- /dev/null
+++ b/probe/issue-5/FINDINGS.md
@@ -0,0 +1,183 @@
+# Issue 5 probe: int8 weight-only compressed-tensors DiT on the RTX 3090 (24 GB)
+
+Probed 2026-10-08 on `cachyos-alexander` (RTX 3090, sm_86, driver 615.71.09,
+CUDA 13.4), at upstream main `8484d85d1` plus this branch's two loader/factory
+fixes, vLLM-Omni 0.31.0 recipe (Python 3.13 venv, torch 2.13.0+cu130,
+transformers 5.14.1, diffusers git `122b1e11`, compressed-tensors 0.17.0).
+The DiT was quantized **offline** from the sealed snapshot
+`Qwen/Qwen-Image-2.1@790c9263` to int8 weight-only (W8A16), per-output-channel
+symmetric, stored as a compressed-tensors pack-quantized checkpoint; encoder
+and VAE stay BF16 on disk via the artifact's per-component config wrapper.
+The probe mirrors the sealed Diffusers attempt `2026-10-07T170729Z-q3-4r`
+fixture-for-fixture (same frozen references-suite-2 images, same rendered
+prompts, seeds 2301–2304, 20 steps, `true_cfg_scale 1.0`).
+
+## Verdict
+
+**Capacity: delivered. Quality: collapsed in the serve kernel — the int8
+sidestep does not rescue 3–4 references on this card with this stack, and the
+blocker is now sharply isolated: the 8-bit Marlin W8A16 path itself.**
+
+- Every step of the artifact chain is verified correct: the quantizer's
+  round-trip reproduces the sealed BF16 weights at ~1.1% rel-L2
+  (`evidence/rtx3090/artifact-verify.txt`), and the serve path loads the
+  packed tensors **bitwise-identically** — including the fused q/k/v stacked
+  shards — with finite scales before and after the Marlin repack.
+- Serving works mechanically: init ~40 s, all 16 capacity-matrix requests
+  (1/2/3/4 references × seeds 2301–2304) complete at BF16-comparable speed
+  (4-ref 46.4 s; BF16 OOMs at 4 refs, online-FP8 was 44.3 s), peak 23,935 MiB.
+- But the output is structurally broken on **every** counted run (gate FAIL
+  16/16: opaque 0.0–0.20 vs 1.0 at BF16; PSNR vs the sealed baseline
+  7.1–7.5 dB — the same collapsed band as online FP8). The images are
+  horizontal-stripe noise.
+- A one-layer offline isolation (`marlin_isolate.py`) pins it: vLLM parallel
+  linears built exactly as the qwen_image_21 transformer builds them, loaded
+  with the artifact's verified tensors, emit **NaN** for all four layer
+  geometries (fused QKV, to_out, img_mlp proj/out), for both bf16 and fp16
+  activations — and do the same for **synthetic random int8 weights**, so the
+  checkpoint and its quantization are exonerated. The 8-bit Marlin W8A16 path
+  on this stack (sm_86, vllm 0.31.0, torch 2.13.0+cu130) is broken even though
+  `query_marlin_supported_quant_types` accepts `uint8b128` on the card, and
+  `choose_mp_linear_kernel` offers no non-Marlin W8A16 fallback on sm_86
+  (Machete is sm90+, ExLlama int4-only).
+
+The torchao-int8-in-Diffusers prior (multi-model-helper-server #160, blinded
+review within label noise of BF16) stands unchallenged: int8 quantization is
+fine; the serve-path kernel is not. The online-FP8 collapse from PR #4 and
+this int8 collapse now share one root suspect — the Marlin mixed-precision
+path on this fork/card — with the fp8 case previously isolated to the
+Marlin-quantized DiT and this run extending it to int8 with a fully verified
+checkpoint on the far side of the kernel.
+
+## Two serve-path fixes this issue required (in this PR)
+
+1. **Quant factory could not build compressed-tensors at all.**
+   `_normalize_method_name` canonicalizes `compressed-tensors` to
+   `compressed_tensors`, which then missed vLLM's hyphenated registry — so
+   both `--quantization compressed-tensors` and checkpoint auto-detection
+   raised "Unknown quantization method". The registry lookup now resolves
+   both spellings, and registry configs whose `__init__` does not take the
+   serialized schema fall back to `from_config` (`vllm_omni/quantization/factory.py`,
+   tests in `tests/quantization/test_quant_factory.py`).
+2. **The diffusion loader misclassified pre-quantized checkpoints as online
+   quantization under CPU offload.** With `--enable-cpu-offload` it loaded
+   every component on the GPU before offloading (encoder 17 GiB + VAE + the
+   packed DiT skeleton > 24 GiB → init OOM). Serialized formats are now
+   detected (`is_prequantized_config`, component-wrapper aware) and load
+   directly on the host, with each quantized module round-tripped through the
+   accelerator only for the post-load Marlin repack
+   (`vllm_omni/diffusion/model_loader/diffusers_loader.py`). With these two
+   fixes the artifact serves: init healthy in ~40 s, logs
+   `Per-component quantization: {'transformer': 'compressed-tensors'}`.
+
+## Artifact (outside Git, under the helper state root)
+
+`~/.local/state/multi-model-helper-server/state/image-service/artifacts/qwen-image-2.1-dit-int8wo/`
+— quantized `transformer/` (6.76 GiB packed int8 + BF16 boundary layers;
+224 tensors = 32 blocks × {to_q,to_k,to_v,to_out.0,proj,gate_layer,out};
+6.98 B params quantized, 0.14 B BF16), `model_index.json`/`scheduler`/
+`text_encoder`/`vae`/`processor` symlinked to the sealed snapshot, and
+`MANIFEST.json` with inputs (snapshot path, revision, tree digest), versions,
+per-file sha256, and quantization stats (mean rel-L2 1.10%, max 1.73%).
+Built by `quantize_dit_int8wo.py` using compressed-tensors' own
+`pack_to_int32` (uint8 biased by 128, 4 per int32) and `QuantizationConfig`
+serialization, so the storage convention matches llm-compressor bit-for-bit.
+`transformer/config.json` carries the quantization metadata wrapped
+per-component (`{"transformer": {...}}`) so auto-detection quantizes the DiT
+only — the encoder and VAE resolve to None and stay BF16
+(the plain global form would hand the encoder a compressed-tensors config
+it has no scales for).
+
+## Run matrix — RTX 3090
+
+Single run `int8wo`: `vllm serve <artifact> --omni --port 8091
+--served-model-name Qwen/Qwen-Image-2.1 --enforce-eager
+--enable-cpu-offload`, readiness via `/health`, VRAM via the 1 Hz
+`nvidia-smi` series (issue 2's instrument). Two precursor runs OOM'd at init
+for reasons fixed on this branch (see Verdict item 2); their logs survive
+under `runs/int8wo-expandable/` on the host (the first attempt's log was
+overwritten by the successful rerun in the same label).
+
+| Refs | Fixture | Wall (mean, s) | Peak (MiB) | Gate | PSNR vs Diffusers (dB) |
+| --- | --- | --- | --- | --- | --- |
+| 1 | edit-expression-ysolde | 31.0 | 23,935 (run max) | FAIL 4/4 | 7.14 |
+| 2 | edit-compose-pair | 36.6 | ” | FAIL 4/4 | 7.46 |
+| 3 | edit-compose-three | 41.4 | ” | FAIL 4/4 | no sealed row (derived fixture) |
+| 4 | edit-compose-four | 46.4 | ” | FAIL 4/4 | 7.11 |
+
+BF16 comparison rows (issue 2): 1-ref 34.2 s, 2-ref 36.7 s, 4-ref OOM.
+Online-fp8 (broken output, capacity reference): 29.9 / 34.1 / 38.9 / 44.3 s.
+So the int8 artifact matches the fp8 capacity/speed profile exactly —
+including its broken-output signature (PSNR band 7.0–7.5 dB) — while the
+checkpoint side is proven numerically sound, moving the defect wholly into
+the kernel.
+
+## Structure gate and quality read
+
+- Gate rule (issue 1's lesson): `opaque_fraction ≥ 0.99` and `mean(rgb_std)
+  ≥ 10`; BF16 outputs measure 1.0 opaque, 26–29 dB. Every int8-artifact run
+  fails (opaque 0.0–0.20, alpha mean 114–137; rgb_std spans 3–93 — several
+  outputs have healthy RGB spread yet near-total transparency, and one
+  (seed 2302) is near-flat grey like the fp8 signature). Outputs are
+  horizontal-stripe noise on visual inspection.
+- PSNR vs the sealed 4-reference Diffusers baseline (RGBA-on-RGBA,
+  #1's comparator): 7.14 / 7.46 / 7.11 dB mean for 1/2/4 refs — the
+  collapsed band, not the 26–29 dB BF16-row parity the issue hoped for.
+
+## Kernel isolation (the issue's "different kernels" prior, answered)
+
+`marlin_isolate.py` builds vLLM `QKVParallelLinear` / `RowParallelLinear` /
+`ColumnParallelLinear` exactly as `qwen_image_21_transformer.py` does, loads
+the artifact tensors through the production weight-loader machinery
+(bitwise-verified after load, fused q/k/v shards included), runs
+`process_weights_after_loading` (the Marlin repack) on the GPU, and compares
+`layer(x)` against `float32 x @ dequant(W)ᵀ` computed from the artifact's own
+unpacked weights:
+
+| Layer | bf16 acts | fp16 acts | packed bitwise | scales post-repack |
+| --- | --- | --- | --- | --- |
+| attn.to_qkv (fused) | NaN | NaN | equal | finite (6.6e-4…1.2e-2) |
+| attn.to_out.0 (row) | NaN | NaN | equal | finite |
+| img_mlp.proj (column) | NaN | NaN | equal | finite |
+| img_mlp.out (row, wide-in) | NaN | NaN | equal | finite |
+
+Synthetic random int8 weights reproduce the NaN identically, and the runtime
+capability query accepts `uint8b128` on this card — so this is a defect of
+the 8-bit Marlin W8A16 path on this stack (repack or kernel; sm_86,
+vllm 0.31.0, torch 2.13.0+cu130), not of the checkpoint, the quantizer, the
+loader, or the fused-QKV stacking. No alternative W8A16 kernel exists on
+sm_86 in this build to switch to.
+
+## Reproducibility
+
+Scripts in this directory: `quantize_dit_int8wo.py` (artifact + manifest),
+`verify_artifact.py` (round-trip check), `serve.sh` / `probe.py` (adapted
+issue-2 kit; probe records the structure gate inline per request), `gate.py`
+(gate + capacity summarizer), `psnr.py`, `marlin_isolate.py` (kernel
+isolation, `--synthetic` for the stack-only variant). All were pushed
+verbatim to `~/Projects/vllm-omni-val/issue5/` on the host and run from the
+validation clone checked out on this branch. Per-run serve/vram logs and all
+output PNGs live under `~/Projects/vllm-omni-val/issue5/runs/int8wo/`;
+condensed evidence in `evidence/rtx3090/`.
+
+## Helper-system follow-through (input to multi-model-helper-server #27 — not implemented here)
+
+- The 3090's 3–4 reference serving path remains blocked, now with a single
+  named suspect: the 8-bit Marlin W8A16 kernel path on sm_86 (this fork's
+  vllm 0.31.0 / torch 2.13.0 stack). Both quantized routes (online FP8,
+  offline int8 compressed-tensors with a verified checkpoint) collapse at the
+  same place. The unlock order for #27 candidacy is now: upstream Marlin
+  8-bit fix (or a non-Marlin W8A16 kernel on sm_86), then serving — capacity
+  and speed are already proven sufficient (4-ref at 46.4 s, 2.9× the sealed
+  Diffusers nocache baseline).
+- The two vllm-omni loader/factory fixes on this branch are prerequisites for
+  any compressed-tensors artifact serving under CPU offload and are
+  independent of the kernel defect.
+
+## Out-of-scope confirmation
+
+No upstream PRs; no W8A8/int4/NVFP4 artifacts built; no helper-system
+deployment evidence. Root-causing the Marlin 8-bit defect remains the
+separate lane (PR #4 carries the fp8 isolation; this branch adds the int8 +
+synthetic + bitwise-load isolation). The only code changes are the two
+serve-path fixes required to make the issue's plan steps executable at all.
diff --git a/probe/issue-5/evidence/rtx3090/artifact-manifest.json b/probe/issue-5/evidence/rtx3090/artifact-manifest.json
new file mode 100644
index 000000000..855a35194
--- /dev/null
+++ b/probe/issue-5/evidence/rtx3090/artifact-manifest.json
@@ -0,0 +1,89 @@
+{
+ "schema": "issue5-int8wo-v1",
+ "created_utc": "2026-10-08T11:16:41+00:00",
+ "description": "Qwen-Image-2.1 DiT int8 weight-only (W8A16), compressed-tensors pack-quantized",
+ "inputs": {
+  "snapshot": "/home/anodyine/.local/state/multi-model-helper-server/state/image-service/weights/hub/models--Qwen--Qwen-Image-2.1/snapshots/790c92633540aa0cb11d9abf19eb46d861714758",
+  "revision": "790c92633540aa0cb11d9abf19eb46d861714758",
+  "tree_manifest": "/home/anodyine/.local/state/multi-model-helper-server/state/image-service/weights/hub/models--Qwen--Qwen-Image-2.1/trees/790c92633540aa0cb11d9abf19eb46d861714758.json",
+  "tree_sha256": "1f35d0a5a99f2c79bf2a6818b96582828094feb8544a992bc81c1d4100aed436"
+ },
+ "versions": {
+  "python": "3.13.15",
+  "torch": "2.13.0+cu130",
+  "compressed_tensors": "0.17.0"
+ },
+ "quantization": {
+  "method": "compressed-tensors",
+  "format": "pack-quantized",
+  "weights": "int8 symmetric per-output-channel, static, minmax",
+  "config": {
+   "config_groups": {
+    "group_0": {
+     "targets": [
+      "Linear"
+     ],
+     "weights": {
+      "num_bits": 8,
+      "type": "int",
+      "symmetric": true,
+      "group_size": null,
+      "strategy": "channel",
+      "block_structure": null,
+      "dynamic": false,
+      "actorder": null,
+      "scale_dtype": null,
+      "zp_dtype": null,
+      "observer": "minmax",
+      "observer_kwargs": {}
+     },
+     "input_activations": null,
+     "output_activations": null,
+     "format": null
+    }
+   },
+   "quant_method": "compressed-tensors",
+   "kv_cache_scheme": null,
+   "format": "pack-quantized",
+   "quantization_status": "frozen",
+   "global_compression_ratio": null,
+   "ignore": [
+    "img_in",
+    "txt_in",
+    "time_text_embed",
+    "modulation",
+    "norm_out",
+    "proj_out"
+   ]
+  },
+  "ignored_layers": [
+   "img_in",
+   "txt_in",
+   "time_text_embed",
+   "modulation",
+   "norm_out",
+   "proj_out"
+  ],
+  "note": "ignore list mirrors the vllm-omni model's quant_config=None boundary layers"
+ },
+ "stats": {
+  "quantized_tensors": 224,
+  "quantized_params": 6979321856,
+  "bf16_params": 135802880,
+  "mean_rel_l2": 0.010959165191577216,
+  "max_rel_l2": 0.017284497618675232,
+  "weights_file_bytes": 7256783616
+ },
+ "files": [
+  {
+   "path": "transformer/config.json",
+   "bytes": 1405,
+   "sha256": "43b8c94380afb473f252d2707d1398b2be180f0248acee31cad4d6613f4c0a84"
+  },
+  {
+   "path": "transformer/diffusion_pytorch_model.safetensors",
+   "bytes": 7256783616,
+   "sha256": "947d9c0660776b1125ec98762817af55d59d326b8e223120aa1bdb0eced91aa9"
+  }
+ ]
+}
diff --git a/probe/issue-5/evidence/rtx3090/artifact-verify.txt b/probe/issue-5/evidence/rtx3090/artifact-verify.txt
new file mode 100644
index 000000000..33607efae
--- /dev/null
+++ b/probe/issue-5/evidence/rtx3090/artifact-verify.txt
@@ -0,0 +1,5 @@
+transformer_blocks.0.attn.to_k.weight: shape (4096, 4096) rel_l2=0.01092 scale (4096, 1)/torch.float32
+transformer_blocks.0.attn.to_out.0.weight: shape (4096, 4096) rel_l2=0.00978 scale (4096, 1)/torch.float32
+transformer_blocks.0.attn.to_q.weight: shape (4096, 4096) rel_l2=0.01106 scale (4096, 1)/torch.float32
+transformer_blocks.0.attn.to_v.weight: shape (4096, 4096) rel_l2=0.00976 scale (4096, 1)/torch.float32
+checked 4 tensors
diff --git a/probe/issue-5/evidence/rtx3090/gate-and-psnr-summary.txt b/probe/issue-5/evidence/rtx3090/gate-and-psnr-summary.txt
new file mode 100644
index 000000000..d25065c49
--- /dev/null
+++ b/probe/issue-5/evidence/rtx3090/gate-and-psnr-summary.txt
@@ -0,0 +1,119 @@
+== int8wo ==
+  1-ref: n=4 gate=FAIL wall 30.5-31.7s (mean 31.0s)
+    edit-expression-ysolde s2301: 31.73s {"opaque_fraction": 0.0294, "alpha_mean": 114.2, "rgb_std": [59.9, 61.7, 63.8]} BROKEN
+    edit-expression-ysolde s2302: 30.53s {"opaque_fraction": 0.0, "alpha_mean": 126.9, "rgb_std": [3.2, 2.9, 3.3]} BROKEN
+    edit-expression-ysolde s2303: 30.88s {"opaque_fraction": 0.0391, "alpha_mean": 125.5, "rgb_std": [63.5, 66.2, 67.0]} BROKEN
+    edit-expression-ysolde s2304: 30.95s {"opaque_fraction": 0.0352, "alpha_mean": 122.4, "rgb_std": [63.6, 66.3, 67.1]} BROKEN
+  2-ref: n=4 gate=FAIL wall 36.5-36.7s (mean 36.6s)
+    edit-compose-pair s2301: 36.46s {"opaque_fraction": 0.0303, "alpha_mean": 115.4, "rgb_std": [59.2, 61.0, 63.3]} BROKEN
+    edit-compose-pair s2302: 36.57s {"opaque_fraction": 0.0, "alpha_mean": 126.9, "rgb_std": [3.1, 2.9, 3.2]} BROKEN
+    edit-compose-pair s2303: 36.61s {"opaque_fraction": 0.03, "alpha_mean": 115.7, "rgb_std": [76.1, 77.9, 69.1]} BROKEN
+    edit-compose-pair s2304: 36.69s {"opaque_fraction": 0.1257, "alpha_mean": 127.1, "rgb_std": [64.3, 64.3, 64.3]} BROKEN
+  3-ref: n=4 gate=FAIL wall 41.3-41.6s (mean 41.4s)
+    edit-compose-three s2301: 41.57s {"opaque_fraction": 0.2029, "alpha_mean": 137.2, "rgb_std": [93.0, 92.7, 91.0]} BROKEN
+    edit-compose-three s2302: 41.32s {"opaque_fraction": 0.0193, "alpha_mean": 131.9, "rgb_std": [40.8, 45.8, 44.3]} BROKEN
+    edit-compose-three s2303: 41.34s {"opaque_fraction": 0.0188, "alpha_mean": 131.6, "rgb_std": [41.0, 45.9, 44.6]} BROKEN
+    edit-compose-three s2304: 41.39s {"opaque_fraction": 0.0189, "alpha_mean": 131.6, "rgb_std": [41.1, 46.0, 44.6]} BROKEN
+  4-ref: n=4 gate=FAIL wall 46.4-46.5s (mean 46.4s)
+    edit-compose-four s2301: 46.4s {"opaque_fraction": 0.0195, "alpha_mean": 131.6, "rgb_std": [41.4, 46.3, 45.0]} BROKEN
+    edit-compose-four s2302: 46.42s {"opaque_fraction": 0.0196, "alpha_mean": 131.9, "rgb_std": [41.1, 46.0, 44.5]} BROKEN
+    edit-compose-four s2303: 46.44s {"opaque_fraction": 0.0195, "alpha_mean": 132.0, "rgb_std": [41.1, 46.1, 44.7]} BROKEN
+    edit-compose-four s2304: 46.48s {"opaque_fraction": 0.0195, "alpha_mean": 131.7, "rgb_std": [41.0, 46.1, 44.5]} BROKEN
+  nvidia-smi peak: 23935 MiB; server peak: None MB
+  STRUCTURE GATE: FAIL
+== psnr edit-expression-ysolde ==
+{
+ "int8wo": {
+  "rows": [
+   {
+    "seed": 2301,
+    "psnr_db": 6.95,
+    "modes": "RGBA/RGBA"
+   },
+   {
+    "seed": 2302,
+    "psnr_db": 8.3,
+    "modes": "RGBA/RGBA"
+   },
+   {
+    "seed": 2303,
+    "psnr_db": 6.67,
+    "modes": "RGBA/RGBA"
+   },
+   {
+    "seed": 2304,
+    "psnr_db": 6.65,
+    "modes": "RGBA/RGBA"
+   }
+  ],
+  "mean_db": 7.14,
+  "min_db": 6.65,
+  "max_db": 8.3,
+  "n": 4
+ }
+}
+== psnr edit-compose-pair ==
+{
+ "int8wo": {
+  "rows": [
+   {
+    "seed": 2301,
+    "psnr_db": 7.03,
+    "modes": "RGBA/RGBA"
+   },
+   {
+    "seed": 2302,
+    "psnr_db": 8.86,
+    "modes": "RGBA/RGBA"
+   },
+   {
+    "seed": 2303,
+    "psnr_db": 6.77,
+    "modes": "RGBA/RGBA"
+   },
+   {
+    "seed": 2304,
+    "psnr_db": 7.19,
+    "modes": "RGBA/RGBA"
+   }
+  ],
+  "mean_db": 7.46,
+  "min_db": 6.77,
+  "max_db": 8.86,
+  "n": 4
+ }
+}
+== psnr edit-compose-four ==
+{
+ "int8wo": {
+  "rows": [
+   {
+    "seed": 2301,
+    "psnr_db": 7.02,
+    "modes": "RGBA/RGBA"
+   },
+   {
+    "seed": 2302,
+    "psnr_db": 7.11,
+    "modes": "RGBA/RGBA"
+   },
+   {
+    "seed": 2303,
+    "psnr_db": 7.1,
+    "modes": "RGBA/RGBA"
+   },
+   {
+    "seed": 2304,
+    "psnr_db": 7.2,
+    "modes": "RGBA/RGBA"
+   }
+  ],
+  "mean_db": 7.11,
+  "min_db": 7.02,
+  "max_db": 7.2,
+  "n": 4
+ }
+}
+23935 MiB, 100 %
+(APIServer pid=473436) INFO 10-08 07:36:28 [data.py:1483] Auto-detected quantization 'component' from model config
+(DiffusionWorker pid=473436) INFO 10-08 07:36:28 [diffusers_loader.py:719] Offline-quantized model with CPU offload, loading weights directly on CPU
diff --git a/probe/issue-5/evidence/rtx3090/marlin-isolate-block0.json b/probe/issue-5/evidence/rtx3090/marlin-isolate-block0.json
new file mode 100644
index 000000000..b313dc513
--- /dev/null
+++ b/probe/issue-5/evidence/rtx3090/marlin-isolate-block0.json
@@ -0,0 +1,82 @@
+[
+ {
+  "layer": "attn.to_qkv (fused q/k/v)",
+  "rel_l2": "NaN",
+  "out_finite": false,
+  "packed_bitwise_ok": true,
+  "scale_finite_pre_repack": true,
+  "scale_post_repack": {
+   "finite": true,
+   "min": 0.0006612942670471966,
+   "max": 0.012303149327635765
+  },
+  "out_shape": [
+   512,
+   12288
+  ],
+  "ref_shape": [
+   512,
+   12288
+  ]
+ },
+ {
+  "layer": "attn.to_out.0 (row)",
+  "rel_l2": "NaN",
+  "out_finite": false,
+  "packed_bitwise_ok": true,
+  "scale_finite_pre_repack": true,
+  "scale_post_repack": {
+   "finite": true,
+   "min": 0.00037870631786063313,
+   "max": 0.011257382109761238
+  },
+  "out_shape": [
+   512,
+   4096
+  ],
+  "ref_shape": [
+   512,
+   4096
+  ]
+ },
+ {
+  "layer": "img_mlp.proj (column)",
+  "rel_l2": "NaN",
+  "out_finite": false,
+  "packed_bitwise_ok": true,
+  "scale_finite_pre_repack": true,
+  "scale_post_repack": {
+   "finite": true,
+   "min": 0.0005651759565807879,
+   "max": 0.005628691054880619
+  },
+  "out_shape": [
+   512,
+   12288
+  ],
+  "ref_shape": [
+   512,
+   12288
+  ]
+ },
+ {
+  "layer": "img_mlp.out (row, wide-in)",
+  "rel_l2": "NaN",
+  "out_finite": false,
+  "packed_bitwise_ok": true,
+  "scale_finite_pre_repack": true,
+  "scale_post_repack": {
+   "finite": true,
+   "min": 0.0008073942153714597,
+   "max": 0.014763779938220978
+  },
+  "out_shape": [
+   512,
+   4096
+  ],
+  "ref_shape": [
+   512,
+   4096
+  ]
+ }
+]
diff --git a/probe/issue-5/evidence/rtx3090/marlin-isolate-synthetic.json b/probe/issue-5/evidence/rtx3090/marlin-isolate-synthetic.json
new file mode 100644
index 000000000..4befc8f40
--- /dev/null
+++ b/probe/issue-5/evidence/rtx3090/marlin-isolate-synthetic.json
@@ -0,0 +1,58 @@
+[
+ {
+  "layer": "attn.to_qkv (fused q/k/v)",
+  "rel_l2_by_act_dtype": {
+   "bfloat16": "NaN",
+   "float16": "NaN"
+  },
+  "packed_bitwise_ok": true,
+  "scale_finite_pre_repack": true,
+  "scale_post_repack": {
+   "finite": true,
+   "min": 0.02410734072327614,
+   "max": 0.043380916118621826
+  }
+ },
+ {
+  "layer": "attn.to_out.0 (row)",
+  "rel_l2_by_act_dtype": {
+   "bfloat16": "NaN",
+   "float16": "NaN"
+  },
+  "packed_bitwise_ok": true,
+  "scale_finite_pre_repack": true,
+  "scale_post_repack": {
+   "finite": true,
+   "min": 0.02417764998972416,
+   "max": 0.042702700942754745
+  }
+ },
+ {
+  "layer": "img_mlp.proj (column)",
+  "rel_l2_by_act_dtype": {
+   "bfloat16": "NaN",
+   "float16": "NaN"
+  },
+  "packed_bitwise_ok": true,
+  "scale_finite_pre_repack": true,
+  "scale_post_repack": {
+   "finite": true,
+   "min": 0.024067752063274384,
+   "max": 0.04336588457226753
+  }
+ },
+ {
+  "layer": "img_mlp.out (row, wide-in)",
+  "rel_l2_by_act_dtype": {
+   "bfloat16": "NaN",
+   "float16": "NaN"
+  },
+  "packed_bitwise_ok": true,
+  "scale_finite_pre_repack": true,
+  "scale_post_repack": {
+   "finite": true,
+   "min": 0.02682814560830593,
+   "max": 0.04223968833684921
+  }
+ }
+]
diff --git a/probe/issue-5/evidence/rtx3090/matrix.json b/probe/issue-5/evidence/rtx3090/matrix.json
new file mode 100644
index 000000000..d82bc9ff7
--- /dev/null
+++ b/probe/issue-5/evidence/rtx3090/matrix.json
@@ -0,0 +1,978 @@
+[
+ {
+  "fixture_id": "edit-expression-ysolde",
+  "seed": 2301,
+  "n_references": 1,
+  "steps": 20,
+  "true_cfg_scale": 1.0,
+  "size": "864x1152",
+  "wall_s": 31.73,
+  "http": 200,
+  "bytes": 3983827,
+  "sha256": "a62cc903858a5026c08f284bdfea58108068fd6ab6425b54c246d65990e0bec4",
+  "structure": {
+   "opaque_fraction": 0.0294,
+   "alpha_mean": 114.2,
+   "rgb_std": [
+    59.9,
+    61.7,
+    63.8
+   ]
+  },
+  "usage": null,
+  "metrics": {
+   "stage_metrics": {
+    "0": {
+     "stage_id": 0,
+     "final_output_type": "image",
+     "finish_reason": null,
+     "num_tokens_in": 0,
+     "num_tokens_out": 0,
+     "stage_gen_time_ms": 31616.730451583862,
+     "postprocess_time_ms": 0.0,
+     "audio_frames": 0,
+     "audio_sample_rate": 0,
+     "audio_duration_s": 0.0,
+     "image_pixels": 995328,
+     "denoise_step_latency_ms": 1580.8365225791931,
+     "output_unit_type": "image",
+     "output_unit_count": 1,
+     "serving_time_to_first_output_ms": 31616.933584213257,
+     "image_time_to_first_output_ms": 31616.7151927948,
+     "time_per_output_unit_ms": 0.0,
+     "inter_output_latency_ms": 0.0,
+     "inter_output_latencies_ms": [],
+     "vllm_ttft_ms": 0.0,
+     "vllm_tpot_ms": 0.0,
+     "vllm_itl_ms": 0.0,
+     "vllm_itls_ms": [],
+     "stage_name": "diffusion"
+    }
+   },
+   "stage_id": 0,
+   "final_output_type": "image",
+   "num_tokens_in": 0,
+   "num_tokens_out": 0,
+   "stage_durations": {
+    "queue_wait_ms": 0.18030800129054114,
+    "stage_0_gen_ms": 31616.730451583862
+   },
+   "peak_memory_mb": 21594.0
+  }
+ },
+ {
+  "fixture_id": "edit-expression-ysolde",
+  "seed": 2302,
+  "n_references": 1,
+  "steps": 20,
+  "true_cfg_scale": 1.0,
+  "size": "864x1152",
+  "wall_s": 30.53,
+  "http": 200,
+  "bytes": 3983827,
+  "sha256": "ae5c51a9a36f8892c15116b90bcd2b85f90a83bae3daea2196f2602b5968e9af",
+  "structure": {
+   "opaque_fraction": 0.0,
+   "alpha_mean": 126.9,
+   "rgb_std": [
+    3.2,
+    2.9,
+    3.3
+   ]
+  },
+  "usage": null,
+  "metrics": {
+   "stage_metrics": {
+    "0": {
+     "stage_id": 0,
+     "final_output_type": "image",
+     "finish_reason": null,
+     "num_tokens_in": 0,
+     "num_tokens_out": 0,
+     "stage_gen_time_ms": 30463.57011795044,
+     "postprocess_time_ms": 0.0,
+     "audio_frames": 0,
+     "audio_sample_rate": 0,
+     "audio_duration_s": 0.0,
+     "image_pixels": 995328,
+     "denoise_step_latency_ms": 1523.178505897522,
+     "output_unit_type": "image",
+     "output_unit_count": 1,
+     "serving_time_to_first_output_ms": 30463.69981765747,
+     "image_time_to_first_output_ms": 30463.540077209473,
+     "time_per_output_unit_ms": 0.0,
+     "inter_output_latency_ms": 0.0,
+     "inter_output_latencies_ms": [],
+     "vllm_ttft_ms": 0.0,
+     "vllm_tpot_ms": 0.0,
+     "vllm_itl_ms": 0.0,
+     "vllm_itls_ms": [],
+     "stage_name": "diffusion"
+    }
+   },
+   "stage_id": 0,
+   "final_output_type": "image",
+   "num_tokens_in": 0,
+   "num_tokens_out": 0,
+   "stage_durations": {
+    "queue_wait_ms": 0.12335599603829905,
+    "stage_0_gen_ms": 30463.57011795044
+   },
+   "peak_memory_mb": 21900.0
+  }
+ },
+ {
+  "fixture_id": "edit-expression-ysolde",
+  "seed": 2303,
+  "n_references": 1,
+  "steps": 20,
+  "true_cfg_scale": 1.0,
+  "size": "864x1152",
+  "wall_s": 30.88,
+  "http": 200,
+  "bytes": 3983827,
+  "sha256": "d05825c0d525ff8522d83db2a9fc6337cc06a3960f7eb2a112898c5b71aa34a1",
+  "structure": {
+   "opaque_fraction": 0.0391,
+   "alpha_mean": 125.5,
+   "rgb_std": [
+    63.5,
+    66.2,
+    67.0
+   ]
+  },
+  "usage": null,
+  "metrics": {
+   "stage_metrics": {
+    "0": {
+     "stage_id": 0,
+     "final_output_type": "image",
+     "finish_reason": null,
+     "num_tokens_in": 0,
+     "num_tokens_out": 0,
+     "stage_gen_time_ms": 30779.66594696045,
+     "postprocess_time_ms": 0.0,
+     "audio_frames": 0,
+     "audio_sample_rate": 0,
+     "audio_duration_s": 0.0,
+     "image_pixels": 995328,
+     "denoise_step_latency_ms": 1538.9832973480225,
+     "output_unit_type": "image",
+     "output_unit_count": 1,
+     "serving_time_to_first_output_ms": 30779.80923652649,
+     "image_time_to_first_output_ms": 30779.641151428223,
+     "time_per_output_unit_ms": 0.0,
+     "inter_output_latency_ms": 0.0,
+     "inter_output_latencies_ms": [],
+     "vllm_ttft_ms": 0.0,
+     "vllm_tpot_ms": 0.0,
+     "vllm_itl_ms": 0.0,
+     "vllm_itls_ms": [],
+     "stage_name": "diffusion"
+    }
+   },
+   "stage_id": 0,
+   "final_output_type": "image",
+   "num_tokens_in": 0,
+   "num_tokens_out": 0,
+   "stage_durations": {
+    "queue_wait_ms": 0.14197500422596931,
+    "stage_0_gen_ms": 30779.66594696045
+   },
+   "peak_memory_mb": 21906.0
+  }
+ },
+ {
+  "fixture_id": "edit-expression-ysolde",
+  "seed": 2304,
+  "n_references": 1,
+  "steps": 20,
+  "true_cfg_scale": 1.0,
+  "size": "864x1152",
+  "wall_s": 30.95,
+  "http": 200,
+  "bytes": 3983827,
+  "sha256": "10e85cf24451f4fb7bf2d8e07b4039158af08e36467a2a14d2baee1d68cd98fa",
+  "structure": {
+   "opaque_fraction": 0.0352,
+   "alpha_mean": 122.4,
+   "rgb_std": [
+    63.6,
+    66.3,
+    67.1
+   ]
+  },
+  "usage": null,
+  "metrics": {
+   "stage_metrics": {
+    "0": {
+     "stage_id": 0,
+     "final_output_type": "image",
+     "finish_reason": null,
+     "num_tokens_in": 0,
+     "num_tokens_out": 0,
+     "stage_gen_time_ms": 30871.540069580078,
+     "postprocess_time_ms": 0.0,
+     "audio_frames": 0,
+     "audio_sample_rate": 0,
+     "audio_duration_s": 0.0,
+     "image_pixels": 995328,
+     "denoise_step_latency_ms": 1543.577003479004,
+     "output_unit_type": "image",
+     "output_unit_count": 1,
+     "serving_time_to_first_output_ms": 30871.644735336304,
+     "image_time_to_first_output_ms": 30871.514797210693,
+     "time_per_output_unit_ms": 0.0,
+     "inter_output_latency_ms": 0.0,
+     "inter_output_latencies_ms": [],
+     "vllm_ttft_ms": 0.0,
+     "vllm_tpot_ms": 0.0,
+     "vllm_itl_ms": 0.0,
+     "vllm_itls_ms": [],
+     "stage_name": "diffusion"
+    }
+   },
+   "stage_id": 0,
+   "final_output_type": "image",
+   "num_tokens_in": 0,
+   "num_tokens_out": 0,
+   "stage_durations": {
+    "queue_wait_ms": 0.10297299741068855,
+    "stage_0_gen_ms": 30871.540069580078
+   },
+   "peak_memory_mb": 21906.0
+  }
+ },
+ {
+  "fixture_id": "edit-compose-pair",
+  "seed": 2301,
+  "n_references": 2,
+  "steps": 20,
+  "true_cfg_scale": 1.0,
+  "size": "1248x832",
+  "wall_s": 36.46,
+  "http": 200,
+  "bytes": 4155590,
+  "sha256": "4bfdf57ecb0e1ed1c2c4dcb10ce0545d89040ad08ed754e01adbe2ddd2b407d2",
+  "structure": {
+   "opaque_fraction": 0.0303,
+   "alpha_mean": 115.4,
+   "rgb_std": [
+    59.2,
+    61.0,
+    63.3
+   ]
+  },
+  "usage": null,
+  "metrics": {
+   "stage_metrics": {
+    "0": {
+     "stage_id": 0,
+     "final_output_type": "image",
+     "finish_reason": null,
+     "num_tokens_in": 0,
+     "num_tokens_out": 0,
+     "stage_gen_time_ms": 36376.55758857727,
+     "postprocess_time_ms": 0.0,
+     "audio_frames": 0,
+     "audio_sample_rate": 0,
+     "audio_duration_s": 0.0,
+     "image_pixels": 1038336,
+     "denoise_step_latency_ms": 1818.8278794288635,
+     "output_unit_type": "image",
+     "output_unit_count": 1,
+     "serving_time_to_first_output_ms": 36376.71780586243,
+     "image_time_to_first_output_ms": 36376.53112411499,
+     "time_per_output_unit_ms": 0.0,
+     "inter_output_latency_ms": 0.0,
+     "inter_output_latencies_ms": [],
+     "vllm_ttft_ms": 0.0,
+     "vllm_tpot_ms": 0.0,
+     "vllm_itl_ms": 0.0,
+     "vllm_itls_ms": [],
+     "stage_name": "diffusion"
+    }
+   },
+   "stage_id": 0,
+   "final_output_type": "image",
+   "num_tokens_in": 0,
+   "num_tokens_out": 0,
+   "stage_durations": {
+    "queue_wait_ms": 0.15923400496831164,
+    "stage_0_gen_ms": 36376.55758857727
+   },
+   "peak_memory_mb": 22676.0
+  }
+ },
+ {
+  "fixture_id": "edit-compose-pair",
+  "seed": 2302,
+  "n_references": 2,
+  "steps": 20,
+  "true_cfg_scale": 1.0,
+  "size": "1248x832",
+  "wall_s": 36.57,
+  "http": 200,
+  "bytes": 4155590,
+  "sha256": "e5ad17605624fdb636dc02e272c0dba2d3286dcfa4c27f297a605e785f3177a2",
+  "structure": {
+   "opaque_fraction": 0.0,
+   "alpha_mean": 126.9,
+   "rgb_std": [
+    3.1,
+    2.9,
+    3.2
+   ]
+  },
+  "usage": null,
+  "metrics": {
+   "stage_metrics": {
+    "0": {
+     "stage_id": 0,
+     "final_output_type": "image",
+     "finish_reason": null,
+     "num_tokens_in": 0,
+     "num_tokens_out": 0,
+     "stage_gen_time_ms": 36508.753061294556,
+     "postprocess_time_ms": 0.0,
+     "audio_frames": 0,
+     "audio_sample_rate": 0,
+     "audio_duration_s": 0.0,
+     "image_pixels": 1038336,
+     "denoise_step_latency_ms": 1825.4376530647278,
+     "output_unit_type": "image",
+     "output_unit_count": 1,
+     "serving_time_to_first_output_ms": 36508.851528167725,
+     "image_time_to_first_output_ms": 36508.72349739075,
+     "time_per_output_unit_ms": 0.0,
+     "inter_output_latency_ms": 0.0,
+     "inter_output_latencies_ms": [],
+     "vllm_ttft_ms": 0.0,
+     "vllm_tpot_ms": 0.0,
+     "vllm_itl_ms": 0.0,
+     "vllm_itls_ms": [],
+     "stage_name": "diffusion"
+    }
+   },
+   "stage_id": 0,
+   "final_output_type": "image",
+   "num_tokens_in": 0,
+   "num_tokens_out": 0,
+   "stage_durations": {
+    "queue_wait_ms": 0.1011220010695979,
+    "stage_0_gen_ms": 36508.753061294556
+   },
+   "peak_memory_mb": 22556.0
+  }
+ },
+ {
+  "fixture_id": "edit-compose-pair",
+  "seed": 2303,
+  "n_references": 2,
+  "steps": 20,
+  "true_cfg_scale": 1.0,
+  "size": "1248x832",
+  "wall_s": 36.61,
+  "http": 200,
+  "bytes": 4155590,
+  "sha256": "f58a982cf1f3afd4a81fcc6b349aa3b147fecea0594e33f49cab5673f300b2ac",
+  "structure": {
+   "opaque_fraction": 0.03,
+   "alpha_mean": 115.7,
+   "rgb_std": [
+    76.1,
+    77.9,
+    69.1
+   ]
+  },
+  "usage": null,
+  "metrics": {
+   "stage_metrics": {
+    "0": {
+     "stage_id": 0,
+     "final_output_type": "image",
+     "finish_reason": null,
+     "num_tokens_in": 0,
+     "num_tokens_out": 0,
+     "stage_gen_time_ms": 36523.38361740112,
+     "postprocess_time_ms": 0.0,
+     "audio_frames": 0,
+     "audio_sample_rate": 0,
+     "audio_duration_s": 0.0,
+     "image_pixels": 1038336,
+     "denoise_step_latency_ms": 1826.1691808700562,
+     "output_unit_type": "image",
+     "output_unit_count": 1,
+     "serving_time_to_first_output_ms": 36523.181676864624,
+     "image_time_to_first_output_ms": 36523.05722236633,
+     "time_per_output_unit_ms": 0.0,
+     "inter_output_latency_ms": 0.0,
+     "inter_output_latencies_ms": [],
+     "vllm_ttft_ms": 0.0,
+     "vllm_tpot_ms": 0.0,
+     "vllm_itl_ms": 0.0,
+     "vllm_itls_ms": [],
+     "stage_name": "diffusion"
+    }
+   },
+   "stage_id": 0,
+   "final_output_type": "image",
+   "num_tokens_in": 0,
+   "num_tokens_out": 0,
+   "stage_durations": {
+    "queue_wait_ms": 0.0983119971351698,
+    "stage_0_gen_ms": 36523.38361740112
+   },
+   "peak_memory_mb": 22554.0
+  }
+ },
+ {
+  "fixture_id": "edit-compose-pair",
+  "seed": 2304,
+  "n_references": 2,
+  "steps": 20,
+  "true_cfg_scale": 1.0,
+  "size": "1248x832",
+  "wall_s": 36.69,
+  "http": 200,
+  "bytes": 4155590,
+  "sha256": "871c8147b27310508fb2c9e72b3b47499d7f8ab4ea705719aa30c465c8a5476a",
+  "structure": {
+   "opaque_fraction": 0.1257,
+   "alpha_mean": 127.1,
+   "rgb_std": [
+    64.3,
+    64.3,
+    64.3
+   ]
+  },
+  "usage": null,
+  "metrics": {
+   "stage_metrics": {
+    "0": {
+     "stage_id": 0,
+     "final_output_type": "image",
+     "finish_reason": null,
+     "num_tokens_in": 0,
+     "num_tokens_out": 0,
+     "stage_gen_time_ms": 36608.7908744812,
+     "postprocess_time_ms": 0.0,
+     "audio_frames": 0,
+     "audio_sample_rate": 0,
+     "audio_duration_s": 0.0,
+     "image_pixels": 1038336,
+     "denoise_step_latency_ms": 1830.43954372406,
+     "output_unit_type": "image",
+     "output_unit_count": 1,
+     "serving_time_to_first_output_ms": 36608.89911651611,
+     "image_time_to_first_output_ms": 36608.76417160034,
+     "time_per_output_unit_ms": 0.0,
+     "inter_output_latency_ms": 0.0,
+     "inter_output_latencies_ms": [],
+     "vllm_ttft_ms": 0.0,
+     "vllm_tpot_ms": 0.0,
+     "vllm_itl_ms": 0.0,
+     "vllm_itls_ms": [],
+     "stage_name": "diffusion"
+    }
+   },
+   "stage_id": 0,
+   "final_output_type": "image",
+   "num_tokens_in": 0,
+   "num_tokens_out": 0,
+   "stage_durations": {
+    "queue_wait_ms": 0.10716100223362446,
+    "stage_0_gen_ms": 36608.7908744812
+   },
+   "peak_memory_mb": 22556.0
+  }
+ },
+ {
+  "fixture_id": "edit-compose-three",
+  "seed": 2301,
+  "n_references": 3,
+  "steps": 20,
+  "true_cfg_scale": 1.0,
+  "size": "1344x768",
+  "wall_s": 41.57,
+  "http": 200,
+  "bytes": 4130915,
+  "sha256": "810cb635438bac76de76fa44172687e2f18b212379458580a32bbddd028bd78c",
+  "structure": {
+   "opaque_fraction": 0.2029,
+   "alpha_mean": 137.2,
+   "rgb_std": [
+    93.0,
+    92.7,
+    91.0
+   ]
+  },
+  "usage": null,
+  "metrics": {
+   "stage_metrics": {
+    "0": {
+     "stage_id": 0,
+     "final_output_type": "image",
+     "finish_reason": null,
+     "num_tokens_in": 0,
+     "num_tokens_out": 0,
+     "stage_gen_time_ms": 41476.71937942505,
+     "postprocess_time_ms": 0.0,
+     "audio_frames": 0,
+     "audio_sample_rate": 0,
+     "audio_duration_s": 0.0,
+     "image_pixels": 1032192,
+     "denoise_step_latency_ms": 2073.8359689712524,
+     "output_unit_type": "image",
+     "output_unit_count": 1,
+     "serving_time_to_first_output_ms": 41476.8271446228,
+     "image_time_to_first_output_ms": 41476.691246032715,
+     "time_per_output_unit_ms": 0.0,
+     "inter_output_latency_ms": 0.0,
+     "inter_output_latencies_ms": [],
+     "vllm_ttft_ms": 0.0,
+     "vllm_tpot_ms": 0.0,
+     "vllm_itl_ms": 0.0,
+     "vllm_itls_ms": [],
+     "stage_name": "diffusion"
+    }
+   },
+   "stage_id": 0,
+   "final_output_type": "image",
+   "num_tokens_in": 0,
+   "num_tokens_out": 0,
+   "stage_durations": {
+    "queue_wait_ms": 0.1088700009859167,
+    "stage_0_gen_ms": 41476.71937942505
+   },
+   "peak_memory_mb": 23468.0
+  }
+ },
+ {
+  "fixture_id": "edit-compose-three",
+  "seed": 2302,
+  "n_references": 3,
+  "steps": 20,
+  "true_cfg_scale": 1.0,
+  "size": "1344x768",
+  "wall_s": 41.32,
+  "http": 200,
+  "bytes": 4130915,
+  "sha256": "2f085a5089ab0618a2f17df7c29c52fde091247cb4a1ccf0cd8d662171b18fc9",
+  "structure": {
+   "opaque_fraction": 0.0193,
+   "alpha_mean": 131.9,
+   "rgb_std": [
+    40.8,
+    45.8,
+    44.3
+   ]
+  },
+  "usage": null,
+  "metrics": {
+   "stage_metrics": {
+    "0": {
+     "stage_id": 0,
+     "final_output_type": "image",
+     "finish_reason": null,
+     "num_tokens_in": 0,
+     "num_tokens_out": 0,
+     "stage_gen_time_ms": 41233.43777656555,
+     "postprocess_time_ms": 0.0,
+     "audio_frames": 0,
+     "audio_sample_rate": 0,
+     "audio_duration_s": 0.0,
+     "image_pixels": 1032192,
+     "denoise_step_latency_ms": 2061.6718888282776,
+     "output_unit_type": "image",
+     "output_unit_count": 1,
+     "serving_time_to_first_output_ms": 41233.57892036438,
+     "image_time_to_first_output_ms": 41233.40702056885,
+     "time_per_output_unit_ms": 0.0,
+     "inter_output_latency_ms": 0.0,
+     "inter_output_latencies_ms": [],
+     "vllm_ttft_ms": 0.0,
+     "vllm_tpot_ms": 0.0,
+     "vllm_itl_ms": 0.0,
+     "vllm_itls_ms": [],
+     "stage_name": "diffusion"
+    }
+   },
+   "stage_id": 0,
+   "final_output_type": "image",
+   "num_tokens_in": 0,
+   "num_tokens_out": 0,
+   "stage_durations": {
+    "queue_wait_ms": 0.14587400073651224,
+    "stage_0_gen_ms": 41233.43777656555
+   },
+   "peak_memory_mb": 23460.0
+  }
+ },
+ {
+  "fixture_id": "edit-compose-three",
+  "seed": 2303,
+  "n_references": 3,
+  "steps": 20,
+  "true_cfg_scale": 1.0,
+  "size": "1344x768",
+  "wall_s": 41.34,
+  "http": 200,
+  "bytes": 4130915,
+  "sha256": "81e9b7436d392c8420d1529976a0d88e5de03af1e70823cc8c7b279b3d7bb3fa",
+  "structure": {
+   "opaque_fraction": 0.0188,
+   "alpha_mean": 131.6,
+   "rgb_std": [
+    41.0,
+    45.9,
+    44.6
+   ]
+  },
+  "usage": null,
+  "metrics": {
+   "stage_metrics": {
+    "0": {
+     "stage_id": 0,
+     "final_output_type": "image",
+     "finish_reason": null,
+     "num_tokens_in": 0,
+     "num_tokens_out": 0,
+     "stage_gen_time_ms": 41250.11157989502,
+     "postprocess_time_ms": 0.0,
+     "audio_frames": 0,
+     "audio_sample_rate": 0,
+     "audio_duration_s": 0.0,
+     "image_pixels": 1032192,
+     "denoise_step_latency_ms": 2062.505578994751,
+     "output_unit_type": "image",
+     "output_unit_count": 1,
+     "serving_time_to_first_output_ms": 41250.218868255615,
+     "image_time_to_first_output_ms": 41250.08296966553,
+     "time_per_output_unit_ms": 0.0,
+     "inter_output_latency_ms": 0.0,
+     "inter_output_latencies_ms": [],
+     "vllm_ttft_ms": 0.0,
+     "vllm_tpot_ms": 0.0,
+     "vllm_itl_ms": 0.0,
+     "vllm_itls_ms": [],
+     "stage_name": "diffusion"
+    }
+   },
+   "stage_id": 0,
+   "final_output_type": "image",
+   "num_tokens_in": 0,
+   "num_tokens_out": 0,
+   "stage_durations": {
+    "queue_wait_ms": 0.10969900176860392,
+    "stage_0_gen_ms": 41250.11157989502
+   },
+   "peak_memory_mb": 23394.0
+  }
+ },
+ {
+  "fixture_id": "edit-compose-three",
+  "seed": 2304,
+  "n_references": 3,
+  "steps": 20,
+  "true_cfg_scale": 1.0,
+  "size": "1344x768",
+  "wall_s": 41.39,
+  "http": 200,
+  "bytes": 4130915,
+  "sha256": "2221932a1783102d87c01157e231160803cebac2113c86b9416e537e0e738747",
+  "structure": {
+   "opaque_fraction": 0.0189,
+   "alpha_mean": 131.6,
+   "rgb_std": [
+    41.1,
+    46.0,
+    44.6
+   ]
+  },
+  "usage": null,
+  "metrics": {
+   "stage_metrics": {
+    "0": {
+     "stage_id": 0,
+     "final_output_type": "image",
+     "finish_reason": null,
+     "num_tokens_in": 0,
+     "num_tokens_out": 0,
+     "stage_gen_time_ms": 41297.605991363525,
+     "postprocess_time_ms": 0.0,
+     "audio_frames": 0,
+     "audio_sample_rate": 0,
+     "audio_duration_s": 0.0,
+     "image_pixels": 1032192,
+     "denoise_step_latency_ms": 2064.8802995681763,
+     "output_unit_type": "image",
+     "output_unit_count": 1,
+     "serving_time_to_first_output_ms": 41297.31011390686,
+     "image_time_to_first_output_ms": 41297.15037345886,
+     "time_per_output_unit_ms": 0.0,
+     "inter_output_latency_ms": 0.0,
+     "inter_output_latencies_ms": [],
+     "vllm_ttft_ms": 0.0,
+     "vllm_tpot_ms": 0.0,
+     "vllm_itl_ms": 0.0,
+     "vllm_itls_ms": [],
+     "stage_name": "diffusion"
+    }
+   },
+   "stage_id": 0,
+   "final_output_type": "image",
+   "num_tokens_in": 0,
+   "num_tokens_out": 0,
+   "stage_durations": {
+    "queue_wait_ms": 0.134524998429697,
+    "stage_0_gen_ms": 41297.605991363525
+   },
+   "peak_memory_mb": 23394.0
+  }
+ },
+ {
+  "fixture_id": "edit-compose-four",
+  "seed": 2301,
+  "n_references": 4,
+  "steps": 20,
+  "true_cfg_scale": 1.0,
+  "size": "1344x768",
+  "wall_s": 46.4,
+  "http": 200,
+  "bytes": 4130915,
+  "sha256": "abd057bee84e594437f895cc9f51f90f118adfeb4dbf738686791b41487e93b2",
+  "structure": {
+   "opaque_fraction": 0.0195,
+   "alpha_mean": 131.6,
+   "rgb_std": [
+    41.4,
+    46.3,
+    45.0
+   ]
+  },
+  "usage": null,
+  "metrics": {
+   "stage_metrics": {
+    "0": {
+     "stage_id": 0,
+     "final_output_type": "image",
+     "finish_reason": null,
+     "num_tokens_in": 0,
+     "num_tokens_out": 0,
+     "stage_gen_time_ms": 46304.86750602722,
+     "postprocess_time_ms": 0.0,
+     "audio_frames": 0,
+     "audio_sample_rate": 0,
+     "audio_duration_s": 0.0,
+     "image_pixels": 1032192,
+     "denoise_step_latency_ms": 2315.243375301361,
+     "output_unit_type": "image",
+     "output_unit_count": 1,
+     "serving_time_to_first_output_ms": 46304.97694015503,
+     "image_time_to_first_output_ms": 46304.8369884491,
+     "time_per_output_unit_ms": 0.0,
+     "inter_output_latency_ms": 0.0,
+     "inter_output_latencies_ms": [],
+     "vllm_ttft_ms": 0.0,
+     "vllm_tpot_ms": 0.0,
+     "vllm_itl_ms": 0.0,
+     "vllm_itls_ms": [],
+     "stage_name": "diffusion"
+    }
+   },
+   "stage_id": 0,
+   "final_output_type": "image",
+   "num_tokens_in": 0,
+   "num_tokens_out": 0,
+   "stage_durations": {
+    "queue_wait_ms": 0.11391400039428845,
+    "stage_0_gen_ms": 46304.86750602722
+   },
+   "peak_memory_mb": 23534.0
+  }
+ },
+ {
+  "fixture_id": "edit-compose-four",
+  "seed": 2302,
+  "n_references": 4,
+  "steps": 20,
+  "true_cfg_scale": 1.0,
+  "size": "1344x768",
+  "wall_s": 46.42,
+  "http": 200,
+  "bytes": 4130915,
+  "sha256": "24a783a9ca3a002745351476e8000be87854d1364ce92ca944c2986a983922f5",
+  "structure": {
+   "opaque_fraction": 0.0196,
+   "alpha_mean": 131.9,
+   "rgb_std": [
+    41.1,
+    46.0,
+    44.5
+   ]
+  },
+  "usage": null,
+  "metrics": {
+   "stage_metrics": {
+    "0": {
+     "stage_id": 0,
+     "final_output_type": "image",
+     "finish_reason": null,
+     "num_tokens_in": 0,
+     "num_tokens_out": 0,
+     "stage_gen_time_ms": 46325.687408447266,
+     "postprocess_time_ms": 0.0,
+     "audio_frames": 0,
+     "audio_sample_rate": 0,
+     "audio_duration_s": 0.0,
+     "image_pixels": 1032192,
+     "denoise_step_latency_ms": 2316.2843704223633,
+     "output_unit_type": "image",
+     "output_unit_count": 1,
+     "serving_time_to_first_output_ms": 46324.424266815186,
+     "image_time_to_first_output_ms": 46324.246883392334,
+     "time_per_output_unit_ms": 0.0,
+     "inter_output_latency_ms": 0.0,
+     "inter_output_latencies_ms": [],
+     "vllm_ttft_ms": 0.0,
+     "vllm_tpot_ms": 0.0,
+     "vllm_itl_ms": 0.0,
+     "vllm_itls_ms": [],
+     "stage_name": "diffusion"
+    }
+   },
+   "stage_id": 0,
+   "final_output_type": "image",
+   "num_tokens_in": 0,
+   "num_tokens_out": 0,
+   "stage_durations": {
+    "queue_wait_ms": 0.15303100371966138,
+    "stage_0_gen_ms": 46325.687408447266
+   },
+   "peak_memory_mb": 23524.0
+  }
+ },
+ {
+  "fixture_id": "edit-compose-four",
+  "seed": 2303,
+  "n_references": 4,
+  "steps": 20,
+  "true_cfg_scale": 1.0,
+  "size": "1344x768",
+  "wall_s": 46.44,
+  "http": 200,
+  "bytes": 4130915,
+  "sha256": "db507c15651324404b79d6e5446e43fb460797204b302c0c258be37f1e2a7a80",
+  "structure": {
+   "opaque_fraction": 0.0195,
+   "alpha_mean": 132.0,
+   "rgb_std": [
+    41.1,
+    46.1,
+    44.7
+   ]
+  },
+  "usage": null,
+  "metrics": {
+   "stage_metrics": {
+    "0": {
+     "stage_id": 0,
+     "final_output_type": "image",
+     "finish_reason": null,
+     "num_tokens_in": 0,
+     "num_tokens_out": 0,
+     "stage_gen_time_ms": 46352.77342796326,
+     "postprocess_time_ms": 0.0,
+     "audio_frames": 0,
+     "audio_sample_rate": 0,
+     "audio_duration_s": 0.0,
+     "image_pixels": 1032192,
+     "denoise_step_latency_ms": 2317.638671398163,
+     "output_unit_type": "image",
+     "output_unit_count": 1,
+     "serving_time_to_first_output_ms": 46352.887868881226,
+     "image_time_to_first_output_ms": 46352.74004936218,
+     "time_per_output_unit_ms": 0.0,
+     "inter_output_latency_ms": 0.0,
+     "inter_output_latencies_ms": [],
+     "vllm_ttft_ms": 0.0,
+     "vllm_tpot_ms": 0.0,
+     "vllm_itl_ms": 0.0,
+     "vllm_itls_ms": [],
+     "stage_name": "diffusion"
+    }
+   },
+   "stage_id": 0,
+   "final_output_type": "image",
+   "num_tokens_in": 0,
+   "num_tokens_out": 0,
+   "stage_durations": {
+    "queue_wait_ms": 0.1095690022339113,
+    "stage_0_gen_ms": 46352.77342796326
+   },
+   "peak_memory_mb": 23524.0
+  }
+ },
+ {
+  "fixture_id": "edit-compose-four",
+  "seed": 2304,
+  "n_references": 4,
+  "steps": 20,
+  "true_cfg_scale": 1.0,
+  "size": "1344x768",
+  "wall_s": 46.48,
+  "http": 200,
+  "bytes": 4130915,
+  "sha256": "5481e029af2cb8ef3140565c615a5abc97ed755ff180b1970badf3a997c60ac4",
+  "structure": {
+   "opaque_fraction": 0.0195,
+   "alpha_mean": 131.7,
+   "rgb_std": [
+    41.0,
+    46.1,
+    44.5
+   ]
+  },
+  "usage": null,
+  "metrics": {
+   "stage_metrics": {
+    "0": {
+     "stage_id": 0,
+     "final_output_type": "image",
+     "finish_reason": null,
+     "num_tokens_in": 0,
+     "num_tokens_out": 0,
+     "stage_gen_time_ms": 46385.611057281494,
+     "postprocess_time_ms": 0.0,
+     "audio_frames": 0,
+     "audio_sample_rate": 0,
+     "audio_duration_s": 0.0,
+     "image_pixels": 1032192,
+     "denoise_step_latency_ms": 2319.2805528640747,
+     "output_unit_type": "image",
+     "output_unit_count": 1,
+     "serving_time_to_first_output_ms": 46385.305404663086,
+     "image_time_to_first_output_ms": 46385.130643844604,
+     "time_per_output_unit_ms": 0.0,
+     "inter_output_latency_ms": 0.0,
+     "inter_output_latencies_ms": [],
+     "vllm_ttft_ms": 0.0,
+     "vllm_tpot_ms": 0.0,
+     "vllm_itl_ms": 0.0,
+     "vllm_itls_ms": [],
+     "stage_name": "diffusion"
+    }
+   },
+   "stage_id": 0,
+   "final_output_type": "image",
+   "num_tokens_in": 0,
+   "num_tokens_out": 0,
+   "stage_durations": {
+    "queue_wait_ms": 0.14768500113859773,
+    "stage_0_gen_ms": 46385.611057281494
+   },
+   "peak_memory_mb": 23524.0
+  }
+ }
+]
diff --git a/probe/issue-5/evidence/rtx3090/runs-summary.json b/probe/issue-5/evidence/rtx3090/runs-summary.json
new file mode 100644
index 000000000..5911d4c82
--- /dev/null
+++ b/probe/issue-5/evidence/rtx3090/runs-summary.json
@@ -0,0 +1,60 @@
+{
+ "schema": "issue5-runs-summary-v1",
+ "probed_utc": "2026-10-08",
+ "host": "cachyos-alexander (RTX 3090 24 GB, sm_86, driver 615.71.09, CUDA 13.4)",
+ "stack": {
+  "vllm_omni": "ISSUE-5 branch e778d21f..67e17786 over main 8484d85d1",
+  "vllm": "0.31.0",
+  "torch": "2.13.0+cu130",
+  "compressed_tensors": "0.17.0",
+  "transformers": "5.14.1",
+  "diffusers": "0.41.0.dev0 (git 122b1e11)"
+ },
+ "artifact": {
+  "path": "~/.local/state/multi-model-helper-server/state/image-service/artifacts/qwen-image-2.1-dit-int8wo",
+  "quantized_params": 6979321856,
+  "bf16_boundary_params": 135802880,
+  "weights_file_bytes": 7256783616,
+  "mean_rel_l2_quant_error": 0.010959,
+  "max_rel_l2_quant_error": 0.017284,
+  "roundtrip_verification": "unpack+dequant vs sealed BF16: rel_l2 0.0098-0.0117 per tensor (see artifact-verify.txt)"
+ },
+ "serve_run": {
+  "label": "int8wo",
+  "command": "vllm serve <artifact> --omni --port 8091 --served-model-name Qwen/Qwen-Image-2.1 --enforce-eager --enable-cpu-offload",
+  "init": "healthy in ~40 s; per-component quantization auto-detected from transformer/config.json (transformer -> compressed-tensors, encoder/VAE BF16); CompressedTensorsWNA16 + MarlinLinearKernel selected on sm_86",
+  "failed_precursor_runs": [
+   "int8wo (first): init OOM - loader treated the pre-quantized checkpoint as online quantization and loaded all components on the GPU (encoder 17 GiB + VAE + packed DiT skeleton > 24 GiB); fixed by the offline-detection loader change on this branch",
+   "int8wo-expandable: same OOM with PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True (genuine capacity, not fragmentation); also polluted by a leftover 586 MiB DiffusionWorker from an interrupted test run"
+  ]
+ },
+ "capacity_matrix": {
+  "fixtures": {"1-ref": "edit-expression-ysolde", "2-ref": "edit-compose-pair", "3-ref": "edit-compose-three", "4-ref": "edit-compose-four"},
+  "seeds": [2301, 2302, 2303, 2304],
+  "requests": 16,
+  "http_200": 16,
+  "wall_s_mean": {"1-ref": 31.0, "2-ref": 36.6, "3-ref": 41.4, "4-ref": 46.4},
+  "nvidia_smi_peak_mib": 23935,
+  "bf16_reference_walls": {"1-ref": 34.2, "2-ref": 36.7, "4-ref": "OOM at BF16 (issue 2)"},
+  "fp8_reference_walls": {"1-ref": 29.9, "2-ref": 34.1, "3-ref": 38.9, "4-ref": 44.3}
+ },
+ "structure_gate": {
+  "rule": "opaque_fraction >= 0.99 AND mean(rgb_std) >= 10",
+  "result": "FAIL 16/16",
+  "signature": "opaque 0.0-0.20 (vs 1.0 at BF16), alpha mean 114-137, rgb_std 3-93; outputs are horizontal-stripe noise, not rendered scenes",
+  "detail": "gate-and-psnr-summary.txt; per-request records in matrix.json"
+ },
+ "psnr_vs_sealed_diffusers_baseline": {
+  "edit-expression-ysolde_1ref_mean_db": 7.14,
+  "edit-compose-pair_2ref_mean_db": 7.46,
+  "edit-compose-four_4ref_mean_db": 7.11,
+  "bf16_pair_reference_mean_db": 27.57,
+  "fp8_broken_reference_mean_db": 7.11
+ },
+ "marlin_isolation": {
+  "method": "marlin_isolate.py: vLLM QKVParallelLinear/RowParallelLinear/ColumnParallelLinear built exactly as the qwen_image_21 transformer does; artifact tensors loaded through the production weight-loaders; Marlin repack on GPU; layer(x) compared against float32 x @ dequant(W)^T",
+  "results_block0_artifact": "NaN output, all four layer kinds, bf16 AND fp16 activations; loaded weight_packed bitwise-equal to artifact (incl. fused q/k/v stacked shards); scales finite pre and post repack (marlin-isolate-block0.json)",
+  "results_synthetic_random_int8": "NaN identically - the checkpoint and its quantization are exonerated (marlin-isolate-synthetic.json)",
+  "conclusion": "the 8-bit Marlin W8A16 path on this stack (sm_86, vllm 0.31.0, torch 2.13.0+cu130) produces NaN for int8 weight-only layers although query_marlin_supported_quant_types accepts uint8b128 on the card; choose_mp_linear_kernel offers no non-Marlin fallback for W8A16 on sm_86 (Machete is sm90+, ExLlama int4-only)"
+ }
+}
diff --git a/probe/issue-5/marlin_isolate.py b/probe/issue-5/marlin_isolate.py
new file mode 100644
index 000000000..7894dde86
--- /dev/null
+++ b/probe/issue-5/marlin_isolate.py
@@ -0,0 +1,239 @@
+#!/usr/bin/env python
+"""Issue 5 diagnostic: isolate the compressed-tensors W8A16 serve path per layer.
+
+Builds vLLM parallel linear layers exactly as the qwen_image_21 transformer
+does (fused QKV, row-parallel to_out, img_mlp proj/out), loads the int8
+artifact's packed tensors through the production weight-loader machinery,
+runs the Marlin repack (process_weights_after_loading) on the GPU, and
+compares each layer's forward output against a float32 reference computed
+from the artifact's own unpacked+dequantized weights.
+
+Expected rel-L2 when the load->repack->apply chain is correct: ~1-2%
+(int8 quantization error plus bf16 activation rounding). A large error
+pins corruption inside the serve path with the checkpoint already
+verified correct (verify_artifact.py).
+
+Usage: marlin_isolate.py --artifact ARTIFACT_DIR [--block N]
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import pathlib
+
+import torch
+from compressed_tensors.compressors.pack_quantized.helpers import unpack_from_int32
+from safetensors.torch import load_file
+
+
+def dequantized_weight(art: dict, base: str) -> torch.Tensor:
+    shape = tuple(art[f"{base}_shape"].tolist())
+    q = unpack_from_int32(art[f"{base}_packed"].cpu(), 8, shape)
+    return q.to(torch.float32) * art[f"{base}_scale"].cpu().to(torch.float32)
+
+
+def load_tensor(layer: torch.nn.Module, param_name: str, tensor: torch.Tensor, shard: str | None) -> None:
+    param = getattr(layer, param_name)
+    if shard is None:
+        param.weight_loader(param, tensor)
+    else:
+        param.weight_loader(param, tensor, shard)
+
+
+def check(name: str, layer: torch.nn.Module, art: dict, base: str, shards: list[tuple[str, str]], in_features: int, dtype: torch.dtype) -> dict:
+    device = torch.device("cuda")
+    for suffix in ("weight_packed", "weight_scale", "weight_shape"):
+        for ckpt_base, shard in shards:
+            load_tensor(layer, suffix, art[f"{ckpt_base}.{suffix}"].to(device), shard)
+
+    # Bitwise check of the loaded packed weights against the artifact.
+    loaded = layer.weight_packed.detach().cpu()
+    if len(shards) > 1:
+        ref_packed = torch.cat([art[f"{b}.weight_packed"] for b, _ in shards], dim=0)
+    else:
+        ref_packed = art[f"{base}.weight_packed"]
+    packed_ok = bool(torch.equal(loaded, ref_packed))
+    scale_loaded = layer.weight_scale.detach().float()
+    scale_ok = bool(torch.isfinite(scale_loaded).all())
+
+    layer.quant_method.process_weights_after_loading(layer)
+    repacked_scale = getattr(layer, "weight_scale", None)
+    scale_post = (
+        {
+            "finite": bool(torch.isfinite(repacked_scale.detach().float()).all()),
+            "min": float(repacked_scale.detach().float().min()),
+            "max": float(repacked_scale.detach().float().max()),
+        }
+        if repacked_scale is not None
+        else {}
+    )
+
+    torch.manual_seed(2301)
+    ref_w = torch.cat([dequantized_weight(art, f"{b}.weight") for b, _ in shards], dim=0) if len(shards) > 1 else dequantized_weight(art, f"{base}.weight")
+    outs = {}
+    for act_dtype in (torch.bfloat16, torch.float16):
+        x = torch.randn(512, in_features, dtype=act_dtype, device=device)
+        result = layer(x)
+        out = (result[0] if isinstance(result, tuple) else result).float()
+        ref = x.float() @ ref_w.T.to(device)
+        finite = bool(torch.isfinite(out).all())
+        rel = float((out - ref).norm() / ref.norm())
+        outs[str(act_dtype).replace("torch.", "")] = rel if finite else "NaN"
+
+    return {
+        "layer": name,
+        "rel_l2_by_act_dtype": outs,
+        "packed_bitwise_ok": packed_ok,
+        "scale_finite_pre_repack": scale_ok,
+        "scale_post_repack": scale_post,
+    }
+
+
+def synthetic_artifact(block: int) -> dict[str, torch.Tensor]:
+    """Random int8-quantized tensors in the artifact's layout, for stack-only isolation."""
+    from compressed_tensors.compressors.pack_quantized.helpers import pack_to_int32
+
+    def quant(name: str, out: int, inp: int) -> None:
+        w = torch.randn(out, inp, dtype=torch.float32)
+        scale = w.abs().amax(dim=1, keepdim=True).clamp_min(1e-12) / 127.0
+        q = torch.round(w / scale).clamp(-127, 127).to(torch.int8)
+        art[f"{name}.weight_packed"] = pack_to_int32(q.contiguous(), 8).contiguous()
+        art[f"{name}.weight_scale"] = scale.to(torch.float32).contiguous()
+        art[f"{name}.weight_shape"] = torch.tensor([out, inp], dtype=torch.int64)
+
+    art: dict[str, torch.Tensor] = {}
+    p = f"transformer_blocks.{block}"
+    for part in ("to_q", "to_k", "to_v"):
+        quant(f"{p}.attn.{part}", 4096, 4096)
+    quant(f"{p}.attn.to_out.0", 4096, 4096)
+    quant(f"{p}.img_mlp.proj", 12288, 4096)
+    quant(f"{p}.img_mlp.out", 4096, 12288)
+    return art
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--artifact", type=pathlib.Path, required=True)
+    parser.add_argument("--block", type=int, default=0)
+    parser.add_argument(
+        "--synthetic",
+        action="store_true",
+        help="ignore --artifact; use random int8 weights (isolates the stack from the checkpoint)",
+    )
+    args = parser.parse_args()
+
+    import vllm.distributed
+    from vllm.config import VllmConfig, set_current_vllm_config
+    from vllm.distributed import (
+        destroy_distributed_environment,
+        destroy_model_parallel,
+        initialize_model_parallel,
+    )
+    from vllm.model_executor.layers.linear import (
+        ColumnParallelLinear,
+        QKVParallelLinear,
+        RowParallelLinear,
+    )
+
+    vllm.distributed.init_distributed_environment(
+        backend="gloo",
+        world_size=1,
+        rank=0,
+        local_rank=0,
+        distributed_init_method="tcp://127.0.0.1:29601",
+    )
+    with set_current_vllm_config(VllmConfig()):
+        initialize_model_parallel(tensor_model_parallel_size=1)
+
+    config = json.loads((args.artifact / "transformer" / "config.json").read_text())
+    from vllm_omni.quantization import build_quant_config, resolve_component_quant_config
+
+    quant_config = resolve_component_quant_config(
+        build_quant_config(config["quantization_config"]), "transformer"
+    )
+    if args.synthetic:
+        art = synthetic_artifact(args.block)
+    else:
+        art = load_file(str(args.artifact / "transformer" / "diffusion_pytorch_model.safetensors"))
+    b = args.block
+    p = f"transformer.transformer_blocks.{b}"
+
+    results = []
+    with torch.device("cuda"):
+        to_qkv = QKVParallelLinear(
+            hidden_size=4096, head_size=128, total_num_heads=32, bias=False,
+            quant_config=quant_config, prefix=f"{p}.attn.to_qkv",
+        )
+        results.append(
+            check(
+                "attn.to_qkv (fused q/k/v)",
+                to_qkv,
+                art,
+                f"transformer_blocks.{b}.attn.to_qkv",
+                [
+                    (f"transformer_blocks.{b}.attn.to_q", "q"),
+                    (f"transformer_blocks.{b}.attn.to_k", "k"),
+                    (f"transformer_blocks.{b}.attn.to_v", "v"),
+                ],
+                4096,
+                torch.bfloat16,
+            )
+        )
+
+        to_out = RowParallelLinear(
+            4096, 4096, bias=False, input_is_parallel=True, return_bias=False,
+            quant_config=quant_config, prefix=f"{p}.attn.to_out.0",
+        )
+        results.append(
+            check(
+                "attn.to_out.0 (row)",
+                to_out,
+                art,
+                f"transformer_blocks.{b}.attn.to_out.0",
+                [(f"transformer_blocks.{b}.attn.to_out.0", None)],
+                4096,
+                torch.bfloat16,
+            )
+        )
+
+        proj = ColumnParallelLinear(
+            4096, 12288, bias=False, gather_output=False, return_bias=False,
+            quant_config=quant_config, prefix=f"{p}.img_mlp.proj",
+        )
+        results.append(
+            check(
+                "img_mlp.proj (column)",
+                proj,
+                art,
+                f"transformer_blocks.{b}.img_mlp.proj",
+                [(f"transformer_blocks.{b}.img_mlp.proj", None)],
+                4096,
+                torch.bfloat16,
+            )
+        )
+
+        mlp_out = RowParallelLinear(
+            12288, 4096, bias=False, input_is_parallel=True, return_bias=False,
+            quant_config=quant_config, prefix=f"{p}.img_mlp.out",
+        )
+        results.append(
+            check(
+                "img_mlp.out (row, wide-in)",
+                mlp_out,
+                art,
+                f"transformer_blocks.{b}.img_mlp.out",
+                [(f"transformer_blocks.{b}.img_mlp.out", None)],
+                12288,
+                torch.bfloat16,
+            )
+        )
+
+    destroy_model_parallel()
+    destroy_distributed_environment()
+    print(json.dumps(results, indent=1))
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())