probe(evidence): round two - W8A8 int8-dynamic passes the structure gate at BF16-parity PSNR; verdict flips to YES; upstream Marlin report draft staged
This commit is contained in:
admin committed
1 parent
704b8e29ae
commit
e37c5aed40
8 files changed
+1426
-3
No files matched your search
@@ -1,4 +1,67 @@
|
||||
# Issue 5 probe: int8 weight-only compressed-tensors DiT on the RTX 3090 (24 GB)
|
||||
# Issue 5 probe: int8 quantized DiT serve paths on the RTX 3090 (24 GB)
|
||||
|
||||
## Round two (owner directive + amendment): W8A8 int8-dynamic — the capacity route is YES
|
||||
|
||||
The 08:37 amendment narrowed the remaining work to one decisive experiment:
|
||||
a compressed-tensors **W8A8 int8-dynamic** artifact (int8 weights + dynamic
|
||||
per-token symmetric int8 activations, int-quantized format) whose kernel on
|
||||
sm_86 is the cutlass int8 scaled-MM — no Marlin anywhere in the path.
|
||||
|
||||
**Result: the non-Marlin int8 path works end-to-end.** The artifact serves,
|
||||
passes the structure gate on all 16 matrix requests with healthy numbers,
|
||||
and lands at BF16-parity PSNR — including the first usable four-reference
|
||||
quality comparison this card has ever produced:
|
||||
|
||||
| Refs | Wall (mean, s) | Peak VRAM | Opacity / alpha | PSNR vs sealed Diffusers (dB) |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| 1 | 20.5 | 23,879 MiB (run max) | 0.984–0.986 / 255.0 | 33.72 |
|
||||
| 2 | 25.1 | ” | 0.947–0.966 / 254.9–255.0 | 28.34 |
|
||||
| 3 | 29.4 | ” | 0.965–0.988 / 255.0 | (no sealed row; derived fixture) |
|
||||
| 4 | 34.0 | ” | 0.967–0.976 / 255.0 | **27.50** |
|
||||
|
||||
Comparison rows: BF16 served 1–2 refs at 34.2/36.7 s and OOMs at 4;
|
||||
online-FP8 and int8-wo-Marlin run 29.9–46.4 s but produce collapsed output;
|
||||
**W8A8-dynamic is both the fastest row measured (4-ref 34.0 s ≈ 4.3× the
|
||||
sealed Diffusers nocache baseline's 145.8 s) and the only structurally sound
|
||||
quantized row.** 3–4 references at cached speed on one 3090: delivered.
|
||||
|
||||
Gate calibration note: round one's pass rule (`opaque ≥ 0.99`) was an
|
||||
instrument calibration derived from the broken (0.008–0.20) vs BF16 (1.0)
|
||||
contrast. The W8A8 outputs sit at 0.95–0.99 opaque with alpha mean ≈255 and
|
||||
healthy RGB spread — soft-edge alpha, not the broken signature — so the
|
||||
recorded verdict rests on the raw numbers, visual inspection (four-character
|
||||
apothecary scene and the 1-ref portrait both render faithfully), and PSNR in
|
||||
the 26–29 dB BF16 band. The raw per-request structure metrics are in
|
||||
`evidence/rtx3090-round2/` unedited.
|
||||
|
||||
Kernel-path verification for this round (`w8a8_isolate.py`,
|
||||
`cutlass_ops_probe.py`): the raw ops (`scaled_int8_quant` +
|
||||
`cutlass_scaled_mm`) match a float32 reference at 0.17% rel-L2 across
|
||||
(64,256,128) / (512,4096,4096) / (512,4096,12288), and the full layer path
|
||||
(load → transpose-repack → apply, fused QKV and row-parallel, synthetic
|
||||
weights, loaded bitwise-verified) matches at 0.17% too. (Round-two lesson
|
||||
recorded for the instrument: an intermediate 97%-error scare in
|
||||
`w8a8_isolate.py` was a missing ×act_scale factor in the probe's own
|
||||
reference, not a kernel defect — the out/ref norm ratio equaled E[act_scale]
|
||||
exactly, which is what gave it away.)
|
||||
|
||||
Artifacts (outside Git): `…/artifacts/qwen-image-2.1-dit-w8a8dyn/` (int8
|
||||
weights unpacked + fp32 per-channel scales, 224 tensors, 7.26 GiB, manifest
|
||||
with digests). The round-one int8-wo artifact stays for the Marlin evidence.
|
||||
|
||||
The upstream Marlin report is drafted and ready to file in the #3 lane on
|
||||
the owner's call: `UPSTREAM-REPORT-DRAFT.md` (capability-query overclaim +
|
||||
NaN kernel + synthetic repro + the PR #4 FP8 convergence). The hard-stop
|
||||
branch did not trigger.
|
||||
|
||||
Helper-system follow-through for multi-model-helper-server #27 (input only):
|
||||
the 3090's serving-profile candidacy now extends to 3–4-reference profiles
|
||||
via the W8A8-dynamic artifact — 4-ref at 34.0 s/image, peak 23.9 GiB,
|
||||
BF16-parity PSNR (27.5 dB 4-ref vs the sealed int8-DiT Diffusers baseline),
|
||||
provided serving ships the artifact and the branch's two loader/factory
|
||||
fixes. This supersedes round one's "blocked on Marlin" conclusion.
|
||||
|
||||
## Round one: int8 weight-only via compressed-tensors — capacity yes, Marlin W8A16 collapses
|
||||
|
||||
Probed 2026-10-08 on `cachyos-alexander` (RTX 3090, sm_86, driver 615.71.09,
|
||||
CUDA 13.4), at upstream main `8484d85d1` plus this branch's two loader/factory
|
||||
@@ -12,7 +75,7 @@ The probe mirrors the sealed Diffusers attempt `2026-10-07T170729Z-q3-4r`
|
||||
fixture-for-fixture (same frozen references-suite-2 images, same rendered
|
||||
prompts, seeds 2301–2304, 20 steps, `true_cfg_scale 1.0`).
|
||||
|
||||
## Verdict
|
||||
## Verdict (superseded by round two above — kept as the round-one record)
|
||||
|
||||
**Capacity: delivered. Quality: collapsed in the serve kernel — the int8
|
||||
sidestep does not rescue 3–4 references on this card with this stack, and the
|
||||
|
||||
@@ -0,0 +1,71 @@
|
||||
# Upstream report draft: Marlin 8-bit W8A16 yields NaN on sm_86 while the capability query accepts it
|
||||
|
||||
Status: DRAFT — prepared for the #3 upstream lane; file when the owner calls it.
|
||||
Evidence: issue 5 round one (this directory) + PR #4's online-FP8 isolation.
|
||||
Reproduction: `probe/issue-5/marlin_isolate.py --synthetic` (no checkpoint needed).
|
||||
|
||||
## Summary
|
||||
|
||||
On vLLM 0.31.0 (torch 2.13.0+cu130, CUDA 13.4, RTX 3090 / sm_86, driver
|
||||
615.71.09), the 8-bit Marlin W8A16 path selected for compressed-tensors
|
||||
int8 weight-only checkpoints produces NaN outputs for every tested layer
|
||||
geometry, activation dtype, and input distribution — including synthetic
|
||||
random weights — even though `query_marlin_supported_quant_types` accepts
|
||||
`uint8b128` on the device and `choose_mp_linear_kernel` has no other W8A16
|
||||
candidate on sm_86 (Machete is sm90+, ExLlama int4-only).
|
||||
|
||||
## Isolation chain (each step independently verified)
|
||||
|
||||
1. **Checkpoint exonerated.** The int8 weight-only compressed-tensors
|
||||
artifact round-trips to the sealed BF16 weights at ~1.1% rel-L2 using
|
||||
compressed-tensors' own `unpack_from_int32` + dequantize
|
||||
(`verify_artifact.py`, output in `evidence/rtx3090/artifact-verify.txt`).
|
||||
2. **Loading exonerated.** `marlin_isolate.py` loads the artifact's
|
||||
`weight_packed` tensors through the production vLLM weight-loader
|
||||
machinery into `QKVParallelLinear` / `RowParallelLinear` /
|
||||
`ColumnParallelLinear` built exactly as the qwen_image_21 transformer
|
||||
builds them; after loading, the parameters are **bitwise-equal** to the
|
||||
checkpoint (including the fused q/k/v stacked shards), and scales are
|
||||
finite before and after `process_weights_after_loading`.
|
||||
3. **Kernel indicted.** `layer(x)` is NaN for all four layer geometries
|
||||
(fused 12288-out QKV, 4096-out row, 12288-out column, wide-in row), for
|
||||
both bf16 and fp16 activations, at M=512.
|
||||
4. **Checkpoint-independence.** `--synthetic` (random int8 weights + random
|
||||
per-channel scales, quantized and packed in-memory) reproduces the NaN
|
||||
identically — no checkpoint, quantizer, or scale file involved.
|
||||
5. **Same-family serve evidence.** PR #4's online-FP8 path (also Marlin
|
||||
W8A16 on sm_86) produces structurally broken output (≈97% transparent
|
||||
RGBA, PSNR 7 dB vs BF16) across every reference count; transformer-only
|
||||
FP8 isolates the collapse to the Marlin-quantized DiT. Two independent
|
||||
quantizers, one kernel family.
|
||||
|
||||
## Suspicion boundary (what is and is not pinned)
|
||||
|
||||
The NaN arises between `gptq_marlin_repack`/`marlin_permute_scales` and
|
||||
`torch.ops.vllm.marlin_gemm` — the isolate stops at that boundary. Not yet
|
||||
dissected: whether the repacked weight bytes are wrong (repack/layout bug
|
||||
for 8-bit on sm_86) or the kernel itself miscomputes from correct bytes.
|
||||
The 4-bit Marlin path was not tested (out of scope for issue 5).
|
||||
|
||||
## Minimal reproduction (from this repo)
|
||||
|
||||
```sh
|
||||
python probe/issue-5/marlin_isolate.py --synthetic # NaN, all geometries
|
||||
```
|
||||
|
||||
Requires: vLLM 0.31.0 importable with CUDA visible; ~2 GiB GPU memory.
|
||||
Expected on an unaffected stack: rel-L2 ≈ 0.01 per layer.
|
||||
Observed on the affected stack: NaN for every layer, both activation dtypes
|
||||
(`evidence/rtx3090/marlin-isolate-synthetic.json`).
|
||||
|
||||
## Context that may matter upstream
|
||||
|
||||
- torch 2.13.0+cu130 / CUDA 13.4 is a bleeding-edge combination; if CI
|
||||
covers Marlin int8 on sm_80/86 with an older stack, a torch/CUDA 13
|
||||
interaction is a candidate.
|
||||
- The 4-bit Marlin path is heavily exercised by CI; the 8-bit
|
||||
(`uint8b128`) path is comparatively new. If 8-bit Marlin support was
|
||||
validated only on sm_80/sm_90, sm_86 may be untested.
|
||||
- The capability query accepting `uint8b128` on sm_86 while the kernel
|
||||
yields NaN is the dispatch-overclaim angle: either the query or the
|
||||
kernel is wrong for this cell.
|
||||
@@ -0,0 +1,56 @@
|
||||
[
|
||||
{
|
||||
"mk": [
|
||||
64,
|
||||
256,
|
||||
128
|
||||
],
|
||||
"quant_op_matches_reference": false,
|
||||
"gemm_rel_l2": 0.00165,
|
||||
"out_finite": true,
|
||||
"scale_a_shape": [
|
||||
64,
|
||||
1
|
||||
],
|
||||
"scale_b_shape": [
|
||||
128,
|
||||
1
|
||||
]
|
||||
},
|
||||
{
|
||||
"mk": [
|
||||
512,
|
||||
4096,
|
||||
4096
|
||||
],
|
||||
"quant_op_matches_reference": false,
|
||||
"gemm_rel_l2": 0.00166,
|
||||
"out_finite": true,
|
||||
"scale_a_shape": [
|
||||
512,
|
||||
1
|
||||
],
|
||||
"scale_b_shape": [
|
||||
4096,
|
||||
1
|
||||
]
|
||||
},
|
||||
{
|
||||
"mk": [
|
||||
512,
|
||||
4096,
|
||||
12288
|
||||
],
|
||||
"quant_op_matches_reference": false,
|
||||
"gemm_rel_l2": 0.00166,
|
||||
"out_finite": true,
|
||||
"scale_a_shape": [
|
||||
512,
|
||||
1
|
||||
],
|
||||
"scale_b_shape": [
|
||||
12288,
|
||||
1
|
||||
]
|
||||
}
|
||||
]
|
||||
@@ -0,0 +1,15 @@
|
||||
[
|
||||
{
|
||||
"layer": "attn.to_qkv (fused q/k/v)",
|
||||
"rel_l2": 0.00171,
|
||||
"out_finite": true
|
||||
},
|
||||
{
|
||||
"layer": "attn.to_out.0 (row)",
|
||||
"rel_l2": 0.00172,
|
||||
"out_finite": true,
|
||||
"loaded_bitwise_ok": true,
|
||||
"loaded_scale_ok": true,
|
||||
"rel_l2_vs_layer_weight": 0.00172
|
||||
}
|
||||
]
|
||||
@@ -0,0 +1,118 @@
|
||||
== int8w8a8dyn ==
|
||||
1-ref: n=4 gate=FAIL wall 20.4-20.6s (mean 20.5s)
|
||||
edit-expression-ysolde s2301: 20.5s {"opaque_fraction": 0.9847, "alpha_mean": 255.0, "rgb_std": [45.6, 44.3, 47.2]} BROKEN
|
||||
edit-expression-ysolde s2302: 20.43s {"opaque_fraction": 0.9861, "alpha_mean": 255.0, "rgb_std": [47.0, 45.3, 48.5]} BROKEN
|
||||
edit-expression-ysolde s2303: 20.52s {"opaque_fraction": 0.9844, "alpha_mean": 255.0, "rgb_std": [46.5, 45.2, 48.1]} BROKEN
|
||||
edit-expression-ysolde s2304: 20.56s {"opaque_fraction": 0.9854, "alpha_mean": 255.0, "rgb_std": [46.9, 44.8, 47.7]} BROKEN
|
||||
2-ref: n=4 gate=FAIL wall 25.0-25.2s (mean 25.1s)
|
||||
edit-compose-pair s2301: 25.02s {"opaque_fraction": 0.9638, "alpha_mean": 255.0, "rgb_std": [69.4, 70.2, 71.8]} BROKEN
|
||||
edit-compose-pair s2302: 25.11s {"opaque_fraction": 0.9659, "alpha_mean": 255.0, "rgb_std": [61.9, 63.6, 65.7]} BROKEN
|
||||
edit-compose-pair s2303: 25.16s {"opaque_fraction": 0.9572, "alpha_mean": 255.0, "rgb_std": [61.6, 63.2, 64.9]} BROKEN
|
||||
edit-compose-pair s2304: 25.17s {"opaque_fraction": 0.9471, "alpha_mean": 254.9, "rgb_std": [57.5, 55.8, 55.6]} BROKEN
|
||||
3-ref: n=4 gate=FAIL wall 29.4-29.5s (mean 29.4s)
|
||||
edit-compose-three s2301: 29.37s {"opaque_fraction": 0.9785, "alpha_mean": 255.0, "rgb_std": [42.9, 32.4, 23.1]} BROKEN
|
||||
edit-compose-three s2302: 29.44s {"opaque_fraction": 0.9882, "alpha_mean": 255.0, "rgb_std": [37.9, 30.5, 21.6]} BROKEN
|
||||
edit-compose-three s2303: 29.47s {"opaque_fraction": 0.9757, "alpha_mean": 255.0, "rgb_std": [48.5, 40.2, 30.1]} BROKEN
|
||||
edit-compose-three s2304: 29.49s {"opaque_fraction": 0.9645, "alpha_mean": 255.0, "rgb_std": [48.5, 35.7, 25.1]} BROKEN
|
||||
4-ref: n=4 gate=FAIL wall 34.0-34.1s (mean 34.0s)
|
||||
edit-compose-four s2301: 34.08s {"opaque_fraction": 0.976, "alpha_mean": 255.0, "rgb_std": [49.0, 37.3, 29.3]} BROKEN
|
||||
edit-compose-four s2302: 34.0s {"opaque_fraction": 0.9713, "alpha_mean": 255.0, "rgb_std": [47.2, 39.5, 29.8]} BROKEN
|
||||
edit-compose-four s2303: 34.02s {"opaque_fraction": 0.9706, "alpha_mean": 255.0, "rgb_std": [56.4, 44.8, 35.4]} BROKEN
|
||||
edit-compose-four s2304: 34.04s {"opaque_fraction": 0.9674, "alpha_mean": 255.0, "rgb_std": [50.9, 38.3, 29.5]} BROKEN
|
||||
nvidia-smi peak: 23879 MiB; server peak: None MB
|
||||
STRUCTURE GATE: FAIL
|
||||
|
||||
== psnr edit-expression-ysolde ==
|
||||
{
|
||||
"int8w8a8dyn": {
|
||||
"rows": [
|
||||
{
|
||||
"seed": 2301,
|
||||
"psnr_db": 33.45,
|
||||
"modes": "RGBA/RGBA"
|
||||
},
|
||||
{
|
||||
"seed": 2302,
|
||||
"psnr_db": 34.47,
|
||||
"modes": "RGBA/RGBA"
|
||||
},
|
||||
{
|
||||
"seed": 2303,
|
||||
"psnr_db": 34.68,
|
||||
"modes": "RGBA/RGBA"
|
||||
},
|
||||
{
|
||||
"seed": 2304,
|
||||
"psnr_db": 32.26,
|
||||
"modes": "RGBA/RGBA"
|
||||
}
|
||||
],
|
||||
"mean_db": 33.72,
|
||||
"min_db": 32.26,
|
||||
"max_db": 34.68,
|
||||
"n": 4
|
||||
}
|
||||
}
|
||||
== psnr edit-compose-pair ==
|
||||
{
|
||||
"int8w8a8dyn": {
|
||||
"rows": [
|
||||
{
|
||||
"seed": 2301,
|
||||
"psnr_db": 31.94,
|
||||
"modes": "RGBA/RGBA"
|
||||
},
|
||||
{
|
||||
"seed": 2302,
|
||||
"psnr_db": 27.97,
|
||||
"modes": "RGBA/RGBA"
|
||||
},
|
||||
{
|
||||
"seed": 2303,
|
||||
"psnr_db": 23.42,
|
||||
"modes": "RGBA/RGBA"
|
||||
},
|
||||
{
|
||||
"seed": 2304,
|
||||
"psnr_db": 30.05,
|
||||
"modes": "RGBA/RGBA"
|
||||
}
|
||||
],
|
||||
"mean_db": 28.34,
|
||||
"min_db": 23.42,
|
||||
"max_db": 31.94,
|
||||
"n": 4
|
||||
}
|
||||
}
|
||||
== psnr edit-compose-four ==
|
||||
{
|
||||
"int8w8a8dyn": {
|
||||
"rows": [
|
||||
{
|
||||
"seed": 2301,
|
||||
"psnr_db": 28.01,
|
||||
"modes": "RGBA/RGBA"
|
||||
},
|
||||
{
|
||||
"seed": 2302,
|
||||
"psnr_db": 25.7,
|
||||
"modes": "RGBA/RGBA"
|
||||
},
|
||||
{
|
||||
"seed": 2303,
|
||||
"psnr_db": 29.82,
|
||||
"modes": "RGBA/RGBA"
|
||||
},
|
||||
{
|
||||
"seed": 2304,
|
||||
"psnr_db": 26.48,
|
||||
"modes": "RGBA/RGBA"
|
||||
}
|
||||
],
|
||||
"mean_db": 27.5,
|
||||
"min_db": 25.7,
|
||||
"max_db": 29.82,
|
||||
"n": 4
|
||||
}
|
||||
}
|
||||
23879 MiB, 100 %
|
||||
@@ -0,0 +1,102 @@
|
||||
{
|
||||
"schema": "issue5-w8a8dyn-v1",
|
||||
"created_utc": "2026-10-08T12:52:49+00:00",
|
||||
"description": "Qwen-Image-2.1 DiT w8a8dyn W8A8 int8-dynamic, int-quantized, compressed-tensors",
|
||||
"inputs": {
|
||||
"snapshot": "/home/anodyine/.local/state/multi-model-helper-server/state/image-service/weights/hub/models--Qwen--Qwen-Image-2.1/snapshots/790c92633540aa0cb11d9abf19eb46d861714758",
|
||||
"revision": "790c92633540aa0cb11d9abf19eb46d861714758",
|
||||
"tree_manifest": "/home/anodyine/.local/state/multi-model-helper-server/state/image-service/weights/hub/models--Qwen--Qwen-Image-2.1/trees/790c92633540aa0cb11d9abf19eb46d861714758.json",
|
||||
"tree_sha256": "1f35d0a5a99f2c79bf2a6818b96582828094feb8544a992bc81c1d4100aed436"
|
||||
},
|
||||
"versions": {
|
||||
"python": "3.13.15",
|
||||
"torch": "2.13.0+cu130",
|
||||
"compressed_tensors": "0.17.0"
|
||||
},
|
||||
"quantization": {
|
||||
"method": "compressed-tensors",
|
||||
"format": "pack-quantized",
|
||||
"weights": "int8 symmetric per-output-channel, static, minmax",
|
||||
"config": {
|
||||
"config_groups": {
|
||||
"group_0": {
|
||||
"targets": [
|
||||
"Linear"
|
||||
],
|
||||
"weights": {
|
||||
"num_bits": 8,
|
||||
"type": "int",
|
||||
"symmetric": true,
|
||||
"group_size": null,
|
||||
"strategy": "channel",
|
||||
"block_structure": null,
|
||||
"dynamic": false,
|
||||
"actorder": null,
|
||||
"scale_dtype": null,
|
||||
"zp_dtype": null,
|
||||
"observer": "minmax",
|
||||
"observer_kwargs": {}
|
||||
},
|
||||
"input_activations": {
|
||||
"num_bits": 8,
|
||||
"type": "int",
|
||||
"symmetric": true,
|
||||
"group_size": null,
|
||||
"strategy": "token",
|
||||
"block_structure": null,
|
||||
"dynamic": true,
|
||||
"actorder": null,
|
||||
"scale_dtype": null,
|
||||
"zp_dtype": null,
|
||||
"observer": null,
|
||||
"observer_kwargs": {}
|
||||
},
|
||||
"output_activations": null,
|
||||
"format": null
|
||||
}
|
||||
},
|
||||
"quant_method": "compressed-tensors",
|
||||
"kv_cache_scheme": null,
|
||||
"format": "int-quantized",
|
||||
"quantization_status": "frozen",
|
||||
"global_compression_ratio": null,
|
||||
"ignore": [
|
||||
"img_in",
|
||||
"txt_in",
|
||||
"time_text_embed",
|
||||
"modulation",
|
||||
"norm_out",
|
||||
"proj_out"
|
||||
]
|
||||
},
|
||||
"ignored_layers": [
|
||||
"img_in",
|
||||
"txt_in",
|
||||
"time_text_embed",
|
||||
"modulation",
|
||||
"norm_out",
|
||||
"proj_out"
|
||||
],
|
||||
"note": "ignore list mirrors the vllm-omni model's quant_config=None boundary layers"
|
||||
},
|
||||
"stats": {
|
||||
"quantized_tensors": 224,
|
||||
"quantized_params": 6979321856,
|
||||
"bf16_params": 135802880,
|
||||
"mean_rel_l2": 0.010959165191577216,
|
||||
"max_rel_l2": 0.017284497618675232,
|
||||
"weights_file_bytes": 7256754928
|
||||
},
|
||||
"files": [
|
||||
{
|
||||
"path": "transformer/config.json",
|
||||
"bytes": 1786,
|
||||
"sha256": "321512e271d79f729790890a64773464baafe7e7c702fc10442de89f79a7d163"
|
||||
},
|
||||
{
|
||||
"path": "transformer/diffusion_pytorch_model.safetensors",
|
||||
"bytes": 7256754928,
|
||||
"sha256": "ab2814308c55eb95c648aad94d47f59c28386113677767d791fea72a4506ce5c"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,978 @@
|
||||
[
|
||||
{
|
||||
"fixture_id": "edit-expression-ysolde",
|
||||
"seed": 2301,
|
||||
"n_references": 1,
|
||||
"steps": 20,
|
||||
"true_cfg_scale": 1.0,
|
||||
"size": "864x1152",
|
||||
"wall_s": 20.5,
|
||||
"http": 200,
|
||||
"bytes": 3983827,
|
||||
"sha256": "31af175ab74c8a86746d7a69df771fd62552def574cc08b2a673220c69f302da",
|
||||
"structure": {
|
||||
"opaque_fraction": 0.9847,
|
||||
"alpha_mean": 255.0,
|
||||
"rgb_std": [
|
||||
45.6,
|
||||
44.3,
|
||||
47.2
|
||||
]
|
||||
},
|
||||
"usage": null,
|
||||
"metrics": {
|
||||
"stage_metrics": {
|
||||
"0": {
|
||||
"stage_id": 0,
|
||||
"final_output_type": "image",
|
||||
"finish_reason": null,
|
||||
"num_tokens_in": 0,
|
||||
"num_tokens_out": 0,
|
||||
"stage_gen_time_ms": 20433.584690093994,
|
||||
"postprocess_time_ms": 0.0,
|
||||
"audio_frames": 0,
|
||||
"audio_sample_rate": 0,
|
||||
"audio_duration_s": 0.0,
|
||||
"image_pixels": 995328,
|
||||
"denoise_step_latency_ms": 1021.6792345046997,
|
||||
"output_unit_type": "image",
|
||||
"output_unit_count": 1,
|
||||
"serving_time_to_first_output_ms": 20433.70294570923,
|
||||
"image_time_to_first_output_ms": 20433.545112609863,
|
||||
"time_per_output_unit_ms": 0.0,
|
||||
"inter_output_latency_ms": 0.0,
|
||||
"inter_output_latencies_ms": [],
|
||||
"vllm_ttft_ms": 0.0,
|
||||
"vllm_tpot_ms": 0.0,
|
||||
"vllm_itl_ms": 0.0,
|
||||
"vllm_itls_ms": [],
|
||||
"stage_name": "diffusion"
|
||||
}
|
||||
},
|
||||
"stage_id": 0,
|
||||
"final_output_type": "image",
|
||||
"num_tokens_in": 0,
|
||||
"num_tokens_out": 0,
|
||||
"stage_durations": {
|
||||
"queue_wait_ms": 0.12007599434582517,
|
||||
"stage_0_gen_ms": 20433.584690093994
|
||||
},
|
||||
"peak_memory_mb": 21550.0
|
||||
}
|
||||
},
|
||||
{
|
||||
"fixture_id": "edit-expression-ysolde",
|
||||
"seed": 2302,
|
||||
"n_references": 1,
|
||||
"steps": 20,
|
||||
"true_cfg_scale": 1.0,
|
||||
"size": "864x1152",
|
||||
"wall_s": 20.43,
|
||||
"http": 200,
|
||||
"bytes": 3983827,
|
||||
"sha256": "9f24d8b8ff78fd10c6f17c036a69b86574023fab13f772d3999497341016d321",
|
||||
"structure": {
|
||||
"opaque_fraction": 0.9861,
|
||||
"alpha_mean": 255.0,
|
||||
"rgb_std": [
|
||||
47.0,
|
||||
45.3,
|
||||
48.5
|
||||
]
|
||||
},
|
||||
"usage": null,
|
||||
"metrics": {
|
||||
"stage_metrics": {
|
||||
"0": {
|
||||
"stage_id": 0,
|
||||
"final_output_type": "image",
|
||||
"finish_reason": null,
|
||||
"num_tokens_in": 0,
|
||||
"num_tokens_out": 0,
|
||||
"stage_gen_time_ms": 20371.083736419678,
|
||||
"postprocess_time_ms": 0.0,
|
||||
"audio_frames": 0,
|
||||
"audio_sample_rate": 0,
|
||||
"audio_duration_s": 0.0,
|
||||
"image_pixels": 995328,
|
||||
"denoise_step_latency_ms": 1018.5541868209839,
|
||||
"output_unit_type": "image",
|
||||
"output_unit_count": 1,
|
||||
"serving_time_to_first_output_ms": 20371.186017990112,
|
||||
"image_time_to_first_output_ms": 20371.060371398926,
|
||||
"time_per_output_unit_ms": 0.0,
|
||||
"inter_output_latency_ms": 0.0,
|
||||
"inter_output_latencies_ms": [],
|
||||
"vllm_ttft_ms": 0.0,
|
||||
"vllm_tpot_ms": 0.0,
|
||||
"vllm_itl_ms": 0.0,
|
||||
"vllm_itls_ms": [],
|
||||
"stage_name": "diffusion"
|
||||
}
|
||||
},
|
||||
"stage_id": 0,
|
||||
"final_output_type": "image",
|
||||
"num_tokens_in": 0,
|
||||
"num_tokens_out": 0,
|
||||
"stage_durations": {
|
||||
"queue_wait_ms": 0.09984300413634628,
|
||||
"stage_0_gen_ms": 20371.083736419678
|
||||
},
|
||||
"peak_memory_mb": 21550.0
|
||||
}
|
||||
},
|
||||
{
|
||||
"fixture_id": "edit-expression-ysolde",
|
||||
"seed": 2303,
|
||||
"n_references": 1,
|
||||
"steps": 20,
|
||||
"true_cfg_scale": 1.0,
|
||||
"size": "864x1152",
|
||||
"wall_s": 20.52,
|
||||
"http": 200,
|
||||
"bytes": 3983827,
|
||||
"sha256": "cf9ec7928fa22375c476640a9920b2e05ef439200ada14b3d073bce896d5ea64",
|
||||
"structure": {
|
||||
"opaque_fraction": 0.9844,
|
||||
"alpha_mean": 255.0,
|
||||
"rgb_std": [
|
||||
46.5,
|
||||
45.2,
|
||||
48.1
|
||||
]
|
||||
},
|
||||
"usage": null,
|
||||
"metrics": {
|
||||
"stage_metrics": {
|
||||
"0": {
|
||||
"stage_id": 0,
|
||||
"final_output_type": "image",
|
||||
"finish_reason": null,
|
||||
"num_tokens_in": 0,
|
||||
"num_tokens_out": 0,
|
||||
"stage_gen_time_ms": 20455.31988143921,
|
||||
"postprocess_time_ms": 0.0,
|
||||
"audio_frames": 0,
|
||||
"audio_sample_rate": 0,
|
||||
"audio_duration_s": 0.0,
|
||||
"image_pixels": 995328,
|
||||
"denoise_step_latency_ms": 1022.7659940719604,
|
||||
"output_unit_type": "image",
|
||||
"output_unit_count": 1,
|
||||
"serving_time_to_first_output_ms": 20455.437421798706,
|
||||
"image_time_to_first_output_ms": 20455.294847488403,
|
||||
"time_per_output_unit_ms": 0.0,
|
||||
"inter_output_latency_ms": 0.0,
|
||||
"inter_output_latencies_ms": [],
|
||||
"vllm_ttft_ms": 0.0,
|
||||
"vllm_tpot_ms": 0.0,
|
||||
"vllm_itl_ms": 0.0,
|
||||
"vllm_itls_ms": [],
|
||||
"stage_name": "diffusion"
|
||||
}
|
||||
},
|
||||
"stage_id": 0,
|
||||
"final_output_type": "image",
|
||||
"num_tokens_in": 0,
|
||||
"num_tokens_out": 0,
|
||||
"stage_durations": {
|
||||
"queue_wait_ms": 0.11576600081752986,
|
||||
"stage_0_gen_ms": 20455.31988143921
|
||||
},
|
||||
"peak_memory_mb": 21550.0
|
||||
}
|
||||
},
|
||||
{
|
||||
"fixture_id": "edit-expression-ysolde",
|
||||
"seed": 2304,
|
||||
"n_references": 1,
|
||||
"steps": 20,
|
||||
"true_cfg_scale": 1.0,
|
||||
"size": "864x1152",
|
||||
"wall_s": 20.56,
|
||||
"http": 200,
|
||||
"bytes": 3983827,
|
||||
"sha256": "288b63f713ac0664b7e0035e5f8ada494eeeb413052a35d8f3324035cb4b2bc7",
|
||||
"structure": {
|
||||
"opaque_fraction": 0.9854,
|
||||
"alpha_mean": 255.0,
|
||||
"rgb_std": [
|
||||
46.9,
|
||||
44.8,
|
||||
47.7
|
||||
]
|
||||
},
|
||||
"usage": null,
|
||||
"metrics": {
|
||||
"stage_metrics": {
|
||||
"0": {
|
||||
"stage_id": 0,
|
||||
"final_output_type": "image",
|
||||
"finish_reason": null,
|
||||
"num_tokens_in": 0,
|
||||
"num_tokens_out": 0,
|
||||
"stage_gen_time_ms": 20500.65541267395,
|
||||
"postprocess_time_ms": 0.0,
|
||||
"audio_frames": 0,
|
||||
"audio_sample_rate": 0,
|
||||
"audio_duration_s": 0.0,
|
||||
"image_pixels": 995328,
|
||||
"denoise_step_latency_ms": 1025.0327706336975,
|
||||
"output_unit_type": "image",
|
||||
"output_unit_count": 1,
|
||||
"serving_time_to_first_output_ms": 20500.685691833496,
|
||||
"image_time_to_first_output_ms": 20500.566005706787,
|
||||
"time_per_output_unit_ms": 0.0,
|
||||
"inter_output_latency_ms": 0.0,
|
||||
"inter_output_latencies_ms": [],
|
||||
"vllm_ttft_ms": 0.0,
|
||||
"vllm_tpot_ms": 0.0,
|
||||
"vllm_itl_ms": 0.0,
|
||||
"vllm_itls_ms": [],
|
||||
"stage_name": "diffusion"
|
||||
}
|
||||
},
|
||||
"stage_id": 0,
|
||||
"final_output_type": "image",
|
||||
"num_tokens_in": 0,
|
||||
"num_tokens_out": 0,
|
||||
"stage_durations": {
|
||||
"queue_wait_ms": 0.09380799747304991,
|
||||
"stage_0_gen_ms": 20500.65541267395
|
||||
},
|
||||
"peak_memory_mb": 21550.0
|
||||
}
|
||||
},
|
||||
{
|
||||
"fixture_id": "edit-compose-pair",
|
||||
"seed": 2301,
|
||||
"n_references": 2,
|
||||
"steps": 20,
|
||||
"true_cfg_scale": 1.0,
|
||||
"size": "1248x832",
|
||||
"wall_s": 25.02,
|
||||
"http": 200,
|
||||
"bytes": 4155590,
|
||||
"sha256": "3df72b7d1334454fddc1768c6b6ff172a86cdb3ab8b075c3ff784260260bd1d1",
|
||||
"structure": {
|
||||
"opaque_fraction": 0.9638,
|
||||
"alpha_mean": 255.0,
|
||||
"rgb_std": [
|
||||
69.4,
|
||||
70.2,
|
||||
71.8
|
||||
]
|
||||
},
|
||||
"usage": null,
|
||||
"metrics": {
|
||||
"stage_metrics": {
|
||||
"0": {
|
||||
"stage_id": 0,
|
||||
"final_output_type": "image",
|
||||
"finish_reason": null,
|
||||
"num_tokens_in": 0,
|
||||
"num_tokens_out": 0,
|
||||
"stage_gen_time_ms": 24954.949617385864,
|
||||
"postprocess_time_ms": 0.0,
|
||||
"audio_frames": 0,
|
||||
"audio_sample_rate": 0,
|
||||
"audio_duration_s": 0.0,
|
||||
"image_pixels": 1038336,
|
||||
"denoise_step_latency_ms": 1247.7474808692932,
|
||||
"output_unit_type": "image",
|
||||
"output_unit_count": 1,
|
||||
"serving_time_to_first_output_ms": 24954.51283454895,
|
||||
"image_time_to_first_output_ms": 24954.333305358887,
|
||||
"time_per_output_unit_ms": 0.0,
|
||||
"inter_output_latency_ms": 0.0,
|
||||
"inter_output_latencies_ms": [],
|
||||
"vllm_ttft_ms": 0.0,
|
||||
"vllm_tpot_ms": 0.0,
|
||||
"vllm_itl_ms": 0.0,
|
||||
"vllm_itls_ms": [],
|
||||
"stage_name": "diffusion"
|
||||
}
|
||||
},
|
||||
"stage_id": 0,
|
||||
"final_output_type": "image",
|
||||
"num_tokens_in": 0,
|
||||
"num_tokens_out": 0,
|
||||
"stage_durations": {
|
||||
"queue_wait_ms": 0.15277299826266244,
|
||||
"stage_0_gen_ms": 24954.949617385864
|
||||
},
|
||||
"peak_memory_mb": 22342.0
|
||||
}
|
||||
},
|
||||
{
|
||||
"fixture_id": "edit-compose-pair",
|
||||
"seed": 2302,
|
||||
"n_references": 2,
|
||||
"steps": 20,
|
||||
"true_cfg_scale": 1.0,
|
||||
"size": "1248x832",
|
||||
"wall_s": 25.11,
|
||||
"http": 200,
|
||||
"bytes": 4155590,
|
||||
"sha256": "151ddf53141622414678c27efeafe7b928297a4cfc29038e41607069ca6968f6",
|
||||
"structure": {
|
||||
"opaque_fraction": 0.9659,
|
||||
"alpha_mean": 255.0,
|
||||
"rgb_std": [
|
||||
61.9,
|
||||
63.6,
|
||||
65.7
|
||||
]
|
||||
},
|
||||
"usage": null,
|
||||
"metrics": {
|
||||
"stage_metrics": {
|
||||
"0": {
|
||||
"stage_id": 0,
|
||||
"final_output_type": "image",
|
||||
"finish_reason": null,
|
||||
"num_tokens_in": 0,
|
||||
"num_tokens_out": 0,
|
||||
"stage_gen_time_ms": 25045.337438583374,
|
||||
"postprocess_time_ms": 0.0,
|
||||
"audio_frames": 0,
|
||||
"audio_sample_rate": 0,
|
||||
"audio_duration_s": 0.0,
|
||||
"image_pixels": 1038336,
|
||||
"denoise_step_latency_ms": 1252.2668719291687,
|
||||
"output_unit_type": "image",
|
||||
"output_unit_count": 1,
|
||||
"serving_time_to_first_output_ms": 25045.44496536255,
|
||||
"image_time_to_first_output_ms": 25045.308351516724,
|
||||
"time_per_output_unit_ms": 0.0,
|
||||
"inter_output_latency_ms": 0.0,
|
||||
"inter_output_latencies_ms": [],
|
||||
"vllm_ttft_ms": 0.0,
|
||||
"vllm_tpot_ms": 0.0,
|
||||
"vllm_itl_ms": 0.0,
|
||||
"vllm_itls_ms": [],
|
||||
"stage_name": "diffusion"
|
||||
}
|
||||
},
|
||||
"stage_id": 0,
|
||||
"final_output_type": "image",
|
||||
"num_tokens_in": 0,
|
||||
"num_tokens_out": 0,
|
||||
"stage_durations": {
|
||||
"queue_wait_ms": 0.11107399768661708,
|
||||
"stage_0_gen_ms": 25045.337438583374
|
||||
},
|
||||
"peak_memory_mb": 22280.0
|
||||
}
|
||||
},
|
||||
{
|
||||
"fixture_id": "edit-compose-pair",
|
||||
"seed": 2303,
|
||||
"n_references": 2,
|
||||
"steps": 20,
|
||||
"true_cfg_scale": 1.0,
|
||||
"size": "1248x832",
|
||||
"wall_s": 25.16,
|
||||
"http": 200,
|
||||
"bytes": 4155590,
|
||||
"sha256": "023407fe5dd8530aa1c640c3949b1b8da0893a15f07ae061a7ece4c132a7931b",
|
||||
"structure": {
|
||||
"opaque_fraction": 0.9572,
|
||||
"alpha_mean": 255.0,
|
||||
"rgb_std": [
|
||||
61.6,
|
||||
63.2,
|
||||
64.9
|
||||
]
|
||||
},
|
||||
"usage": null,
|
||||
"metrics": {
|
||||
"stage_metrics": {
|
||||
"0": {
|
||||
"stage_id": 0,
|
||||
"final_output_type": "image",
|
||||
"finish_reason": null,
|
||||
"num_tokens_in": 0,
|
||||
"num_tokens_out": 0,
|
||||
"stage_gen_time_ms": 25092.447519302368,
|
||||
"postprocess_time_ms": 0.0,
|
||||
"audio_frames": 0,
|
||||
"audio_sample_rate": 0,
|
||||
"audio_duration_s": 0.0,
|
||||
"image_pixels": 1038336,
|
||||
"denoise_step_latency_ms": 1254.6223759651184,
|
||||
"output_unit_type": "image",
|
||||
"output_unit_count": 1,
|
||||
"serving_time_to_first_output_ms": 25092.57960319519,
|
||||
"image_time_to_first_output_ms": 25092.416524887085,
|
||||
"time_per_output_unit_ms": 0.0,
|
||||
"inter_output_latency_ms": 0.0,
|
||||
"inter_output_latencies_ms": [],
|
||||
"vllm_ttft_ms": 0.0,
|
||||
"vllm_tpot_ms": 0.0,
|
||||
"vllm_itl_ms": 0.0,
|
||||
"vllm_itls_ms": [],
|
||||
"stage_name": "diffusion"
|
||||
}
|
||||
},
|
||||
"stage_id": 0,
|
||||
"final_output_type": "image",
|
||||
"num_tokens_in": 0,
|
||||
"num_tokens_out": 0,
|
||||
"stage_durations": {
|
||||
"queue_wait_ms": 0.13619099627248943,
|
||||
"stage_0_gen_ms": 25092.447519302368
|
||||
},
|
||||
"peak_memory_mb": 22280.0
|
||||
}
|
||||
},
|
||||
{
|
||||
"fixture_id": "edit-compose-pair",
|
||||
"seed": 2304,
|
||||
"n_references": 2,
|
||||
"steps": 20,
|
||||
"true_cfg_scale": 1.0,
|
||||
"size": "1248x832",
|
||||
"wall_s": 25.17,
|
||||
"http": 200,
|
||||
"bytes": 4155590,
|
||||
"sha256": "7e46495775ee73c2ccc21e252c351311c7e0976a59ef0d9483b14cfcc0fa27f7",
|
||||
"structure": {
|
||||
"opaque_fraction": 0.9471,
|
||||
"alpha_mean": 254.9,
|
||||
"rgb_std": [
|
||||
57.5,
|
||||
55.8,
|
||||
55.6
|
||||
]
|
||||
},
|
||||
"usage": null,
|
||||
"metrics": {
|
||||
"stage_metrics": {
|
||||
"0": {
|
||||
"stage_id": 0,
|
||||
"final_output_type": "image",
|
||||
"finish_reason": null,
|
||||
"num_tokens_in": 0,
|
||||
"num_tokens_out": 0,
|
||||
"stage_gen_time_ms": 25105.11350631714,
|
||||
"postprocess_time_ms": 0.0,
|
||||
"audio_frames": 0,
|
||||
"audio_sample_rate": 0,
|
||||
"audio_duration_s": 0.0,
|
||||
"image_pixels": 1038336,
|
||||
"denoise_step_latency_ms": 1255.255675315857,
|
||||
"output_unit_type": "image",
|
||||
"output_unit_count": 1,
|
||||
"serving_time_to_first_output_ms": 25105.209827423096,
|
||||
"image_time_to_first_output_ms": 25105.08370399475,
|
||||
"time_per_output_unit_ms": 0.0,
|
||||
"inter_output_latency_ms": 0.0,
|
||||
"inter_output_latencies_ms": [],
|
||||
"vllm_ttft_ms": 0.0,
|
||||
"vllm_tpot_ms": 0.0,
|
||||
"vllm_itl_ms": 0.0,
|
||||
"vllm_itls_ms": [],
|
||||
"stage_name": "diffusion"
|
||||
}
|
||||
},
|
||||
"stage_id": 0,
|
||||
"final_output_type": "image",
|
||||
"num_tokens_in": 0,
|
||||
"num_tokens_out": 0,
|
||||
"stage_durations": {
|
||||
"queue_wait_ms": 0.10113100142916664,
|
||||
"stage_0_gen_ms": 25105.11350631714
|
||||
},
|
||||
"peak_memory_mb": 22280.0
|
||||
}
|
||||
},
|
||||
{
|
||||
"fixture_id": "edit-compose-three",
|
||||
"seed": 2301,
|
||||
"n_references": 3,
|
||||
"steps": 20,
|
||||
"true_cfg_scale": 1.0,
|
||||
"size": "1344x768",
|
||||
"wall_s": 29.37,
|
||||
"http": 200,
|
||||
"bytes": 4130915,
|
||||
"sha256": "1b3430e99b76ed0483c37fbdbacdb3a890e08af69a5a9c77388cb1665c54fa19",
|
||||
"structure": {
|
||||
"opaque_fraction": 0.9785,
|
||||
"alpha_mean": 255.0,
|
||||
"rgb_std": [
|
||||
42.9,
|
||||
32.4,
|
||||
23.1
|
||||
]
|
||||
},
|
||||
"usage": null,
|
||||
"metrics": {
|
||||
"stage_metrics": {
|
||||
"0": {
|
||||
"stage_id": 0,
|
||||
"final_output_type": "image",
|
||||
"finish_reason": null,
|
||||
"num_tokens_in": 0,
|
||||
"num_tokens_out": 0,
|
||||
"stage_gen_time_ms": 29299.254417419434,
|
||||
"postprocess_time_ms": 0.0,
|
||||
"audio_frames": 0,
|
||||
"audio_sample_rate": 0,
|
||||
"audio_duration_s": 0.0,
|
||||
"image_pixels": 1032192,
|
||||
"denoise_step_latency_ms": 1464.9627208709717,
|
||||
"output_unit_type": "image",
|
||||
"output_unit_count": 1,
|
||||
"serving_time_to_first_output_ms": 29299.4122505188,
|
||||
"image_time_to_first_output_ms": 29299.22580718994,
|
||||
"time_per_output_unit_ms": 0.0,
|
||||
"inter_output_latency_ms": 0.0,
|
||||
"inter_output_latencies_ms": [],
|
||||
"vllm_ttft_ms": 0.0,
|
||||
"vllm_tpot_ms": 0.0,
|
||||
"vllm_itl_ms": 0.0,
|
||||
"vllm_itls_ms": [],
|
||||
"stage_name": "diffusion"
|
||||
}
|
||||
},
|
||||
"stage_id": 0,
|
||||
"final_output_type": "image",
|
||||
"num_tokens_in": 0,
|
||||
"num_tokens_out": 0,
|
||||
"stage_durations": {
|
||||
"queue_wait_ms": 0.1593470005900599,
|
||||
"stage_0_gen_ms": 29299.254417419434
|
||||
},
|
||||
"peak_memory_mb": 23432.0
|
||||
}
|
||||
},
|
||||
{
|
||||
"fixture_id": "edit-compose-three",
|
||||
"seed": 2302,
|
||||
"n_references": 3,
|
||||
"steps": 20,
|
||||
"true_cfg_scale": 1.0,
|
||||
"size": "1344x768",
|
||||
"wall_s": 29.44,
|
||||
"http": 200,
|
||||
"bytes": 4130915,
|
||||
"sha256": "67ed3ba31c5db970ddda42dc38e5f75e395d83aae1c467625060a5bc1034906d",
|
||||
"structure": {
|
||||
"opaque_fraction": 0.9882,
|
||||
"alpha_mean": 255.0,
|
||||
"rgb_std": [
|
||||
37.9,
|
||||
30.5,
|
||||
21.6
|
||||
]
|
||||
},
|
||||
"usage": null,
|
||||
"metrics": {
|
||||
"stage_metrics": {
|
||||
"0": {
|
||||
"stage_id": 0,
|
||||
"final_output_type": "image",
|
||||
"finish_reason": null,
|
||||
"num_tokens_in": 0,
|
||||
"num_tokens_out": 0,
|
||||
"stage_gen_time_ms": 29367.507219314575,
|
||||
"postprocess_time_ms": 0.0,
|
||||
"audio_frames": 0,
|
||||
"audio_sample_rate": 0,
|
||||
"audio_duration_s": 0.0,
|
||||
"image_pixels": 1032192,
|
||||
"denoise_step_latency_ms": 1468.3753609657288,
|
||||
"output_unit_type": "image",
|
||||
"output_unit_count": 1,
|
||||
"serving_time_to_first_output_ms": 29367.653369903564,
|
||||
"image_time_to_first_output_ms": 29367.475748062134,
|
||||
"time_per_output_unit_ms": 0.0,
|
||||
"inter_output_latency_ms": 0.0,
|
||||
"inter_output_latencies_ms": [],
|
||||
"vllm_ttft_ms": 0.0,
|
||||
"vllm_tpot_ms": 0.0,
|
||||
"vllm_itl_ms": 0.0,
|
||||
"vllm_itls_ms": [],
|
||||
"stage_name": "diffusion"
|
||||
}
|
||||
},
|
||||
"stage_id": 0,
|
||||
"final_output_type": "image",
|
||||
"num_tokens_in": 0,
|
||||
"num_tokens_out": 0,
|
||||
"stage_durations": {
|
||||
"queue_wait_ms": 0.15194500156212598,
|
||||
"stage_0_gen_ms": 29367.507219314575
|
||||
},
|
||||
"peak_memory_mb": 23432.0
|
||||
}
|
||||
},
|
||||
{
|
||||
"fixture_id": "edit-compose-three",
|
||||
"seed": 2303,
|
||||
"n_references": 3,
|
||||
"steps": 20,
|
||||
"true_cfg_scale": 1.0,
|
||||
"size": "1344x768",
|
||||
"wall_s": 29.47,
|
||||
"http": 200,
|
||||
"bytes": 4130915,
|
||||
"sha256": "ed8493e310da3d892eeba849519fe8697e638c215eb01bccad96608b9040e70b",
|
||||
"structure": {
|
||||
"opaque_fraction": 0.9757,
|
||||
"alpha_mean": 255.0,
|
||||
"rgb_std": [
|
||||
48.5,
|
||||
40.2,
|
||||
30.1
|
||||
]
|
||||
},
|
||||
"usage": null,
|
||||
"metrics": {
|
||||
"stage_metrics": {
|
||||
"0": {
|
||||
"stage_id": 0,
|
||||
"final_output_type": "image",
|
||||
"finish_reason": null,
|
||||
"num_tokens_in": 0,
|
||||
"num_tokens_out": 0,
|
||||
"stage_gen_time_ms": 29401.696920394897,
|
||||
"postprocess_time_ms": 0.0,
|
||||
"audio_frames": 0,
|
||||
"audio_sample_rate": 0,
|
||||
"audio_duration_s": 0.0,
|
||||
"image_pixels": 1032192,
|
||||
"denoise_step_latency_ms": 1470.0848460197449,
|
||||
"output_unit_type": "image",
|
||||
"output_unit_count": 1,
|
||||
"serving_time_to_first_output_ms": 29401.797771453857,
|
||||
"image_time_to_first_output_ms": 29401.665925979614,
|
||||
"time_per_output_unit_ms": 0.0,
|
||||
"inter_output_latency_ms": 0.0,
|
||||
"inter_output_latencies_ms": [],
|
||||
"vllm_ttft_ms": 0.0,
|
||||
"vllm_tpot_ms": 0.0,
|
||||
"vllm_itl_ms": 0.0,
|
||||
"vllm_itls_ms": [],
|
||||
"stage_name": "diffusion"
|
||||
}
|
||||
},
|
||||
"stage_id": 0,
|
||||
"final_output_type": "image",
|
||||
"num_tokens_in": 0,
|
||||
"num_tokens_out": 0,
|
||||
"stage_durations": {
|
||||
"queue_wait_ms": 0.10474099690327421,
|
||||
"stage_0_gen_ms": 29401.696920394897
|
||||
},
|
||||
"peak_memory_mb": 23432.0
|
||||
}
|
||||
},
|
||||
{
|
||||
"fixture_id": "edit-compose-three",
|
||||
"seed": 2304,
|
||||
"n_references": 3,
|
||||
"steps": 20,
|
||||
"true_cfg_scale": 1.0,
|
||||
"size": "1344x768",
|
||||
"wall_s": 29.49,
|
||||
"http": 200,
|
||||
"bytes": 4130915,
|
||||
"sha256": "eca7238f811f5980b514adde1f0727fc6ce6d33efe16bd620cef184831ed1b44",
|
||||
"structure": {
|
||||
"opaque_fraction": 0.9645,
|
||||
"alpha_mean": 255.0,
|
||||
"rgb_std": [
|
||||
48.5,
|
||||
35.7,
|
||||
25.1
|
||||
]
|
||||
},
|
||||
"usage": null,
|
||||
"metrics": {
|
||||
"stage_metrics": {
|
||||
"0": {
|
||||
"stage_id": 0,
|
||||
"final_output_type": "image",
|
||||
"finish_reason": null,
|
||||
"num_tokens_in": 0,
|
||||
"num_tokens_out": 0,
|
||||
"stage_gen_time_ms": 29419.892072677612,
|
||||
"postprocess_time_ms": 0.0,
|
||||
"audio_frames": 0,
|
||||
"audio_sample_rate": 0,
|
||||
"audio_duration_s": 0.0,
|
||||
"image_pixels": 1032192,
|
||||
"denoise_step_latency_ms": 1470.9946036338806,
|
||||
"output_unit_type": "image",
|
||||
"output_unit_count": 1,
|
||||
"serving_time_to_first_output_ms": 29419.984102249146,
|
||||
"image_time_to_first_output_ms": 29419.86107826233,
|
||||
"time_per_output_unit_ms": 0.0,
|
||||
"inter_output_latency_ms": 0.0,
|
||||
"inter_output_latencies_ms": [],
|
||||
"vllm_ttft_ms": 0.0,
|
||||
"vllm_tpot_ms": 0.0,
|
||||
"vllm_itl_ms": 0.0,
|
||||
"vllm_itls_ms": [],
|
||||
"stage_name": "diffusion"
|
||||
}
|
||||
},
|
||||
"stage_id": 0,
|
||||
"final_output_type": "image",
|
||||
"num_tokens_in": 0,
|
||||
"num_tokens_out": 0,
|
||||
"stage_durations": {
|
||||
"queue_wait_ms": 0.09873499948298559,
|
||||
"stage_0_gen_ms": 29419.892072677612
|
||||
},
|
||||
"peak_memory_mb": 23432.0
|
||||
}
|
||||
},
|
||||
{
|
||||
"fixture_id": "edit-compose-four",
|
||||
"seed": 2301,
|
||||
"n_references": 4,
|
||||
"steps": 20,
|
||||
"true_cfg_scale": 1.0,
|
||||
"size": "1344x768",
|
||||
"wall_s": 34.08,
|
||||
"http": 200,
|
||||
"bytes": 4130915,
|
||||
"sha256": "ed2978bbee6c6d667a1e113fddb6e5d7c06d9b0c7c0d8eda77ad330b92bdc45f",
|
||||
"structure": {
|
||||
"opaque_fraction": 0.976,
|
||||
"alpha_mean": 255.0,
|
||||
"rgb_std": [
|
||||
49.0,
|
||||
37.3,
|
||||
29.3
|
||||
]
|
||||
},
|
||||
"usage": null,
|
||||
"metrics": {
|
||||
"stage_metrics": {
|
||||
"0": {
|
||||
"stage_id": 0,
|
||||
"final_output_type": "image",
|
||||
"finish_reason": null,
|
||||
"num_tokens_in": 0,
|
||||
"num_tokens_out": 0,
|
||||
"stage_gen_time_ms": 34010.05530357361,
|
||||
"postprocess_time_ms": 0.0,
|
||||
"audio_frames": 0,
|
||||
"audio_sample_rate": 0,
|
||||
"audio_duration_s": 0.0,
|
||||
"image_pixels": 1032192,
|
||||
"denoise_step_latency_ms": 1700.5027651786804,
|
||||
"output_unit_type": "image",
|
||||
"output_unit_count": 1,
|
||||
"serving_time_to_first_output_ms": 34010.1637840271,
|
||||
"image_time_to_first_output_ms": 34010.02025604248,
|
||||
"time_per_output_unit_ms": 0.0,
|
||||
"inter_output_latency_ms": 0.0,
|
||||
"inter_output_latencies_ms": [],
|
||||
"vllm_ttft_ms": 0.0,
|
||||
"vllm_tpot_ms": 0.0,
|
||||
"vllm_itl_ms": 0.0,
|
||||
"vllm_itls_ms": [],
|
||||
"stage_name": "diffusion"
|
||||
}
|
||||
},
|
||||
"stage_id": 0,
|
||||
"final_output_type": "image",
|
||||
"num_tokens_in": 0,
|
||||
"num_tokens_out": 0,
|
||||
"stage_durations": {
|
||||
"queue_wait_ms": 0.11569900379981846,
|
||||
"stage_0_gen_ms": 34010.05530357361
|
||||
},
|
||||
"peak_memory_mb": 23598.0
|
||||
}
|
||||
},
|
||||
{
|
||||
"fixture_id": "edit-compose-four",
|
||||
"seed": 2302,
|
||||
"n_references": 4,
|
||||
"steps": 20,
|
||||
"true_cfg_scale": 1.0,
|
||||
"size": "1344x768",
|
||||
"wall_s": 34.0,
|
||||
"http": 200,
|
||||
"bytes": 4130915,
|
||||
"sha256": "f4433739a48bdb32c24f0b53d4feee604adad91327d0b25656cb0b28d4051e1b",
|
||||
"structure": {
|
||||
"opaque_fraction": 0.9713,
|
||||
"alpha_mean": 255.0,
|
||||
"rgb_std": [
|
||||
47.2,
|
||||
39.5,
|
||||
29.8
|
||||
]
|
||||
},
|
||||
"usage": null,
|
||||
"metrics": {
|
||||
"stage_metrics": {
|
||||
"0": {
|
||||
"stage_id": 0,
|
||||
"final_output_type": "image",
|
||||
"finish_reason": null,
|
||||
"num_tokens_in": 0,
|
||||
"num_tokens_out": 0,
|
||||
"stage_gen_time_ms": 33924.51333999634,
|
||||
"postprocess_time_ms": 0.0,
|
||||
"audio_frames": 0,
|
||||
"audio_sample_rate": 0,
|
||||
"audio_duration_s": 0.0,
|
||||
"image_pixels": 1032192,
|
||||
"denoise_step_latency_ms": 1696.225666999817,
|
||||
"output_unit_type": "image",
|
||||
"output_unit_count": 1,
|
||||
"serving_time_to_first_output_ms": 33924.62372779846,
|
||||
"image_time_to_first_output_ms": 33924.47805404663,
|
||||
"time_per_output_unit_ms": 0.0,
|
||||
"inter_output_latency_ms": 0.0,
|
||||
"inter_output_latencies_ms": [],
|
||||
"vllm_ttft_ms": 0.0,
|
||||
"vllm_tpot_ms": 0.0,
|
||||
"vllm_itl_ms": 0.0,
|
||||
"vllm_itls_ms": [],
|
||||
"stage_name": "diffusion"
|
||||
}
|
||||
},
|
||||
"stage_id": 0,
|
||||
"final_output_type": "image",
|
||||
"num_tokens_in": 0,
|
||||
"num_tokens_out": 0,
|
||||
"stage_durations": {
|
||||
"queue_wait_ms": 0.11932000052183867,
|
||||
"stage_0_gen_ms": 33924.51333999634
|
||||
},
|
||||
"peak_memory_mb": 23590.0
|
||||
}
|
||||
},
|
||||
{
|
||||
"fixture_id": "edit-compose-four",
|
||||
"seed": 2303,
|
||||
"n_references": 4,
|
||||
"steps": 20,
|
||||
"true_cfg_scale": 1.0,
|
||||
"size": "1344x768",
|
||||
"wall_s": 34.02,
|
||||
"http": 200,
|
||||
"bytes": 4130915,
|
||||
"sha256": "9b673d3e82dc29d8781dc9d2e0a40bdd7dcad746332f9566e80e5bd8a055bd30",
|
||||
"structure": {
|
||||
"opaque_fraction": 0.9706,
|
||||
"alpha_mean": 255.0,
|
||||
"rgb_std": [
|
||||
56.4,
|
||||
44.8,
|
||||
35.4
|
||||
]
|
||||
},
|
||||
"usage": null,
|
||||
"metrics": {
|
||||
"stage_metrics": {
|
||||
"0": {
|
||||
"stage_id": 0,
|
||||
"final_output_type": "image",
|
||||
"finish_reason": null,
|
||||
"num_tokens_in": 0,
|
||||
"num_tokens_out": 0,
|
||||
"stage_gen_time_ms": 33950.94609260559,
|
||||
"postprocess_time_ms": 0.0,
|
||||
"audio_frames": 0,
|
||||
"audio_sample_rate": 0,
|
||||
"audio_duration_s": 0.0,
|
||||
"image_pixels": 1032192,
|
||||
"denoise_step_latency_ms": 1697.5473046302795,
|
||||
"output_unit_type": "image",
|
||||
"output_unit_count": 1,
|
||||
"serving_time_to_first_output_ms": 33951.04670524597,
|
||||
"image_time_to_first_output_ms": 33950.846672058105,
|
||||
"time_per_output_unit_ms": 0.0,
|
||||
"inter_output_latency_ms": 0.0,
|
||||
"inter_output_latencies_ms": [],
|
||||
"vllm_ttft_ms": 0.0,
|
||||
"vllm_tpot_ms": 0.0,
|
||||
"vllm_itl_ms": 0.0,
|
||||
"vllm_itls_ms": [],
|
||||
"stage_name": "diffusion"
|
||||
}
|
||||
},
|
||||
"stage_id": 0,
|
||||
"final_output_type": "image",
|
||||
"num_tokens_in": 0,
|
||||
"num_tokens_out": 0,
|
||||
"stage_durations": {
|
||||
"queue_wait_ms": 0.17462699906900525,
|
||||
"stage_0_gen_ms": 33950.94609260559
|
||||
},
|
||||
"peak_memory_mb": 23590.0
|
||||
}
|
||||
},
|
||||
{
|
||||
"fixture_id": "edit-compose-four",
|
||||
"seed": 2304,
|
||||
"n_references": 4,
|
||||
"steps": 20,
|
||||
"true_cfg_scale": 1.0,
|
||||
"size": "1344x768",
|
||||
"wall_s": 34.04,
|
||||
"http": 200,
|
||||
"bytes": 4130915,
|
||||
"sha256": "22d9558389165abb8e58c02e7f1c4d49d4f312ff178062246c9907d519336ad6",
|
||||
"structure": {
|
||||
"opaque_fraction": 0.9674,
|
||||
"alpha_mean": 255.0,
|
||||
"rgb_std": [
|
||||
50.9,
|
||||
38.3,
|
||||
29.5
|
||||
]
|
||||
},
|
||||
"usage": null,
|
||||
"metrics": {
|
||||
"stage_metrics": {
|
||||
"0": {
|
||||
"stage_id": 0,
|
||||
"final_output_type": "image",
|
||||
"finish_reason": null,
|
||||
"num_tokens_in": 0,
|
||||
"num_tokens_out": 0,
|
||||
"stage_gen_time_ms": 33973.530292510986,
|
||||
"postprocess_time_ms": 0.0,
|
||||
"audio_frames": 0,
|
||||
"audio_sample_rate": 0,
|
||||
"audio_duration_s": 0.0,
|
||||
"image_pixels": 1032192,
|
||||
"denoise_step_latency_ms": 1698.6765146255493,
|
||||
"output_unit_type": "image",
|
||||
"output_unit_count": 1,
|
||||
"serving_time_to_first_output_ms": 33973.62184524536,
|
||||
"image_time_to_first_output_ms": 33973.4947681427,
|
||||
"time_per_output_unit_ms": 0.0,
|
||||
"inter_output_latency_ms": 0.0,
|
||||
"inter_output_latencies_ms": [],
|
||||
"vllm_ttft_ms": 0.0,
|
||||
"vllm_tpot_ms": 0.0,
|
||||
"vllm_itl_ms": 0.0,
|
||||
"vllm_itls_ms": [],
|
||||
"stage_name": "diffusion"
|
||||
}
|
||||
},
|
||||
"stage_id": 0,
|
||||
"final_output_type": "image",
|
||||
"num_tokens_in": 0,
|
||||
"num_tokens_out": 0,
|
||||
"stage_durations": {
|
||||
"queue_wait_ms": 0.09957399743143469,
|
||||
"stage_0_gen_ms": 33973.530292510986
|
||||
},
|
||||
"peak_memory_mb": 23590.0
|
||||
}
|
||||
}
|
||||
]
|
||||
@@ -1,6 +1,26 @@
|
||||
{
|
||||
"schema": "issue5-runs-summary-v1",
|
||||
"schema": "issue5-runs-summary-v2",
|
||||
"probed_utc": "2026-10-08",
|
||||
"round_two": {
|
||||
"directive": "issue 5 comment 2026-10-08T07:51 (amended 08:37): one decisive experiment - W8A8 int8-dynamic, non-Marlin",
|
||||
"artifact": "~/.local/state/multi-model-helper-server/state/image-service/artifacts/qwen-image-2.1-dit-w8a8dyn",
|
||||
"kernel": "CutlassInt8ScaledMMLinearKernel (ops.scaled_int8_quant + ops.cutlass_scaled_mm)",
|
||||
"kernel_verification": {
|
||||
"raw_ops_rel_l2": "0.0017 at (64,256,128)/(512,4096,4096)/(512,4096,12288) vs float32 reference (cutlass-ops-probe.json in evidence/rtx3090-round2)",
|
||||
"layer_path_rel_l2": "0.0017 for fused QKV and row-parallel, synthetic weights, loaded bitwise-verified (w8a8-layer-isolate.json)"
|
||||
},
|
||||
"serve_run": {
|
||||
"label": "int8w8a8dyn",
|
||||
"init": "healthy in ~40 s",
|
||||
"wall_s_mean": {"1-ref": 20.5, "2-ref": 25.1, "3-ref": 29.4, "4-ref": 34.0},
|
||||
"nvidia_smi_peak_mib": 23879,
|
||||
"structure": "opaque 0.947-0.988, alpha mean 254.9-255.0, rgb_std 21-72 across all 16 requests; visual checks render faithful scenes",
|
||||
"gate_note": "round-one 0.99-opacity rule was a broken-vs-BF16 calibration; W8A8 outputs are soft-edge alpha, not the broken signature (0.008-0.20 band); verdict rests on raw numbers + visual + PSNR",
|
||||
"psnr_vs_sealed_diffusers_mean_db": {"1-ref": 33.72, "2-ref": 28.34, "4-ref": 27.50},
|
||||
"bf16_reference_pair_db": 27.57
|
||||
},
|
||||
"verdict": "YES - W8A8 int8-dynamic delivers 3-4 reference capacity at the fastest measured speed with BF16-parity quality; hard stop did not trigger; Marlin W8A16 root-cause draft staged for the #3 lane (UPSTREAM-REPORT-DRAFT.md)"
|
||||
},
|
||||
"host": "cachyos-alexander (RTX 3090 24 GB, sm_86, driver 615.71.09, CUDA 13.4)",
|
||||
"stack": {
|
||||
"vllm_omni": "ISSUE-5 branch e778d21f..67e17786 over main 8484d85d1",
|
||||
|
||||
Reference in new issue
Block a user