admin
67eb7fdb48
Evidence-only change: probe kit (serve wrapper, API client, PSNR
comparator, offline fp8_v driver), per-host run summaries, and the
findings report. No code changes to the stack.
Verdicts: not-built on both cards. On the 3090 the served reference
ceiling is two condition images with the conditioning cache on
(2.2x the Diffusers nocache baseline, pair PSNR mean 27.57 dB);
four references OOM on the resident BF16 working set, not the KV
(~0.3 GiB/reference, per-request). On the 5080 the server cannot
initialize under any flag: init_parameters materializes the
unquantized skeleton on-device before quantization/offload engage.
Also documents that the recipe's deploy-YAML extras route for
prefix_kv_cache_dtype does not reach the served transformer on
8484d85d (offline Omni engine kwarg does).
41 lines
831 B
JSON
41 lines
831 B
JSON
{
|
|
"runs": {
|
|
"cpuoff": {
|
|
"flags": "--enable-cpu-offload",
|
|
"serve_rc": 1,
|
|
"serve_wall_s": 9,
|
|
"vram_peak_mib": 15856
|
|
},
|
|
"cpuoff-layerwise": {
|
|
"flags": "--enable-cpu-offload --enable-layerwise-offload",
|
|
"serve_rc": 1,
|
|
"serve_wall_s": 10,
|
|
"vram_peak_mib": 15856
|
|
},
|
|
"default": {
|
|
"flags": "flags:",
|
|
"serve_rc": 1,
|
|
"serve_wall_s": 12,
|
|
"vram_peak_mib": 15926
|
|
},
|
|
"fp8": {
|
|
"flags": "--enable-cpu-offload --quantization fp8",
|
|
"serve_rc": 1,
|
|
"serve_wall_s": 13,
|
|
"vram_peak_mib": 15930
|
|
},
|
|
"fp8-expandable": {
|
|
"flags": "--enable-cpu-offload --quantization fp8",
|
|
"serve_rc": 1,
|
|
"serve_wall_s": 9,
|
|
"vram_peak_mib": 15876
|
|
},
|
|
"int8": {
|
|
"flags": "--enable-cpu-offload --quantization int8",
|
|
"serve_rc": 1,
|
|
"serve_wall_s": 8,
|
|
"vram_peak_mib": 15930
|
|
}
|
|
}
|
|
}
|