admin
67eb7fdb48
Evidence-only change: probe kit (serve wrapper, API client, PSNR
comparator, offline fp8_v driver), per-host run summaries, and the
findings report. No code changes to the stack.
Verdicts: not-built on both cards. On the 3090 the served reference
ceiling is two condition images with the conditioning cache on
(2.2x the Diffusers nocache baseline, pair PSNR mean 27.57 dB);
four references OOM on the resident BF16 working set, not the KV
(~0.3 GiB/reference, per-request). On the 5080 the server cannot
initialize under any flag: init_parameters materializes the
unquantized skeleton on-device before quantization/offload engage.
Also documents that the recipe's deploy-YAML extras route for
prefix_kv_cache_dtype does not reach the served transformer on
8484d85d (offline Omni engine kwarg does).
30 lines
1.1 KiB
Bash
30 lines
1.1 KiB
Bash
#!/usr/bin/env bash
|
|
# Issue 2 probe: serve Qwen-Image-2.1 on the RTX 3090 with one labeled
|
|
# flag set. VRAM is polled once per second for the whole server lifetime,
|
|
# matching issue 1's instrument (nvidia-smi csv series).
|
|
#
|
|
# Usage: serve.sh LABEL [extra vllm flags...]
|
|
set -uo pipefail
|
|
|
|
BASE=$HOME/Projects/vllm-omni-val
|
|
W=$BASE/issue2
|
|
LABEL=$1; shift
|
|
RUN=$W/runs/$LABEL
|
|
mkdir -p "$RUN"
|
|
|
|
SNAP=$HOME/.local/state/multi-model-helper-server/state/image-service/weights/hub/models--Qwen--Qwen-Image-2.1/snapshots/790c92633540aa0cb11d9abf19eb46d861714758
|
|
export HF_HOME=$HOME/.local/state/multi-model-helper-server/state/image-service/weights
|
|
export HF_HUB_OFFLINE=1
|
|
|
|
( while sleep 1; do nvidia-smi --query-gpu=memory.used,utilization.gpu --format=csv,noheader; done ) > "$RUN/vram.log" 2>&1 &
|
|
POLL=$!
|
|
T0=$(date +%s)
|
|
echo "flags: $*" > "$RUN/flags.txt"
|
|
|
|
$BASE/venv/bin/vllm serve "$SNAP" --omni --port 8091 \
|
|
--served-model-name Qwen/Qwen-Image-2.1 \
|
|
--enforce-eager "$@" > "$RUN/serve.log" 2>&1
|
|
RC=$?
|
|
kill $POLL 2>/dev/null
|
|
echo "serve_rc=$RC wall=$(( $(date +%s) - T0 ))s" >> "$RUN/serve.log"
|