Verify public Qwen3.8 FP8 requests and abort recovery

This commit is contained in:
tzhouam committed 2026-09-24 17:26:14 +08:00
1 parent 32a81111bd
commit bf42971416
16 files changed
+1849 -7

No files matched your search

+1 -1
View File
@@ -331,7 +331,7 @@ The [native Windows RTX Spark GGUF follow-up](../../benchmarks/edge_harness/resu
A [native Windows Qwen3.8-27B NVFP4 attempt](../../benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen38_omni_windows_rtx/README.md) did not reach inference despite loading all three weight shards. The installed Windows vLLM 0.29+cu134 wheel failed during Marlin post-load scale permutation with an unsupported PTX toolchain error under RTX driver 610.71/CUDA UMD 13.3. NVIDIA's PTX compatibility guidance makes a driver/toolchain mismatch likely; an upgraded-driver or compatible-wheel control is still needed before rerunning the 512-token text/image cases. This failure constrains the tested Windows route, not the previously passed WSL checkpoint path.
The separate official Qwen3.8-27B FP8 checkpoint now has a [scoped native-Windows Omni RTX text/image pass and 20-request serial profile per modality](../../benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen38_omni_windows_rtx/README.md). The installed Windows wheel lacks an SM120 CUTLASS block-FP8 operator, so an explicit Omni-owned Triton linear backend was added and verified. The 0.75 GPU-memory cap correctly refused the 512-token KV requirement; 0.80 with 12 GiB declared host offload admitted the run. This is a different artifact/backend from NVFP4 and covers only short text plus a synthetic red image. The next gates remain public request, cancellation/recovery, loading-peak admission, longer multimodal and quality cases, plus sustained power/thermal behavior.
The separate official Qwen3.8-27B FP8 checkpoint now has a [scoped native-Windows Omni RTX text/image pass and 20-request serial profile per modality](../../benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen38_omni_windows_rtx/README.md). The installed Windows wheel lacks an SM120 CUTLASS block-FP8 operator, so an explicit Omni-owned Triton linear backend was added and verified. The 0.75 GPU-memory cap correctly refused the 512-token KV requirement; 0.80 with 12 GiB declared host offload admitted the run. A public AsyncOmni text/image check and same-engine abort/recovery also passed: the canceled long request emitted its terminal abort marker, cleared request state and was followed by a correct fresh request. This is a different artifact/backend from NVFP4 and covers only short text plus a synthetic red image. The next gates remain a server-started cancellation boundary, separate-stage restart, loading-peak admission, longer multimodal and quality cases, plus sustained power/thermal behavior.
The [native Windows RTX Qwen3-TTS Omni hybrid follow-up](../../benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen_tts_windows_rtx/README.md) extends the existing whole-session CrispASR stage to the optional discrete-GPU route. The RTX Vulkan0 talker/codec and CPU FP32 predictor produced two named complete WAVs, 20 identical serial measured outputs at 3.153/3.231 s p50/p95 for 2.72 s audio, and public AsyncOmni audio. Pinned Whisper tiny.en transcribed the first sentence exactly; the RTX PCM differs from the Radeon PCM, so the timings are not a same-output acceleration comparison. Separate 8 GiB host-RAM and 4 GiB GPU-VRAM reservations, a prelaunch GPU-budget refusal, in-flight cancellation and fresh-stage restart passed; the original Radeon route passed a regression probe. M2 still needs playable chunks, RTF below 1 for the declared workload, broader speaker quality, measured VRAM/load peaks, concurrency and an underrun-free sustained run. This Vulkan+CPU route does not qualify mobile NPU+GPU co-residency.