Qualify scoped Windows RTX Qwen3.8 FP8 text and image path
This commit is contained in:
1 parent
5bb16af8ee
commit
32a81111bd
23 files changed
+3368
-8
No files matched your search
@@ -331,6 +331,8 @@ The [native Windows RTX Spark GGUF follow-up](../../benchmarks/edge_harness/resu
|
||||
|
||||
A [native Windows Qwen3.8-27B NVFP4 attempt](../../benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen38_omni_windows_rtx/README.md) did not reach inference despite loading all three weight shards. The installed Windows vLLM 0.29+cu134 wheel failed during Marlin post-load scale permutation with an unsupported PTX toolchain error under RTX driver 610.71/CUDA UMD 13.3. NVIDIA's PTX compatibility guidance makes a driver/toolchain mismatch likely; an upgraded-driver or compatible-wheel control is still needed before rerunning the 512-token text/image cases. This failure constrains the tested Windows route, not the previously passed WSL checkpoint path.
|
||||
|
||||
The separate official Qwen3.8-27B FP8 checkpoint now has a [scoped native-Windows Omni RTX text/image pass and 20-request serial profile per modality](../../benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen38_omni_windows_rtx/README.md). The installed Windows wheel lacks an SM120 CUTLASS block-FP8 operator, so an explicit Omni-owned Triton linear backend was added and verified. The 0.75 GPU-memory cap correctly refused the 512-token KV requirement; 0.80 with 12 GiB declared host offload admitted the run. This is a different artifact/backend from NVFP4 and covers only short text plus a synthetic red image. The next gates remain public request, cancellation/recovery, loading-peak admission, longer multimodal and quality cases, plus sustained power/thermal behavior.
|
||||
|
||||
The [native Windows RTX Qwen3-TTS Omni hybrid follow-up](../../benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen_tts_windows_rtx/README.md) extends the existing whole-session CrispASR stage to the optional discrete-GPU route. The RTX Vulkan0 talker/codec and CPU FP32 predictor produced two named complete WAVs, 20 identical serial measured outputs at 3.153/3.231 s p50/p95 for 2.72 s audio, and public AsyncOmni audio. Pinned Whisper tiny.en transcribed the first sentence exactly; the RTX PCM differs from the Radeon PCM, so the timings are not a same-output acceleration comparison. Separate 8 GiB host-RAM and 4 GiB GPU-VRAM reservations, a prelaunch GPU-budget refusal, in-flight cancellation and fresh-stage restart passed; the original Radeon route passed a regression probe. M2 still needs playable chunks, RTF below 1 for the declared workload, broader speaker quality, measured VRAM/load peaks, concurrency and an underrun-free sustained run. This Vulkan+CPU route does not qualify mobile NPU+GPU co-residency.
|
||||
|
||||
The [Qwen3-TTS Omni hybrid follow-up](../../benchmarks/edge_harness/results/e2e_expansion_20260923/evidence/qwen_tts_omni_hybrid/README.md) adds one complete-request stage for the native Windows HX370 CPU+Radeon 890M route. A pinned Q8_0 talker and F16 codec produced two named PCM outputs exactly matching the prior standalone backend; one warmup plus 20 serial requests measured 3.298/3.336 s p50/p95 for 2.56 s of audio. Public `AsyncOmni` audio, cancellation drain and explicit 8 GiB host admission passed. A separate [in-flight cancellation audit](../../benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen_tts_omni_windows_restart/README.md) confirmed that the request reached server prefill before cancellation; a fresh stage with the same artifacts and hybrid placement emitted the identical PCM under a new worker generation, without stale output or a retained reservation. This is progress toward M2 integration, not M2 acceptance: RTF remains above 1, and playable chunk streaming, same-session continuation, loading/iGPU peaks, broader voice quality and 30-minute thermal behavior remain open. No NPU participated.
|
||||
|
||||
Reference in new issue
Block a user