Record native Windows Qwen3.8 NVFP4 PTX startup blocker

This commit is contained in:
tzhouam committed 2026-09-24 15:52:59 +08:00
1 parent 605b68cc5f
commit 74d284acba
11 files changed
+714 -2

No files matched your search

+2
View File
@@ -329,6 +329,8 @@ The 2026-09-23 [Spark GGUF Omni follow-up](../../benchmarks/edge_harness/results
The [native Windows RTX Spark GGUF follow-up](../../benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/spark_omni_windows_rtx/README.md) runs that same 1.7B Q4_K_M artifact through an Omni whole-session stage on RTX Vulkan0, with 29/29 layers offloaded. Two named requests, 20/20 serial inventory requests, a public `AsyncOmni` request, context refusal and fresh-stage restart after confirmed in-flight cancellation passed. Its 0.294/0.302 s complete-request p50/p95 is specific to the 2,463-prompt-token/five-output-token serial workload and cannot be compared directly to the earlier Spark 4B BF16 vLLM CUDA results. Token streaming, broad quality, VRAM/load peaks, concurrency and sustained power remain open; this does not establish mobile M1 support.
A [native Windows Qwen3.8-27B NVFP4 attempt](../../benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen38_omni_windows_rtx/README.md) did not reach inference despite loading all three weight shards. The installed Windows vLLM 0.29+cu134 wheel failed during Marlin post-load scale permutation with an unsupported PTX toolchain error under RTX driver 610.71/CUDA UMD 13.3. NVIDIA's PTX compatibility guidance makes a driver/toolchain mismatch likely; an upgraded-driver or compatible-wheel control is still needed before rerunning the 512-token text/image cases. This failure constrains the tested Windows route, not the previously passed WSL checkpoint path.
The [Qwen3-TTS Omni hybrid follow-up](../../benchmarks/edge_harness/results/e2e_expansion_20260923/evidence/qwen_tts_omni_hybrid/README.md) adds one complete-request stage for the native Windows HX370 CPU+Radeon 890M route. A pinned Q8_0 talker and F16 codec produced two named PCM outputs exactly matching the prior standalone backend; one warmup plus 20 serial requests measured 3.298/3.336 s p50/p95 for 2.56 s of audio. Public `AsyncOmni` audio, cancellation drain and explicit 8 GiB host admission passed. A separate [in-flight cancellation audit](../../benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen_tts_omni_windows_restart/README.md) confirmed that the request reached server prefill before cancellation; a fresh stage with the same artifacts and hybrid placement emitted the identical PCM under a new worker generation, without stale output or a retained reservation. This is progress toward M2 integration, not M2 acceptance: RTF remains above 1, and playable chunk streaming, same-session continuation, loading/iGPU peaks, broader voice quality and 30-minute thermal behavior remain open. No NPU participated.
The [WSL CPU Qwen3-TTS Omni follow-up](../../benchmarks/edge_harness/results/e2e_expansion_20260923/evidence/qwen_tts_omni_wsl_cpu/README.md)