Profile Spark BF16 on fork branch and audit S25 short vocoder

This commit is contained in:
tzhouam committed 2026-09-24 13:37:57 +08:00
1 parent 8978302a1b
commit 68e0fd7cad
51 files changed
+38444 -6

No files matched your search

+18
View File
@@ -13,6 +13,24 @@ three-stage text-to-speech functional pass after correcting request embeddings,
and an InternVLA real-weight synthetic action forward. Model-quality and device gates
remain open; the 2026-09-22 audit remains the historical baseline.
The [2026-09-24 Spark BF16 WSL CPU evidence](../../benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/spark_bf16_wsl_cpu/README.md)
adds a complete pinned dense checkpoint to the fork-branch M0 text route. Twelve
128-token requests passed and exactly matched standalone vLLM greedy tokens;
one warmup plus 20 serial 64-token requests measured complete-request
p50/p95 **5.411/5.751 s** and TTFT **0.129/0.146 s** on HX370 WSL CPU.
This closes only the stated BF16 text workload, not M1 mobile generation.
For M2, a [two-frame S25 vocoder export](../../benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen_tts_vocoder_qualcomm/s25/qnn_short_chunk/README.md)
matched local eager/ONNX CPU on one fixed window, but its FP16 QNN DLC failed
NPU-requested inference at the hosted device-memory limit. The earlier
25-frame FP16 artifact failed the same memory gate. Different precision or
graph layout, and a complete device-local stream, remain separate work.
A [TFLite GPU-requested two-frame S25 component](../../benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen_tts_vocoder_qualcomm/s25/tflite_short_chunk/README.md)
returned finite audio at 2.325% relative L2 against local ONNX CPU on one
fixture. Its 100-sample component profile measured p50/p95 0.559/1.052 s
for 0.16 s audio, with 403 CPU and 335 GPU node rows and median component
RTF 3.49. It still needs listening quality and playable complete streaming
before M2 can advance.
The [Spark hosted Qualcomm component expansion](../../benchmarks/edge_harness/results/e2e_expansion_20260923/evidence/spark_s24_full_attention/README.md) advances M1 only to one real fixed-shape decoder attention layer. The same W8A16 QNN DLC ran on S24, Snapdragon X Elite CRD and SA8775P ADP NPUs with identical outputs on one pinned fixture; S25 had prior separate component evidence. On [RB3 Gen 2](../../benchmarks/edge_harness/results/e2e_expansion_20260923/evidence/spark_rb3_full_attention/README.md), the tested W8A16 graph failed QNN loading even after an exact-device compile. FP32 ONNX ran on CPU with near-source parity, while calibrated W8A8 ran on NPU but had 20.9% hidden-state relative L2 and remains numerically unqualified. These are component C results, not full M1: device-local 28-layer prefill/continuous decode, KV/ring, sampling, token quality, admission/cancellation, complete-request timing and sustained power/thermal gates remain open. AI Hub is the test facility, not a deployment dependency.
The [Qwen3-TTS Code2Wav Qualcomm follow-up](../../benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen_tts_vocoder_qualcomm/README.md) adds a retained same-fixture numerical check to M2. A fresh SA8775P GPU-requested output saturated 96.63% of samples and differs from original ONNX CPU by 11.832 relative L2, so that route fails the numerical gate despite profiling. The same binary's CPU-requested output on the exact SA device matched original ONNX CPU within 4.13e-6 relative L2; its 100-sample CPU-requested component profile passed placement (738/738 execution-detail rows on CPU) at nearest-rank p50/p95 5.384/5.534 s, still without a full stream. A FP32-preserving GPU-option control still saturated 84.89% of samples, so that option did not fix the tested route. RB3 TFLite GPU-requested inference returned an unsaturated but 15.73%-relative-L2 divergent waveform; its fresh 100-sample mixed CPU/GPU component p50/p95 was 3.099/3.133 s. The same binary requested on exact-RB3 CPU matched original ONNX CPU within 4.13e-6 relative L2 on the retained fixture; its 47-sample component profile measured p50/p95 12.727/13.424 s with all 738 execution-detail rows on CPU. That measured component latency is a major blocker for playable streaming on this route. A one-fixture-calibrated RB3 INT8 QNN vocoder returned a finite but 66.93%-relative-L2 divergent waveform (3.49 dB SNR) versus ONNX CPU, with 100 NPU component samples at p50/p95 1.971/1.992 s and all 728 execution-detail rows on NPU; it fails the numerical gate even on its calibration fixture. X Elite ONNX CPU inference matched the local ONNX Runtime output with 8.09e-7 relative L2; its 94-sample component profile was p50/p95 6.384/6.644 s with all execution-detail rows on CPU. Corrected DirectML inference and profile both failed during provider initialization; separate FP16 QNN DLC inference returned an unsaturated waveform at 1.138% relative L2 / 38.88 dB SNR versus local ONNX CPU, but its placement profile timed out during QNN graph preparation without samples. A separate SA8775P FP16 QNN DLC vocoder compile passed, but NPU-requested same-fixture inference failed after compilation without waveform or placement samples. Fresh S24 and S25 CPU-requested TFLite vocoder inferences matched ONNX CPU within 4.04e-6 relative L2 on the retained fixture. S25 measured 100 component samples at p50/p95 4.418/4.641 s with all 738 execution-detail rows on CPU; S24 measured 100 at 4.826/4.958 s with all 738 rows on CPU. Separate explicit INT8 and W8A16 QDQ ONNX exports checked on local ORT CPU failed the same-fixture numerical gate at 246.97% and 331.83% waveform relative L2, respectively. S25 CPU talker inference preserved token 80 and its 100-sample component profile measured p50/p95 30.161/31.301 ms with all 1,800 rows on CPU. M2 still requires a quality-passing talker/predictor/vocoder set, device-local co-residency and handoff, playable streaming and sustained memory/power validation.