Record SA8775P CPU vocoder placement and timing
This commit is contained in:
tzhouam committed
1 parent
f04fdf79da
commit
4d6124f320
6 files changed
+5452
-6
No files matched your search
@@ -1,6 +1,6 @@
|
||||
# 2026-09-24 Qualcomm component expansion
|
||||
|
||||
The [Qwen3-TTS vocoder study](evidence/qwen_tts_vocoder_qualcomm/README.md) reuses a retained fixed-shape Code2Wav export and historical Workbench input to test numerical behavior on SA8775P and other Qualcomm settings. The fresh SA8775P GPU-requested inference produces a saturated waveform, while the same binary's CPU-requested output on the exact SA device matches ONNX CPU within 4.13e-6 relative L2. A FP32-preserving GPU-option control still saturated 84.89% of samples, so that option did not repair the tested path. RB3 produces an unsaturated but 15.73%-divergent waveform; its fresh 100-sample mixed CPU/GPU component p50/p95 was 3.099/3.133 s, not a usable TTS timing claim. X Elite ONNX CPU output matches the local ONNX Runtime reference on the retained fixture; its 94-sample component p50/p95 was 6.384/6.644 s with all execution-detail rows on CPU. Corrected X Elite DirectML inference and profile both failed during provider initialization; separate FP16 QNN DLC compilation passed, with NPU-requested inference pending. SA CPU placement profiling remains pending. No complete on-device TTS stream or model-quality qualification is claimed.
|
||||
The [Qwen3-TTS vocoder study](evidence/qwen_tts_vocoder_qualcomm/README.md) reuses a retained fixed-shape Code2Wav export and historical Workbench input to test numerical behavior on SA8775P and other Qualcomm settings. The fresh SA8775P GPU-requested inference produces a saturated waveform, while the same binary's CPU-requested output on the exact SA device matches ONNX CPU within 4.13e-6 relative L2. Its 100-sample CPU-requested component profile had p50/p95 5.384/5.534 s with all 738 execution-detail rows on CPU. A FP32-preserving GPU-option control still saturated 84.89% of samples, so that option did not repair the tested path. RB3 produces an unsaturated but 15.73%-divergent waveform; its fresh 100-sample mixed CPU/GPU component p50/p95 was 3.099/3.133 s, not a usable TTS timing claim. X Elite ONNX CPU output matches the local ONNX Runtime reference on the retained fixture; its 94-sample component p50/p95 was 6.384/6.644 s with all execution-detail rows on CPU. Corrected X Elite DirectML inference and profile both failed during provider initialization; separate FP16 QNN DLC compilation passed, with NPU-requested inference pending. SA CPU placement profiling remains pending. No complete on-device TTS stream or model-quality qualification is claimed.
|
||||
|
||||
A local HX370 x86-64 LiteRT CPU control of the identical SA/RB3 TFLite binary matched the original ONNX CPU waveform within 3.45e-6 relative L2. The SA saturation is therefore not universal to the converted artifact; hosted CPU and GPU precision controls are still needed to isolate its device/runtime cause.
|
||||
|
||||
|
||||
+2
-2
@@ -10,12 +10,12 @@ The fresh [RB3 inference](rb3/inference_report.json) on the same fixture returne
|
||||
|
||||
A separate [local LiteRT 2.2.0 CPU control](local_litert_cpu/report.json) ran that **same pinned TFLite binary** on the WSL x86-64 HX370 host with four interpreter threads. Its [waveform](local_litert_cpu/device_output.npz) was finite and unsaturated, within **3.45e-6 relative L2 / 109.23 dB SNR** of the original ONNX Runtime CPU output. The [reproducer](../../../../experiments/probe_qwen_tts_vocoder_litert_cpu.py) checks binary/fixture/reference hashes and I/O contracts. This narrows the SA saturation to hosted runtime/device behavior rather than a universally bad TFLite conversion. The one local invoke is not a comparable performance profile; it does not prove either embedded device's CPU route or identify the specific delegate operation at fault.
|
||||
|
||||
On the **exact hosted SA8775P**, a [CPU-requested inference](sa8775p/cpu_control/inference_report.json) of that binary also returned a finite unsaturated waveform. The [same-fixture comparison](sa8775p/cpu_control/inference_comparison.json) was **4.13e-6 relative L2 / 107.67 dB SNR** against original ONNX CPU. The GPU-requested SA output differed from this SA CPU-requested output by **11.832 relative L2** and saturated 96.63% of samples. A fresh [FP32-preserving GPU-option inference](sa8775p/gpu_fp32/inference_report.json) also failed numerically: its [comparison](sa8775p/gpu_fp32/inference_comparison.json) had **11.389 relative L2 / −21.13 dB SNR** versus original ONNX CPU and **84.89%** saturated samples. Disabling the FP32-to-FP16 allowance does not rescue this tested GPU route. The exact faulty delegate operation and all CPU placement details still require further isolation and the [CPU profile](sa8775p/cpu_control/profile_submission.json).
|
||||
On the **exact hosted SA8775P**, a [CPU-requested inference](sa8775p/cpu_control/inference_report.json) of that binary also returned a finite unsaturated waveform. The [same-fixture comparison](sa8775p/cpu_control/inference_comparison.json) was **4.13e-6 relative L2 / 107.67 dB SNR** against original ONNX CPU. Its [CPU-requested profile](sa8775p/cpu_control/profile_report.json) attributed all **738 execution-detail rows to CPU**; 100 component samples had nearest-rank p50/p95 **5.384/5.534 s** (minimum 5.250 s). The [full audit](sa8775p/cpu_control/audit_report.json) ties the numerical and placement results to the same binary and fixture. This is a working **component CPU route**, roughly three times slower than the numerically failed GPU-requested component profile; neither is a complete TTS request. The GPU-requested SA output differed from this SA CPU-requested output by **11.832 relative L2** and saturated 96.63% of samples. A fresh [FP32-preserving GPU-option inference](sa8775p/gpu_fp32/inference_report.json) also failed numerically: its [comparison](sa8775p/gpu_fp32/inference_comparison.json) had **11.389 relative L2 / −21.13 dB SNR** versus original ONNX CPU and **84.89%** saturated samples. Disabling the FP32-to-FP16 allowance does not rescue this tested GPU route. The exact faulty delegate operation still requires isolation.
|
||||
|
||||
The re-fetched *historical* [SA8775P profile](sa8775p/historical_profile_report.json) has 100 component samples, nearest-rank p50/p95 **1.751/1.874 s**, and 403 CPU plus 335 GPU execution-detail rows. The *historical* [RB3 profile](rb3/historical_profile_report.json) has 100 samples, p50/p95 **3.143/3.189 s**, also with 403 CPU plus 335 GPU rows. Their previously quoted 1.701/3.107 s estimated-time fields are sample minima, not medians. Both profiles exclude talker/predictor, stage handoff, streaming and full-request time.
|
||||
|
||||
The X Elite [ONNX CPU inference](xelite/cpu/inference_report.json) succeeded on the pinned fixture. Its [waveform comparison](xelite/cpu/inference_comparison.json) to the original local ONNX Runtime CPU output was **8.09e-7 relative L2 / 121.84 dB SNR**, with no saturation. The fresh [CPU profile](xelite/cpu/profile_report.json) returned **94** component samples, nearest-rank p50/p95 **6.384/6.644 s**, with all **634** execution-detail rows on CPU; the service's 5.917 s estimated-time field is the sample minimum. The [full audit](xelite/cpu/audit_report.json) pins source, fixture, target, waveform, placement and percentiles. This is near-source component parity on one fixture, without a listening-quality or complete-stream gate. The first X Elite [GPU inference](xelite/gpu/inference_report.json) and [profile](xelite/gpu/profile_report.json) jobs failed because Workbench rejects specifying both compute unit and ONNX execution provider. The corrected [DirectML inference](xelite/directml/inference_report.json) and [profile](xelite/directml/profile_report.json) both failed during DML provider initialization with Windows error `80070057` (`The parameter is incorrect`), without an output or timing samples. A separate [FP16 QNN DLC compile](xelite/qnn_npu/compile_report.json) succeeded and [NPU-requested same-fixture inference](xelite/qnn_npu/inference_submission.json) was pending. The [X Elite ONNX CPU compile](xelite/compile_report.json) passed after the older TFLite target was rejected on Windows ARM. These failures constrain the tested DirectML route, not all GPU or NPU artifacts.
|
||||
|
||||
The [SA CPU profile](sa8775p/cpu_control/profile_submission.json) and X Elite [QNN NPU inference](xelite/qnn_npu/inference_submission.json) remained pending in this snapshot. Pending jobs have no numerical or latency conclusion. The [collector](../../../../experiments/collect_aihub_component.py) retrieves their terminal reports without resubmitting jobs, and the [audit script](../../../../experiments/audit_qwen_tts_vocoder_qualcomm.py) checks source/fixture hashes, device and model IDs, actual placement, waveform error, saturation and sample percentiles.
|
||||
The X Elite [QNN NPU inference](xelite/qnn_npu/inference_submission.json) remained pending in this snapshot. Pending jobs have no numerical or latency conclusion. The [collector](../../../../experiments/collect_aihub_component.py) retrieves terminal reports without resubmitting jobs, and the [audit script](../../../../experiments/audit_qwen_tts_vocoder_qualcomm.py) checks source/fixture hashes, device and model IDs, actual placement, waveform error, saturation and sample percentiles.
|
||||
|
||||
This is one fixed vocoder component with a retained input, not complete Qwen3-TTS. The next gates are a quality-passing SA/RB3/X Elite target, talker and code predictor numerical checks, device-resident handoff/co-residency, playable streaming, cancellation, memory admission and sustained power/thermal profiling.
|
||||
+85
@@ -0,0 +1,85 @@
|
||||
{
|
||||
"scope": "one Qwen3-TTS code2wav 48,000-sample component on one retained fixture; no complete TTS stream",
|
||||
"status": "component_executed_quality_unqualified",
|
||||
"device": {
|
||||
"name": "SA8775P ADP",
|
||||
"os": "14",
|
||||
"attributes": [
|
||||
"os:android",
|
||||
"framework:tflite",
|
||||
"framework:onnx",
|
||||
"vendor:qualcomm",
|
||||
"abi:aarch64-android",
|
||||
"format:auto",
|
||||
"chipset:qualcomm-sa8775p",
|
||||
"hexagon:v73",
|
||||
"soc-model:52",
|
||||
"htp-supports-fp16:true",
|
||||
"htp-supports-weight-sharing:true",
|
||||
"framework:qnn"
|
||||
]
|
||||
},
|
||||
"compile_job_id": "jp410o3vp",
|
||||
"inference_job_id": "jp1nj7x7g",
|
||||
"profile_job_id": "jp2rwv7xg",
|
||||
"target_model_id": "mnze95don",
|
||||
"input_dataset_id": "d7m80jl32",
|
||||
"s25_prior_inference_job_id": "jpx428ojg",
|
||||
"requested_run_options": "--compute_unit cpu",
|
||||
"files": {
|
||||
"source_onnx": {
|
||||
"bytes": 423655047,
|
||||
"sha256": "7c07666229c6d404894132e22f784525e3212a49efad55ddd532eda5286c2b2b"
|
||||
},
|
||||
"fixture": {
|
||||
"bytes": 129721,
|
||||
"sha256": "dfca9e8d2a724a568a31b6cd18ff6a52fdc205fd34e13cbf1cd75f0f7937e82d"
|
||||
},
|
||||
"cpu_output": {
|
||||
"bytes": 179942,
|
||||
"sha256": "8de78b94b59f01e49852f7706adbd2508a9976b8e2705cfd3b94d07bb3606cb1"
|
||||
},
|
||||
"s25_output": {
|
||||
"bytes": 115235,
|
||||
"sha256": "4a26486d5e84a6e22d16f9b26f995b8fe645c29bbd70b103615ec35a0fcb10a0"
|
||||
},
|
||||
"device_output": {
|
||||
"bytes": 179497,
|
||||
"sha256": "853feface7a93f7a22a0ffba8d1e297fe5a8780bd64f0444a6ce07b277246d3e"
|
||||
}
|
||||
},
|
||||
"cpu_source_vs_device": {
|
||||
"relative_l2": 4.1337705035866275e-06,
|
||||
"snr_db": 107.67307275926692,
|
||||
"max_abs": 6.854534149169922e-06,
|
||||
"cosine": 0.9999999999914584,
|
||||
"reference_rms": 0.08373439205485074,
|
||||
"output_rms": 0.08373439765610562,
|
||||
"output_saturated_fraction_abs_ge_0_999": 0.0
|
||||
},
|
||||
"s25_gpu_vs_device": {
|
||||
"relative_l2": 0.04236848942339238,
|
||||
"snr_db": 27.459140395118254,
|
||||
"max_abs": 0.10591908544301987,
|
||||
"cosine": 0.9991021766159882,
|
||||
"reference_rms": 0.08376787228276501,
|
||||
"output_rms": 0.08373439765610562,
|
||||
"output_saturated_fraction_abs_ge_0_999": 0.0
|
||||
},
|
||||
"profile": {
|
||||
"sample_count": 100,
|
||||
"sample_min_us": 5250035,
|
||||
"nearest_rank_p50_us": 5384061,
|
||||
"nearest_rank_p95_us": 5534367,
|
||||
"reported_estimated_inference_time_us": 5250035,
|
||||
"reported_peak_memory_bytes": 839372800,
|
||||
"compute_unit_row_counts": {
|
||||
"CPU": 738
|
||||
}
|
||||
},
|
||||
"limits": [
|
||||
"Historical source export does not attest its exact checkpoint revision.",
|
||||
"This fixed component waveform has no listening or task-quality tolerance.",
|
||||
"Hosted component profiling excludes talker, predictor, stage handoff, streaming, admission and sustained power/thermal behavior."
|
||||
]
|
||||
}
|
||||
+5361
File diff suppressed because it is too large.
Load diff
@@ -12,7 +12,7 @@ On a hosted [Galaxy S24 Spark attention-layer experiment](../../../e2e_expansion
|
||||
|
||||
The same W8A16 layer also ran on hosted [Snapdragon X Elite CRD](../../../e2e_expansion_20260923/evidence/spark_xelite_full_attention/README.md) and [SA8775P ADP](../../../e2e_expansion_20260923/evidence/spark_sa8775_full_attention/README.md) NPUs, with pinned-fixture outputs identical to S24/S25; the 100-sample component p50/p95 was 1.433/1.606 ms and 1.837/2.079 ms, respectively. On [RB3 Gen 2 / QCS6490](../../../e2e_expansion_20260923/evidence/spark_rb3_full_attention/README.md), that W8A16 graph failed QNN DLC loading even after target-specific recompilation. An FP32 ONNX CPU layer then passed with source-relative L2 below 9e-7, while a calibrated W8A8 NPU layer executed but had 20.9% hidden-state relative L2 and remains numerically unqualified. These hosted component outcomes do not establish full generation or comparable device speed.
|
||||
|
||||
A [Qwen3-TTS Code2Wav follow-up](../../../e2e_expansion_20260924/evidence/qwen_tts_vocoder_qualcomm/README.md) recovered source and target artifacts, a retained fixture, and raw historical profiles. On SA8775P, fresh GPU-requested inference saturated 96.63% of waveform samples and differed from original ONNX CPU by 11.832 relative L2; the same binary's CPU-requested output on the exact SA device matched ONNX CPU within 4.13e-6 relative L2. The GPU-requested path remains numerically failed while CPU placement profiling is pending. Fresh RB3 inference was unsaturated but differed from CPU by 15.73% relative L2 (16.06 dB SNR), without a listening-quality gate; its 100-sample mixed CPU/GPU component p50/p95 was 3.099/3.133 s. X Elite ONNX CPU vocoder inference passed on the same fixture with 8.09e-7 relative L2 versus local ONNX CPU; its 94-sample component p50/p95 was 6.384/6.644 s with all execution-detail rows on CPU. DirectML retry was pending. None is a complete TTS stream.
|
||||
A [Qwen3-TTS Code2Wav follow-up](../../../e2e_expansion_20260924/evidence/qwen_tts_vocoder_qualcomm/README.md) recovered source and target artifacts, a retained fixture, and raw historical profiles. On SA8775P, fresh GPU-requested inference saturated 96.63% of waveform samples and differed from original ONNX CPU by 11.832 relative L2; the same binary's CPU-requested output on the exact SA device matched ONNX CPU within 4.13e-6 relative L2. The CPU-requested route also has 100 component samples at p50/p95 5.384/5.534 s, with all 738 execution-detail rows on CPU; the GPU-requested path remains numerically failed. Fresh RB3 inference was unsaturated but differed from CPU by 15.73% relative L2 (16.06 dB SNR), without a listening-quality gate; its 100-sample mixed CPU/GPU component p50/p95 was 3.099/3.133 s. X Elite ONNX CPU vocoder inference passed on the same fixture with 8.09e-7 relative L2 versus local ONNX CPU; its 94-sample component p50/p95 was 6.384/6.644 s with all execution-detail rows on CPU. DirectML inference and profile both failed during provider initialization (Windows error 80070057); a separate FP16 QNN NPU vocoder inference is pending. None is a complete TTS stream.
|
||||
|
||||
The identical SA/RB3 TFLite binary matched original ONNX CPU within 3.45e-6 relative L2 in a separate local HX370 x86-64 LiteRT CPU control. This narrows the SA waveform saturation to hosted device/runtime behavior, without proving either embedded CPU route or a specific GPU-delegate fault.
|
||||
|
||||
@@ -102,7 +102,7 @@ NOT E2E means no complete device/model workload was verified. “Scoped E2E” i
|
||||
| Mobile Galaxy S24 / Snapdragon 8 Gen 3 | MiniCPM-o 4.5 | COMPONENT_FP16_SPEECH_HEAD_NUMERIC_MEASURED; NOT_E2E_PROFILEABLE | The retained FP16 speech-head job used a dataset with identical tensors to the recovered S25 fixture. Against original ONNX CPU, logits relative L2 was 0.3459%, maximum new-K/V relative L2 1.235%, and top-1 speech token 1867 matched. Earlier parity reported 49.22 dB SNR/top-1 match; tested W8A16 top-1 failed. Actual NPU placement, generated-audio quality and the rest of the model remain unqualified. | Qualify placement, thinker/encoders/vocoder and stateful local integration; [speech-head evidence](../../../e2e_expansion_20260924/evidence/minicpmo_tts_qualcomm/README.md). |
|
||||
| Mobile Galaxy S24 / Snapdragon 8 Gen 3 | InternVLA-A1 | NOT_E2E_PROFILEABLE | No complete qualified target artifact and device-local stateful Omni backend. Hub jobs provide component execution, not an on-device generation/streaming controller; ADB has no attached device. | Supply and validate the missing matching artifact/backend, then run the complete specified workload. |
|
||||
| Embedded SA8775P ADP / AI Hub | Spark-X2.5 | COMPONENT_NPU_INFERENCE_PROFILED; NOT_E2E_PROFILEABLE | A real Spark-X2.5-1.7B one-layer W8A16 QDQ attention artifact, compiled as QNN DLC, ran on the exact hosted SA8775P ADP (Android 14). Inference returned finite hidden/new-K/new-V tensors bitwise identical to the prior S24/S25 NPU outputs on the same fixture; the profile attributed all 73 execution-detail rows to NPU. Across 100 component samples, nearest-rank p50/p95 was 1.837/2.079 ms; the 1.749 ms estimated-time field is the minimum. Hidden/K/V relative L2 versus original FP32 ONNX CPU was 1.351%/0.377%/0.650%, without a next-token quality tolerance. There is no complete 28-layer prefill/decode/KV-ring/sampling loop, resident Omni controller or complete-request timing. | Qualify a complete embedded resident artifact/controller with state, token quality, memory admission/cancel and sustained power; [SA8775P NPU raw evidence](../../../e2e_expansion_20260923/evidence/spark_sa8775_full_attention/README.md). |
|
||||
| Embedded SA8775P ADP / AI Hub | Qwen3-TTS 0.6B CustomVoice | COMPONENT_CPU_REQUESTED_VOCODER_PARITY_PASS; COMPONENT_GPU_NUMERIC_FAIL; TALKER_FP16_NUMERIC_FAIL; NOT_E2E_PROFILEABLE | On the exact SA8775P, the same TFLite binary's CPU-requested Code2Wav inference matched original ONNX CPU within 4.13e-6 relative L2 / 107.67 dB SNR, without saturation; CPU placement/timing profile was pending. The GPU-requested output saturated 96.63% of samples with 11.832 relative L2; its fresh 100-sample mixed CPU/GPU profile p50/p95 was 1.776/1.880 s, timing failed audio. Even with FP32-preserving GPU options, 84.89% of samples saturated with 11.389 relative L2. The 28-layer FP16 QNN talker step executed but changed CPU top-1 token 80 to 1720 with 9.43% logits relative L2. A TFLite CPU talker compile passed with same-fixture inference pending. No complete pipeline exists. | Isolate GPU delegate and talker numerical failures, audit pending CPU controls, then establish listening quality, co-resident streaming, memory and sustained behavior; [vocoder](../../../e2e_expansion_20260924/evidence/qwen_tts_vocoder_qualcomm/README.md) and [talker](../../../e2e_expansion_20260924/evidence/qwen_tts_talker_qualcomm/README.md) evidence. |
|
||||
| Embedded SA8775P ADP / AI Hub | Qwen3-TTS 0.6B CustomVoice | COMPONENT_CPU_VOCODER_PARITY_AND_PLACEMENT_PASS; COMPONENT_GPU_NUMERIC_FAIL; TALKER_FP16_NUMERIC_FAIL; NOT_E2E_PROFILEABLE | On the exact SA8775P, the same TFLite binary's CPU-requested Code2Wav inference matched original ONNX CPU within 4.13e-6 relative L2 / 107.67 dB SNR, without saturation; Its 100-sample CPU-requested component profile had p50/p95 5.384/5.534 s with all 738 execution-detail rows on CPU; the service estimated-time field 5.250 s is the sample minimum. The GPU-requested output saturated 96.63% of samples with 11.832 relative L2; its fresh 100-sample mixed CPU/GPU profile p50/p95 was 1.776/1.880 s, timing failed audio. Even with FP32-preserving GPU options, 84.89% of samples saturated with 11.389 relative L2. The 28-layer FP16 QNN talker step executed but changed CPU top-1 token 80 to 1720 with 9.43% logits relative L2. A TFLite CPU talker compile passed with same-fixture inference pending. No complete pipeline exists. | Isolate GPU delegate and talker numerical failures, audit pending talker CPU control, then establish listening quality, co-resident streaming, memory and sustained behavior; [vocoder](../../../e2e_expansion_20260924/evidence/qwen_tts_vocoder_qualcomm/README.md) and [talker](../../../e2e_expansion_20260924/evidence/qwen_tts_talker_qualcomm/README.md) evidence. |
|
||||
| Embedded SA8775P ADP / AI Hub | Qwen3.8-27B | NOT_E2E_PROFILEABLE | No complete qualified target artifact and device-local stateful Omni backend. Hub jobs provide component execution, not an on-device generation/streaming controller; ADB has no attached device. | Supply and validate the missing matching artifact/backend, then run the complete specified workload. |
|
||||
| Embedded SA8775P ADP / AI Hub | MiniCPM-o 4.5 | COMPONENT_SPEECH_HEAD_QNN_COMPILE_PASS; INFERENCE_PENDING; NOT_E2E_PROFILEABLE | The real 20-layer MiniCPM-o speech-head source compiled to FP16 QNN DLC for the exact SA8775P ADP. Same-fixture NPU-requested inference was pending, so output parity and actual placement are unknown. No complete thinker/encoders/vocoder bundle or device-local stateful controller exists. | Audit the [SA8775P speech-head inference](../../../e2e_expansion_20260924/evidence/minicpmo_tts_qualcomm/README.md), then qualify the complete multimodal pipeline and sustained behavior. |
|
||||
| Embedded SA8775P ADP / AI Hub | InternVLA-A1 | NOT_E2E_PROFILEABLE | No complete qualified target artifact and device-local stateful Omni backend. Hub jobs provide component execution, not an on-device generation/streaming controller; ADB has no attached device. | Supply and validate the missing matching artifact/backend, then run the complete specified workload. |
|
||||
|
||||
@@ -15,7 +15,7 @@ remain open; the 2026-09-22 audit remains the historical baseline.
|
||||
|
||||
The [Spark hosted Qualcomm component expansion](../../benchmarks/edge_harness/results/e2e_expansion_20260923/evidence/spark_s24_full_attention/README.md) advances M1 only to one real fixed-shape decoder attention layer. The same W8A16 QNN DLC ran on S24, Snapdragon X Elite CRD and SA8775P ADP NPUs with identical outputs on one pinned fixture; S25 had prior separate component evidence. On [RB3 Gen 2](../../benchmarks/edge_harness/results/e2e_expansion_20260923/evidence/spark_rb3_full_attention/README.md), the tested W8A16 graph failed QNN loading even after an exact-device compile. FP32 ONNX ran on CPU with near-source parity, while calibrated W8A8 ran on NPU but had 20.9% hidden-state relative L2 and remains numerically unqualified. These are component C results, not full M1: device-local 28-layer prefill/continuous decode, KV/ring, sampling, token quality, admission/cancellation, complete-request timing and sustained power/thermal gates remain open. AI Hub is the test facility, not a deployment dependency.
|
||||
|
||||
The [Qwen3-TTS Code2Wav Qualcomm follow-up](../../benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen_tts_vocoder_qualcomm/README.md) adds a retained same-fixture numerical check to M2. A fresh SA8775P GPU-requested output saturated 96.63% of samples and differs from original ONNX CPU by 11.832 relative L2, so that route fails the numerical gate despite profiling. The same binary's CPU-requested output on the exact SA device matched original ONNX CPU within 4.13e-6 relative L2; its placement profile remained pending. A FP32-preserving GPU-option control still saturated 84.89% of samples, so that option did not fix the tested route. RB3 TFLite GPU-requested inference returned an unsaturated but 15.73%-relative-L2 divergent waveform; its fresh 100-sample mixed CPU/GPU component p50/p95 was 3.099/3.133 s. X Elite ONNX CPU inference matched the local ONNX Runtime output with 8.09e-7 relative L2; its 94-sample component profile was p50/p95 6.384/6.644 s with all execution-detail rows on CPU. Corrected DirectML inference and profile both failed during provider initialization; separate FP16 QNN DLC compilation passed with NPU-requested inference pending. M2 still requires a quality-passing talker/predictor/vocoder set, device-local co-residency and handoff, playable streaming and sustained memory/power validation.
|
||||
The [Qwen3-TTS Code2Wav Qualcomm follow-up](../../benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen_tts_vocoder_qualcomm/README.md) adds a retained same-fixture numerical check to M2. A fresh SA8775P GPU-requested output saturated 96.63% of samples and differs from original ONNX CPU by 11.832 relative L2, so that route fails the numerical gate despite profiling. The same binary's CPU-requested output on the exact SA device matched original ONNX CPU within 4.13e-6 relative L2; its 100-sample CPU-requested component profile passed placement (738/738 execution-detail rows on CPU) at nearest-rank p50/p95 5.384/5.534 s, still without a full stream. A FP32-preserving GPU-option control still saturated 84.89% of samples, so that option did not fix the tested route. RB3 TFLite GPU-requested inference returned an unsaturated but 15.73%-relative-L2 divergent waveform; its fresh 100-sample mixed CPU/GPU component p50/p95 was 3.099/3.133 s. X Elite ONNX CPU inference matched the local ONNX Runtime output with 8.09e-7 relative L2; its 94-sample component profile was p50/p95 6.384/6.644 s with all execution-detail rows on CPU. Corrected DirectML inference and profile both failed during provider initialization; separate FP16 QNN DLC compilation passed with NPU-requested inference pending. M2 still requires a quality-passing talker/predictor/vocoder set, device-local co-residency and handoff, playable streaming and sustained memory/power validation.
|
||||
|
||||
The identical SA/RB3 TFLite vocoder binary also matched original ONNX CPU within 3.45e-6 relative L2 on a separate local HX370 x86-64 LiteRT 2.2.0 CPU control. Exact-SA CPU-requested parity narrows the failure to the GPU-requested route. The FP32-preserving GPU option did not fix it, so per-node placement and delegate-operation isolation remain open.
|
||||
|
||||
|
||||
Reference in new issue
Block a user