Record SA speech head and X Elite QNN vocoder results
This commit is contained in:
tzhouam committed
1 parent
76941854b1
commit
e5aaf3186c
13 files changed
+457
-11
No files matched your search
@@ -1,11 +1,11 @@
|
||||
# 2026-09-24 Qualcomm component expansion
|
||||
|
||||
The [Qwen3-TTS vocoder study](evidence/qwen_tts_vocoder_qualcomm/README.md) reuses a retained fixed-shape Code2Wav export and historical Workbench input to test numerical behavior on SA8775P and other Qualcomm settings. The fresh SA8775P GPU-requested inference produces a saturated waveform, while the same binary's CPU-requested output on the exact SA device matches ONNX CPU within 4.13e-6 relative L2. Its 100-sample CPU-requested component profile had p50/p95 5.384/5.534 s with all 738 execution-detail rows on CPU. A FP32-preserving GPU-option control still saturated 84.89% of samples, so that option did not repair the tested path. RB3 produces an unsaturated but 15.73%-divergent waveform; its fresh 100-sample mixed CPU/GPU component p50/p95 was 3.099/3.133 s, not a usable TTS timing claim. X Elite ONNX CPU output matches the local ONNX Runtime reference on the retained fixture; its 94-sample component p50/p95 was 6.384/6.644 s with all execution-detail rows on CPU. Corrected X Elite DirectML inference and profile both failed during provider initialization; separate FP16 QNN DLC compilation passed, with NPU-requested inference pending. A separate SA8775P FP16 QNN vocoder compile passed; same-fixture NPU-requested inference is pending. No complete on-device TTS stream or model-quality qualification is claimed.
|
||||
The [Qwen3-TTS vocoder study](evidence/qwen_tts_vocoder_qualcomm/README.md) reuses a retained fixed-shape Code2Wav export and historical Workbench input to test numerical behavior on SA8775P and other Qualcomm settings. The fresh SA8775P GPU-requested inference produces a saturated waveform, while the same binary's CPU-requested output on the exact SA device matches ONNX CPU within 4.13e-6 relative L2. Its 100-sample CPU-requested component profile had p50/p95 5.384/5.534 s with all 738 execution-detail rows on CPU. A FP32-preserving GPU-option control still saturated 84.89% of samples, so that option did not repair the tested path. RB3 produces an unsaturated but 15.73%-divergent waveform; its fresh 100-sample mixed CPU/GPU component p50/p95 was 3.099/3.133 s, not a usable TTS timing claim. X Elite ONNX CPU output matches the local ONNX Runtime reference on the retained fixture; its 94-sample component p50/p95 was 6.384/6.644 s with all execution-detail rows on CPU. Corrected X Elite DirectML inference and profile both failed during provider initialization; separate FP16 QNN DLC inference returned an unsaturated waveform at 1.138% relative L2 / 38.88 dB SNR against original ONNX CPU; placement profiling is pending. A separate SA8775P FP16 QNN vocoder compile passed; same-fixture NPU-requested inference is pending. No complete on-device TTS stream or model-quality qualification is claimed.
|
||||
|
||||
A local HX370 x86-64 LiteRT CPU control of the identical SA/RB3 TFLite binary matched the original ONNX CPU waveform within 3.45e-6 relative L2. The SA saturation is therefore not universal to the converted artifact; the exact SA CPU control passed numerical and placement checks, while the faulty GPU delegate operation remains unidentified.
|
||||
|
||||
The [talker decode-step attempt](evidence/qwen_tts_talker_qualcomm/README.md) pins a separate 28-layer source and synthetic cache fixture. Historical FP16 QNN context-binary inference executed on S24, S25, SA8775P and X Elite, but all four changed the CPU top-1 speech token with 8.57–9.43% logits relative L2. These artifacts are numerically unqualified on the fixture. RB3 rejected the floating-point-input artifact. Explicit X Elite ONNX CPU and SA8775P TFLite CPU fallback compiles passed, with same-fixture inference pending. No complete talker loop or TTS stream passed.
|
||||
|
||||
The [MiniCPM-o speech-head study](evidence/minicpmo_tts_qualcomm/README.md) recovered the real 20-layer ONNX source and a matching retained fixture. S24 and S25 FP16 context-binary outputs preserved the ONNX CPU top-1 speech token with 0.346%/0.360% logits relative L2 on one step. FP16 QNN DLC compiles passed for SA8775P, X Elite and RB3, with same-fixture NPU-requested inference pending. The complete multimodal model remains unqualified on those devices.
|
||||
The [MiniCPM-o speech-head study](evidence/minicpmo_tts_qualcomm/README.md) recovered the real 20-layer ONNX source and a matching retained fixture. S24 and S25 FP16 context-binary outputs preserved the ONNX CPU top-1 speech token with 0.346%/0.360% logits relative L2 on one step. FP16 QNN DLC compiles passed for SA8775P, X Elite and RB3. SA8775P NPU-requested inference preserved CPU top-1 token 1867 with 0.3568% logits relative L2; placement profiling remains pending, as do X Elite and RB3 inferences. The complete multimodal model remains unqualified on those devices.
|
||||
|
||||
The [InternVLA Cosmos S25 attempt](evidence/internvla_cosmos_qualcomm/README.md) repackages the real policy's image encoder from its parity-tested ONNX export. FLOAT16 QNN DLC inference returned a latent within 0.0524% relative L2 of the source on one pattern. Injecting it into the real policy on local CPU changed synthetic actions by 0.742% relative L2, without a task tolerance. NPU placement profiling was pending; device-local action-policy support remains unqualified.
|
||||
+1
-1
@@ -6,6 +6,6 @@ The [S25 historical inference](s25/inference_report.json) used dataset `d7gwxe3y
|
||||
|
||||
The historical [S24](s24/compile_report.json) and [S25](s25/compile_report.json) FP16 QNN context-binary compiles and inference jobs succeeded. Against the **same original ONNX CPU fixture**, [S24](s24/audit_report.json) logits relative L2 was **0.3459%** and max K/V relative L2 **1.235%**; [S25](s25/audit_report.json) logits relative L2 was **0.3605%** and max K/V relative L2 **1.311%**. Both preserved the CPU top-1 speech token **1867** on this one synthetic step. The raw [S24](s24/device_output.npz) and [S25](s25/device_output.npz) arrays are retained. [S24](s24/profile_submission.json) and [S25](s25/profile_submission.json) placement profiles were submitted and pending. These values remain numerical component evidence, without a generated-audio quality threshold or actual NPU placement proof.
|
||||
|
||||
New FP16 QNN DLC [SA8775P](sa8775p/compile_report.json), [Snapdragon X Elite](xelite/compile_report.json) and [RB3 Gen 2](rb3/compile_report.json) compiles passed from the same source. Same-fixture NPU-requested inference jobs were [submitted for SA8775P](sa8775p/inference_submission.json), [X Elite](xelite/inference_submission.json) and [RB3](rb3/inference_submission.json), but were pending at this snapshot. A compile cannot establish device execution or numerical parity. The [collector](../../../../experiments/collect_aihub_component.py) retrieves terminal jobs, and the [auditor](../../../../experiments/audit_minicpmo_tts_qualcomm.py) pins hashes and checks all 41 output tensors.
|
||||
New FP16 QNN DLC [SA8775P](sa8775p/compile_report.json), [Snapdragon X Elite](xelite/compile_report.json) and [RB3 Gen 2](rb3/compile_report.json) compiles passed from the same source. The SA8775P [NPU-requested inference](sa8775p/inference_report.json) returned all 41 finite tensors on the retained fixture. Its [audit](sa8775p/audit_report.json) found **0.3568% logits relative L2** versus ONNX CPU, preserved CPU top-1 speech token **1867**, and measured maximum K/V relative L2 **1.225%**. The [placement profile](sa8775p/profile_submission.json) was submitted and remains pending; the inference request alone does not prove actual NPU node assignment. Same-fixture NPU-requested inference jobs for [X Elite](xelite/inference_submission.json) and [RB3](rb3/inference_submission.json) also remain pending. The [collector](../../../../experiments/collect_aihub_component.py) retrieves terminal jobs, and the [auditor](../../../../experiments/audit_minicpmo_tts_qualcomm.py) pins hashes and checks all 41 output tensors.
|
||||
|
||||
Next: audit those device outputs and placement, then qualify the entire model with real multimodal inputs, stateful speech generation, task/audio quality, memory admission, cancellation and sustained power/thermal behavior. AI Hub is the test facility, not a deployment dependency.
|
||||
+304
@@ -0,0 +1,304 @@
|
||||
{
|
||||
"scope": "one fixed-cache 20-layer MiniCPM-o speech-head decode step; no complete multimodal stream",
|
||||
"status": "component_inference_numeric_measured",
|
||||
"device": {
|
||||
"name": "SA8775P ADP",
|
||||
"os": "14",
|
||||
"attributes": [
|
||||
"os:android",
|
||||
"framework:tflite",
|
||||
"framework:onnx",
|
||||
"vendor:qualcomm",
|
||||
"abi:aarch64-android",
|
||||
"format:auto",
|
||||
"chipset:qualcomm-sa8775p",
|
||||
"hexagon:v73",
|
||||
"soc-model:52",
|
||||
"htp-supports-fp16:true",
|
||||
"htp-supports-weight-sharing:true",
|
||||
"framework:qnn"
|
||||
]
|
||||
},
|
||||
"compile_job_id": "jpe7zlz15",
|
||||
"inference_job_id": "jp3z9wol5",
|
||||
"source_model_id": "mn4gjjpvn",
|
||||
"target_model_id": "mn1lvg1zq",
|
||||
"input_dataset_id": "d7gwxe3y2",
|
||||
"requested_run_options": "--compute_unit npu",
|
||||
"files": {
|
||||
"source_onnx": {
|
||||
"bytes": 776341465,
|
||||
"sha256": "78cf64804f11ad269ee4180da61588ccc5eefe2942773227345499756e4cc229"
|
||||
},
|
||||
"fixture": {
|
||||
"bytes": 29247541,
|
||||
"sha256": "65919ea1f351feec33c1415a3e3db76d60bc9c2beb3d49a72af75dcb418e83b4"
|
||||
},
|
||||
"cpu_reference": {
|
||||
"bytes": 148471,
|
||||
"sha256": "902adcceadbe761d938ef2ade51cf953c8917ea1abfaee33a10c84e300c1bd96"
|
||||
},
|
||||
"device_output": {
|
||||
"bytes": 105664,
|
||||
"sha256": "0678b4a45ad04a43fa3dba60d6df30a3278502319028213a058235bad09541f3"
|
||||
}
|
||||
},
|
||||
"tensor_comparison": {
|
||||
"logits": {
|
||||
"relative_l2": 0.003567560806952834,
|
||||
"max_abs_error": 0.017552852630615234,
|
||||
"source_rms": 1.4974441755558239,
|
||||
"device_rms": 1.49778515160833,
|
||||
"cpu_top1": 1867,
|
||||
"device_top1": 1867,
|
||||
"top1_matches": true
|
||||
},
|
||||
"k_new_0": {
|
||||
"relative_l2": 0.0005147656431265523,
|
||||
"max_abs_error": 0.003742694854736328,
|
||||
"source_rms": 1.307108108274551,
|
||||
"device_rms": 1.307038088812209
|
||||
},
|
||||
"v_new_0": {
|
||||
"relative_l2": 0.00035345905779380455,
|
||||
"max_abs_error": 0.0007248520851135254,
|
||||
"source_rms": 0.5083994128636046,
|
||||
"device_rms": 0.5084141361776617
|
||||
},
|
||||
"k_new_1": {
|
||||
"relative_l2": 0.0005500608894909682,
|
||||
"max_abs_error": 0.0020580291748046875,
|
||||
"source_rms": 0.7542700462780494,
|
||||
"device_rms": 0.7542603027325883
|
||||
},
|
||||
"v_new_1": {
|
||||
"relative_l2": 0.0010050405070637515,
|
||||
"max_abs_error": 0.00030508264899253845,
|
||||
"source_rms": 0.0662130257652199,
|
||||
"device_rms": 0.06622140591939649
|
||||
},
|
||||
"k_new_2": {
|
||||
"relative_l2": 0.0012461611526044514,
|
||||
"max_abs_error": 0.0056171417236328125,
|
||||
"source_rms": 0.9817805619626084,
|
||||
"device_rms": 0.9819102513741523
|
||||
},
|
||||
"v_new_2": {
|
||||
"relative_l2": 0.0013900371877606159,
|
||||
"max_abs_error": 0.0017390847206115723,
|
||||
"source_rms": 0.24438175712240315,
|
||||
"device_rms": 0.24440075632447608
|
||||
},
|
||||
"k_new_3": {
|
||||
"relative_l2": 0.0041692088872177825,
|
||||
"max_abs_error": 0.013458549976348877,
|
||||
"source_rms": 0.9312003114105484,
|
||||
"device_rms": 0.9324818001010153
|
||||
},
|
||||
"v_new_3": {
|
||||
"relative_l2": 0.004415686592245475,
|
||||
"max_abs_error": 0.005837738513946533,
|
||||
"source_rms": 0.27179250329538757,
|
||||
"device_rms": 0.2722113226474472
|
||||
},
|
||||
"k_new_4": {
|
||||
"relative_l2": 0.007806370255517516,
|
||||
"max_abs_error": 0.039246320724487305,
|
||||
"source_rms": 1.2078244698126999,
|
||||
"device_rms": 1.2115194303912318
|
||||
},
|
||||
"v_new_4": {
|
||||
"relative_l2": 0.008498422540275942,
|
||||
"max_abs_error": 0.01820969581604004,
|
||||
"source_rms": 0.4929021962159226,
|
||||
"device_rms": 0.49430772948802415
|
||||
},
|
||||
"k_new_5": {
|
||||
"relative_l2": 0.00766747652531978,
|
||||
"max_abs_error": 0.04518893361091614,
|
||||
"source_rms": 1.0960093751879882,
|
||||
"device_rms": 1.0970983805182493
|
||||
},
|
||||
"v_new_5": {
|
||||
"relative_l2": 0.012251591482087142,
|
||||
"max_abs_error": 0.027927041053771973,
|
||||
"source_rms": 0.49114616760529134,
|
||||
"device_rms": 0.4897185372549937
|
||||
},
|
||||
"k_new_6": {
|
||||
"relative_l2": 0.006232619119844742,
|
||||
"max_abs_error": 0.03511066734790802,
|
||||
"source_rms": 1.1726536779527166,
|
||||
"device_rms": 1.173427212942265
|
||||
},
|
||||
"v_new_6": {
|
||||
"relative_l2": 0.009430287134704689,
|
||||
"max_abs_error": 0.017375268042087555,
|
||||
"source_rms": 0.5369044199454948,
|
||||
"device_rms": 0.5364518269853782
|
||||
},
|
||||
"k_new_7": {
|
||||
"relative_l2": 0.004003676029659081,
|
||||
"max_abs_error": 0.024735689163208008,
|
||||
"source_rms": 1.6917150856043028,
|
||||
"device_rms": 1.6942078465587256
|
||||
},
|
||||
"v_new_7": {
|
||||
"relative_l2": 0.007020680815137411,
|
||||
"max_abs_error": 0.01995902508497238,
|
||||
"source_rms": 0.8116460286615261,
|
||||
"device_rms": 0.8107948375239711
|
||||
},
|
||||
"k_new_8": {
|
||||
"relative_l2": 0.005007475836731906,
|
||||
"max_abs_error": 0.0369257926940918,
|
||||
"source_rms": 1.5021910590457033,
|
||||
"device_rms": 1.5033794338233282
|
||||
},
|
||||
"v_new_8": {
|
||||
"relative_l2": 0.006781734670990391,
|
||||
"max_abs_error": 0.030480504035949707,
|
||||
"source_rms": 0.8807572129778053,
|
||||
"device_rms": 0.8792290592358764
|
||||
},
|
||||
"k_new_9": {
|
||||
"relative_l2": 0.00492410033486205,
|
||||
"max_abs_error": 0.01682913303375244,
|
||||
"source_rms": 1.1054146201792987,
|
||||
"device_rms": 1.1058413215185963
|
||||
},
|
||||
"v_new_9": {
|
||||
"relative_l2": 0.007765860398483337,
|
||||
"max_abs_error": 0.01826465129852295,
|
||||
"source_rms": 0.6580213796976341,
|
||||
"device_rms": 0.6567637071915982
|
||||
},
|
||||
"k_new_10": {
|
||||
"relative_l2": 0.004584154593930242,
|
||||
"max_abs_error": 0.0197451114654541,
|
||||
"source_rms": 1.325937063058518,
|
||||
"device_rms": 1.3267830930986437
|
||||
},
|
||||
"v_new_10": {
|
||||
"relative_l2": 0.00690639583671994,
|
||||
"max_abs_error": 0.021770477294921875,
|
||||
"source_rms": 0.848351970479369,
|
||||
"device_rms": 0.8467916042764727
|
||||
},
|
||||
"k_new_11": {
|
||||
"relative_l2": 0.003568876099022461,
|
||||
"max_abs_error": 0.008406057953834534,
|
||||
"source_rms": 0.6808336872062942,
|
||||
"device_rms": 0.6805778891840785
|
||||
},
|
||||
"v_new_11": {
|
||||
"relative_l2": 0.0062608921146686085,
|
||||
"max_abs_error": 0.008449621498584747,
|
||||
"source_rms": 0.3676426367961426,
|
||||
"device_rms": 0.36720307992091616
|
||||
},
|
||||
"k_new_12": {
|
||||
"relative_l2": 0.004120611025694297,
|
||||
"max_abs_error": 0.009145379066467285,
|
||||
"source_rms": 0.6128882051701765,
|
||||
"device_rms": 0.6123276571726385
|
||||
},
|
||||
"v_new_12": {
|
||||
"relative_l2": 0.005789469448313205,
|
||||
"max_abs_error": 0.007241338491439819,
|
||||
"source_rms": 0.34029547623567724,
|
||||
"device_rms": 0.33996249605749584
|
||||
},
|
||||
"k_new_13": {
|
||||
"relative_l2": 0.0041062443844702284,
|
||||
"max_abs_error": 0.008440196514129639,
|
||||
"source_rms": 0.613775761616868,
|
||||
"device_rms": 0.6135322120417965
|
||||
},
|
||||
"v_new_13": {
|
||||
"relative_l2": 0.0064739931567895405,
|
||||
"max_abs_error": 0.009202837944030762,
|
||||
"source_rms": 0.36907442918730654,
|
||||
"device_rms": 0.36852288769028935
|
||||
},
|
||||
"k_new_14": {
|
||||
"relative_l2": 0.004067945603752527,
|
||||
"max_abs_error": 0.009853065013885498,
|
||||
"source_rms": 0.5980841827735426,
|
||||
"device_rms": 0.597757474085897
|
||||
},
|
||||
"v_new_14": {
|
||||
"relative_l2": 0.006087710181292307,
|
||||
"max_abs_error": 0.006891041994094849,
|
||||
"source_rms": 0.3506656009170703,
|
||||
"device_rms": 0.3503050706758771
|
||||
},
|
||||
"k_new_15": {
|
||||
"relative_l2": 0.0036772003233247075,
|
||||
"max_abs_error": 0.008877158164978027,
|
||||
"source_rms": 0.622821559168189,
|
||||
"device_rms": 0.6225018228505026
|
||||
},
|
||||
"v_new_15": {
|
||||
"relative_l2": 0.005430525985090655,
|
||||
"max_abs_error": 0.00739055871963501,
|
||||
"source_rms": 0.41823950764909973,
|
||||
"device_rms": 0.4179159811522275
|
||||
},
|
||||
"k_new_16": {
|
||||
"relative_l2": 0.003264454976257468,
|
||||
"max_abs_error": 0.007949531078338623,
|
||||
"source_rms": 0.6754924715823191,
|
||||
"device_rms": 0.6754340976948906
|
||||
},
|
||||
"v_new_16": {
|
||||
"relative_l2": 0.005707659819949586,
|
||||
"max_abs_error": 0.009386241436004639,
|
||||
"source_rms": 0.4659773620123202,
|
||||
"device_rms": 0.465740418964803
|
||||
},
|
||||
"k_new_17": {
|
||||
"relative_l2": 0.0034706820491854726,
|
||||
"max_abs_error": 0.008044421672821045,
|
||||
"source_rms": 0.6354120615871959,
|
||||
"device_rms": 0.6352736448954285
|
||||
},
|
||||
"v_new_17": {
|
||||
"relative_l2": 0.005771627117003953,
|
||||
"max_abs_error": 0.012461543083190918,
|
||||
"source_rms": 0.5384818316759848,
|
||||
"device_rms": 0.5380992153565227
|
||||
},
|
||||
"k_new_18": {
|
||||
"relative_l2": 0.0030676398017172924,
|
||||
"max_abs_error": 0.008172482252120972,
|
||||
"source_rms": 0.7283659936841244,
|
||||
"device_rms": 0.7284984246418575
|
||||
},
|
||||
"v_new_18": {
|
||||
"relative_l2": 0.004139600477203072,
|
||||
"max_abs_error": 0.009913921356201172,
|
||||
"source_rms": 0.6382820941879574,
|
||||
"device_rms": 0.6382350948820309
|
||||
},
|
||||
"k_new_19": {
|
||||
"relative_l2": 0.0037395269690038793,
|
||||
"max_abs_error": 0.007453262805938721,
|
||||
"source_rms": 0.5790109825658509,
|
||||
"device_rms": 0.5792377105543774
|
||||
},
|
||||
"v_new_19": {
|
||||
"relative_l2": 0.004201040698803878,
|
||||
"max_abs_error": 0.010617375373840332,
|
||||
"source_rms": 0.7220926350353908,
|
||||
"device_rms": 0.7220641745739507
|
||||
}
|
||||
},
|
||||
"cache_max_relative_l2": 0.012251591482087142,
|
||||
"limits": [
|
||||
"The historical ONNX export does not attest the exact source checkpoint revision.",
|
||||
"One retained decode fixture cannot establish speech-token or generated-audio quality.",
|
||||
"Actual NPU placement requires a device profile; a request or QNN artifact alone is insufficient.",
|
||||
"Thinker, audio/vision encoders, vocoder, state loop and complete Omni request were not tested."
|
||||
]
|
||||
}
|
||||
BIN
Binary file not shown.
+32
@@ -0,0 +1,32 @@
|
||||
{
|
||||
"timestamp_utc": "2026-09-24T00:11:57.384130+00:00",
|
||||
"qai_hub_version": "0.55.0",
|
||||
"facility": "Qualcomm AI Hub Workbench",
|
||||
"job_id": "jp3z9wol5",
|
||||
"url": "https://workbench.aihub.qualcomm.com/jobs/jp3z9wol5/",
|
||||
"status": "SUCCESS",
|
||||
"status_message": "",
|
||||
"options": "--compute_unit npu",
|
||||
"hub_version": "aihub-2026.09.11.0",
|
||||
"device": {
|
||||
"name": "SA8775P ADP",
|
||||
"os": "14",
|
||||
"attributes": [
|
||||
"os:android",
|
||||
"framework:tflite",
|
||||
"framework:onnx",
|
||||
"vendor:qualcomm",
|
||||
"abi:aarch64-android",
|
||||
"format:auto",
|
||||
"chipset:qualcomm-sa8775p",
|
||||
"hexagon:v73",
|
||||
"soc-model:52",
|
||||
"htp-supports-fp16:true",
|
||||
"htp-supports-weight-sharing:true",
|
||||
"framework:qnn"
|
||||
]
|
||||
},
|
||||
"model_id": "mn1lvg1zq",
|
||||
"input_dataset_id": "d7gwxe3y2",
|
||||
"outputs": "device_output.npz"
|
||||
}
|
||||
+13
@@ -0,0 +1,13 @@
|
||||
{
|
||||
"timestamp_utc": "2026-09-24T00:12:31.486172+00:00",
|
||||
"facility": "Qualcomm AI Hub Workbench",
|
||||
"qai_hub_version": "0.55.0",
|
||||
"source_compile_job_id": "jpe7zlz15",
|
||||
"target_model_id": "mn1lvg1zq",
|
||||
"device": "SA8775P ADP",
|
||||
"options": "--compute_unit npu",
|
||||
"kind": "profile",
|
||||
"status": "SUBMITTED",
|
||||
"job_id": "jpvld7w75",
|
||||
"url": "https://workbench.aihub.qualcomm.com/jobs/jpvld7w75/"
|
||||
}
|
||||
+2
-2
@@ -14,8 +14,8 @@ On the **exact hosted SA8775P**, a [CPU-requested inference](sa8775p/cpu_control
|
||||
|
||||
The re-fetched *historical* [SA8775P profile](sa8775p/historical_profile_report.json) has 100 component samples, nearest-rank p50/p95 **1.751/1.874 s**, and 403 CPU plus 335 GPU execution-detail rows. The *historical* [RB3 profile](rb3/historical_profile_report.json) has 100 samples, p50/p95 **3.143/3.189 s**, also with 403 CPU plus 335 GPU rows. Their previously quoted 1.701/3.107 s estimated-time fields are sample minima, not medians. Both profiles exclude talker/predictor, stage handoff, streaming and full-request time.
|
||||
|
||||
The X Elite [ONNX CPU inference](xelite/cpu/inference_report.json) succeeded on the pinned fixture. Its [waveform comparison](xelite/cpu/inference_comparison.json) to the original local ONNX Runtime CPU output was **8.09e-7 relative L2 / 121.84 dB SNR**, with no saturation. The fresh [CPU profile](xelite/cpu/profile_report.json) returned **94** component samples, nearest-rank p50/p95 **6.384/6.644 s**, with all **634** execution-detail rows on CPU; the service's 5.917 s estimated-time field is the sample minimum. The [full audit](xelite/cpu/audit_report.json) pins source, fixture, target, waveform, placement and percentiles. This is near-source component parity on one fixture, without a listening-quality or complete-stream gate. The first X Elite [GPU inference](xelite/gpu/inference_report.json) and [profile](xelite/gpu/profile_report.json) jobs failed because Workbench rejects specifying both compute unit and ONNX execution provider. The corrected [DirectML inference](xelite/directml/inference_report.json) and [profile](xelite/directml/profile_report.json) both failed during DML provider initialization with Windows error `80070057` (`The parameter is incorrect`), without an output or timing samples. A separate [FP16 QNN DLC compile](xelite/qnn_npu/compile_report.json) succeeded and [NPU-requested same-fixture inference](xelite/qnn_npu/inference_submission.json) was pending. The [X Elite ONNX CPU compile](xelite/compile_report.json) passed after the older TFLite target was rejected on Windows ARM. These failures constrain the tested DirectML route, not all GPU or NPU artifacts.
|
||||
The X Elite [ONNX CPU inference](xelite/cpu/inference_report.json) succeeded on the pinned fixture. Its [waveform comparison](xelite/cpu/inference_comparison.json) to the original local ONNX Runtime CPU output was **8.09e-7 relative L2 / 121.84 dB SNR**, with no saturation. The fresh [CPU profile](xelite/cpu/profile_report.json) returned **94** component samples, nearest-rank p50/p95 **6.384/6.644 s**, with all **634** execution-detail rows on CPU; the service's 5.917 s estimated-time field is the sample minimum. The [full audit](xelite/cpu/audit_report.json) pins source, fixture, target, waveform, placement and percentiles. This is near-source component parity on one fixture, without a listening-quality or complete-stream gate. The first X Elite [GPU inference](xelite/gpu/inference_report.json) and [profile](xelite/gpu/profile_report.json) jobs failed because Workbench rejects specifying both compute unit and ONNX execution provider. The corrected [DirectML inference](xelite/directml/inference_report.json) and [profile](xelite/directml/profile_report.json) both failed during DML provider initialization with Windows error `80070057` (`The parameter is incorrect`), without an output or timing samples. A separate [FP16 QNN DLC compile](xelite/qnn_npu/compile_report.json) succeeded. Its [NPU-requested same-fixture inference](xelite/qnn_npu/inference_report.json) returned a finite, unsaturated waveform at **1.138% relative L2 / 38.88 dB SNR** versus original ONNX CPU ([comparison](xelite/qnn_npu/inference_comparison.json)). This is better waveform agreement on this fixture than the historical S25 GPU output, but lacks a listening-quality tolerance and verified NPU node placement; the [profile](xelite/qnn_npu/profile_submission.json) is pending. The [X Elite ONNX CPU compile](xelite/compile_report.json) passed after the older TFLite target was rejected on Windows ARM. These failures constrain the tested DirectML route, not all GPU or NPU artifacts.
|
||||
|
||||
The X Elite [QNN NPU inference](xelite/qnn_npu/inference_submission.json) remained pending. A new SA8775P [FP16 QNN DLC compile](sa8775p/qnn_npu/compile_report.json) passed from the same source ONNX, producing target `mqvovl9lm`; [same-fixture NPU-requested inference](sa8775p/qnn_npu/inference_submission.json) was submitted. This tests a separate accelerator artifact because the existing TFLite GPU output fails numerically. The compile alone has no numerical or placement conclusion. The [collector](../../../../experiments/collect_aihub_component.py) retrieves terminal inference/profile reports without resubmitting jobs, and the [audit script](../../../../experiments/audit_qwen_tts_vocoder_qualcomm.py) checks source/fixture hashes, device and model IDs, actual placement, waveform error, saturation and sample percentiles.
|
||||
A new SA8775P [FP16 QNN DLC compile](sa8775p/qnn_npu/compile_report.json) passed from the same source ONNX, producing target `mqvovl9lm`; [same-fixture NPU-requested inference](sa8775p/qnn_npu/inference_submission.json) was submitted and remains pending. This tests a separate accelerator artifact because the existing TFLite GPU output fails numerically. The compile alone has no numerical or placement conclusion. The [collector](../../../../experiments/collect_aihub_component.py) retrieves terminal inference/profile reports without resubmitting jobs, and the [audit script](../../../../experiments/audit_qwen_tts_vocoder_qualcomm.py) checks source/fixture hashes, device and model IDs, actual placement, waveform error, saturation and sample percentiles.
|
||||
|
||||
This is one fixed vocoder component with a retained input, not complete Qwen3-TTS. The next gates are a quality-passing SA/RB3/X Elite target, talker and code predictor numerical checks, device-resident handoff/co-residency, playable streaming, cancellation, memory admission and sustained power/thermal profiling.
|
||||
BIN
Binary file not shown.
+52
@@ -0,0 +1,52 @@
|
||||
{
|
||||
"scope": "one Qwen3-TTS Code2Wav output on retained fixture; no complete TTS or profile",
|
||||
"status": "component_inference_numerically_compared",
|
||||
"device": {
|
||||
"name": "Snapdragon X Elite CRD",
|
||||
"os": "11",
|
||||
"attributes": [
|
||||
"os:windows",
|
||||
"format:compute",
|
||||
"framework:onnx",
|
||||
"vendor:qualcomm",
|
||||
"abi:aarch64-windows",
|
||||
"chipset:qualcomm-snapdragon-x-elite",
|
||||
"chipset:sc8380xp",
|
||||
"hexagon:v73",
|
||||
"soc-model:60",
|
||||
"htp-supports-fp16:true",
|
||||
"htp-supports-weight-sharing:true",
|
||||
"framework:qnn"
|
||||
]
|
||||
},
|
||||
"compile_job_id": "jgdd3zrkg",
|
||||
"inference_job_id": "jprl0n9ep",
|
||||
"source_model_id": "mqyer339n",
|
||||
"target_model_id": "mmxj098kq",
|
||||
"input_dataset_id": "d7m80jl32",
|
||||
"requested_run_options": "--compute_unit npu",
|
||||
"output_sha256": "345db154c238eb4a488a672a06c84fbef6ecfc57ba0631e6ebdb3c8762b0eb0d",
|
||||
"cpu_source_vs_xelite_npu": {
|
||||
"relative_l2": 0.011379857331086082,
|
||||
"snr_db": 38.87726365286192,
|
||||
"max_abs": 0.01465548574924469,
|
||||
"cosine": 0.9999354554131855,
|
||||
"reference_rms": 0.08373439205485074,
|
||||
"output_rms": 0.0836749709653091,
|
||||
"output_saturated_fraction_abs_ge_0_999": 0.0
|
||||
},
|
||||
"s25_gpu_vs_xelite_npu": {
|
||||
"relative_l2": 0.04252948781511364,
|
||||
"snr_db": 27.426196949721074,
|
||||
"max_abs": 0.1015625037252903,
|
||||
"cosine": 0.9990952328935694,
|
||||
"reference_rms": 0.08376787228276501,
|
||||
"output_rms": 0.0836749709653091,
|
||||
"output_saturated_fraction_abs_ge_0_999": 0.0
|
||||
},
|
||||
"limits": [
|
||||
"No listening or task-level waveform tolerance.",
|
||||
"NPU node-placement profile not yet measured.",
|
||||
"Historical export lacks attested exact checkpoint revision."
|
||||
]
|
||||
}
|
||||
+32
@@ -0,0 +1,32 @@
|
||||
{
|
||||
"timestamp_utc": "2026-09-24T00:13:50.876473+00:00",
|
||||
"qai_hub_version": "0.55.0",
|
||||
"facility": "Qualcomm AI Hub Workbench",
|
||||
"job_id": "jprl0n9ep",
|
||||
"url": "https://workbench.aihub.qualcomm.com/jobs/jprl0n9ep/",
|
||||
"status": "SUCCESS",
|
||||
"status_message": "",
|
||||
"options": "--compute_unit npu",
|
||||
"hub_version": "aihub-2026.09.11.0",
|
||||
"device": {
|
||||
"name": "Snapdragon X Elite CRD",
|
||||
"os": "11",
|
||||
"attributes": [
|
||||
"os:windows",
|
||||
"format:compute",
|
||||
"framework:onnx",
|
||||
"vendor:qualcomm",
|
||||
"abi:aarch64-windows",
|
||||
"chipset:qualcomm-snapdragon-x-elite",
|
||||
"chipset:sc8380xp",
|
||||
"hexagon:v73",
|
||||
"soc-model:60",
|
||||
"htp-supports-fp16:true",
|
||||
"htp-supports-weight-sharing:true",
|
||||
"framework:qnn"
|
||||
]
|
||||
},
|
||||
"model_id": "mmxj098kq",
|
||||
"input_dataset_id": "d7m80jl32",
|
||||
"outputs": "device_output.npz"
|
||||
}
|
||||
+13
@@ -0,0 +1,13 @@
|
||||
{
|
||||
"timestamp_utc": "2026-09-24T00:14:35.588365+00:00",
|
||||
"facility": "Qualcomm AI Hub Workbench",
|
||||
"qai_hub_version": "0.55.0",
|
||||
"source_compile_job_id": "jgdd3zrkg",
|
||||
"target_model_id": "mmxj098kq",
|
||||
"device": "Snapdragon X Elite CRD",
|
||||
"options": "--compute_unit npu",
|
||||
"kind": "profile",
|
||||
"status": "SUBMITTED",
|
||||
"job_id": "jpyoxmvr5",
|
||||
"url": "https://workbench.aihub.qualcomm.com/jobs/jpyoxmvr5/"
|
||||
}
|
||||
@@ -12,7 +12,7 @@ On a hosted [Galaxy S24 Spark attention-layer experiment](../../../e2e_expansion
|
||||
|
||||
The same W8A16 layer also ran on hosted [Snapdragon X Elite CRD](../../../e2e_expansion_20260923/evidence/spark_xelite_full_attention/README.md) and [SA8775P ADP](../../../e2e_expansion_20260923/evidence/spark_sa8775_full_attention/README.md) NPUs, with pinned-fixture outputs identical to S24/S25; the 100-sample component p50/p95 was 1.433/1.606 ms and 1.837/2.079 ms, respectively. On [RB3 Gen 2 / QCS6490](../../../e2e_expansion_20260923/evidence/spark_rb3_full_attention/README.md), that W8A16 graph failed QNN DLC loading even after target-specific recompilation. An FP32 ONNX CPU layer then passed with source-relative L2 below 9e-7, while a calibrated W8A8 NPU layer executed but had 20.9% hidden-state relative L2 and remains numerically unqualified. These hosted component outcomes do not establish full generation or comparable device speed.
|
||||
|
||||
A [Qwen3-TTS Code2Wav follow-up](../../../e2e_expansion_20260924/evidence/qwen_tts_vocoder_qualcomm/README.md) recovered source and target artifacts, a retained fixture, and raw historical profiles. On SA8775P, fresh GPU-requested inference saturated 96.63% of waveform samples and differed from original ONNX CPU by 11.832 relative L2; the same binary's CPU-requested output on the exact SA device matched ONNX CPU within 4.13e-6 relative L2. The CPU-requested route also has 100 component samples at p50/p95 5.384/5.534 s, with all 738 execution-detail rows on CPU; the GPU-requested path remains numerically failed. Fresh RB3 inference was unsaturated but differed from CPU by 15.73% relative L2 (16.06 dB SNR), without a listening-quality gate; its 100-sample mixed CPU/GPU component p50/p95 was 3.099/3.133 s. X Elite ONNX CPU vocoder inference passed on the same fixture with 8.09e-7 relative L2 versus local ONNX CPU; its 94-sample component p50/p95 was 6.384/6.644 s with all execution-detail rows on CPU. DirectML inference and profile both failed during provider initialization (Windows error 80070057); a separate FP16 QNN NPU vocoder inference is pending. None is a complete TTS stream.
|
||||
A [Qwen3-TTS Code2Wav follow-up](../../../e2e_expansion_20260924/evidence/qwen_tts_vocoder_qualcomm/README.md) recovered source and target artifacts, a retained fixture, and raw historical profiles. On SA8775P, fresh GPU-requested inference saturated 96.63% of waveform samples and differed from original ONNX CPU by 11.832 relative L2; the same binary's CPU-requested output on the exact SA device matched ONNX CPU within 4.13e-6 relative L2. The CPU-requested route also has 100 component samples at p50/p95 5.384/5.534 s, with all 738 execution-detail rows on CPU; the GPU-requested path remains numerically failed. Fresh RB3 inference was unsaturated but differed from CPU by 15.73% relative L2 (16.06 dB SNR), without a listening-quality gate; its 100-sample mixed CPU/GPU component p50/p95 was 3.099/3.133 s. X Elite ONNX CPU vocoder inference passed on the same fixture with 8.09e-7 relative L2 versus local ONNX CPU; its 94-sample component p50/p95 was 6.384/6.644 s with all execution-detail rows on CPU. DirectML inference and profile both failed during provider initialization (Windows error 80070057); a separate FP16 QNN NPU vocoder inference returned an unsaturated waveform at 1.138% relative L2 / 38.88 dB SNR versus local ONNX CPU, with placement profiling pending. None is a complete TTS stream.
|
||||
|
||||
The identical SA/RB3 TFLite binary matched original ONNX CPU within 3.45e-6 relative L2 in a separate local HX370 x86-64 LiteRT CPU control. This narrows the SA waveform saturation to hosted device/runtime behavior, without proving either embedded CPU route or a specific GPU-delegate fault.
|
||||
|
||||
@@ -20,7 +20,7 @@ A separate [Qwen3-TTS talker decode-step attempt](../../../e2e_expansion_2026092
|
||||
|
||||
A [Galaxy S25 InternVLA Cosmos attempt](../../../e2e_expansion_20260924/evidence/internvla_cosmos_qualcomm/README.md) packaged the real Place_Markpen image encoder into one ONNX file after two CPU parity fixtures. FLOAT16 QNN DLC inference returned a latent within 0.0524% relative L2 of the source on one pattern; device placement profile was pending. Injecting that latent into the real policy on local CPU changed synthetic actions by 0.742% relative L2, without a task tolerance. This advances component and action-sensitivity evidence, not device-local full-policy qualification.
|
||||
|
||||
A [MiniCPM-o speech-head study](../../../e2e_expansion_20260924/evidence/minicpmo_tts_qualcomm/README.md) recovered the historical 20-layer source and matching S24/S25 fixtures. Both FP16 context-binary outputs preserved the ONNX CPU top-1 speech token on this one decode step with logits relative L2 of 0.346%/0.360%. New FP16 QNN DLC compiles passed for SA8775P, X Elite and RB3; same-fixture NPU-requested inference was pending. These are speech-head component results, not multimodal MiniCPM-o requests.
|
||||
A [MiniCPM-o speech-head study](../../../e2e_expansion_20260924/evidence/minicpmo_tts_qualcomm/README.md) recovered the historical 20-layer source and matching S24/S25 fixtures. Both FP16 context-binary outputs preserved the ONNX CPU top-1 speech token on this one decode step with logits relative L2 of 0.346%/0.360%. New FP16 QNN DLC compiles passed for SA8775P, X Elite and RB3. SA8775P same-fixture NPU-requested inference returned finite outputs, preserved CPU top-1 token 1867 and had 0.3568% logits relative L2; actual NPU placement profiling is pending. X Elite and RB3 inference remains pending. These are speech-head component results, not multimodal MiniCPM-o requests.
|
||||
|
||||
## Configuration × model
|
||||
|
||||
@@ -87,7 +87,7 @@ NOT E2E means no complete device/model workload was verified. “Scoped E2E” i
|
||||
| PC CPU+iGPU+NPU +/- discrete GPU, joint model execution | MiniCPM-o 4.5 | NOT_E2E_PROFILEABLE | No qualifying same-model multi-accelerator deployment/artifact plan or overlap/recovery evidence. Individual routes are not a joint pipeline. | Supply and validate the missing matching artifact/backend, then run the complete specified workload. |
|
||||
| PC CPU+iGPU+NPU +/- discrete GPU, joint model execution | InternVLA-A1 | NOT_E2E_PROFILEABLE | No qualifying same-model multi-accelerator deployment/artifact plan or overlap/recovery evidence. Individual routes are not a joint pipeline. | Supply and validate the missing matching artifact/backend, then run the complete specified workload. |
|
||||
| PC Snapdragon X Elite CRD / AI Hub | Spark-X2.5 | COMPONENT_NPU_INFERENCE_PROFILED; NOT_E2E_PROFILEABLE | A real Spark-X2.5-1.7B one-layer W8A16 QDQ attention artifact, compiled as QNN DLC, ran on the exact hosted Snapdragon X Elite CRD (Windows 11 ARM64). Inference returned finite hidden/new-K/new-V tensors bitwise identical to the prior S24/S25 NPU outputs on the same fixture; the profile attributed all 73 execution-detail rows to NPU. Across 100 component samples, nearest-rank p50/p95 was 1.433/1.606 ms; the 1.346 ms estimated-time field is the minimum. Hidden/K/V relative L2 versus original FP32 ONNX CPU was 1.351%/0.377%/0.650%, without a next-token quality tolerance. There is no complete 28-layer prefill/decode/KV-ring/sampling loop, resident Omni controller or complete-request timing. | Qualify a complete X Elite resident artifact/controller with state, token quality, memory admission/cancel and sustained power; [X Elite NPU raw evidence](../../../e2e_expansion_20260923/evidence/spark_xelite_full_attention/README.md). |
|
||||
| PC Snapdragon X Elite CRD / AI Hub | Qwen3-TTS 0.6B CustomVoice | COMPONENT_ONNX_CPU_VOCODER_PARITY_PROFILED; DIRECTML_RUNTIME_FAILED; TALKER_FP16_NUMERIC_FAIL; QNN_NPU_VOCODER_COMPILE_PASS; NOT_E2E_PROFILEABLE | The fresh ONNX vocoder inference on the exact Windows 11 ARM64 X Elite CRD requested CPU and returned an unsaturated waveform on the pinned fixture, matching local ONNX Runtime CPU with 8.09e-7 relative L2 and 121.84 dB SNR. Its 94-sample component profile had nearest-rank p50/p95 6.384/6.644 s and all 634 execution-detail rows on CPU; 5.917 s estimated time is the sample minimum. This is one component, without a listening-quality gate. Corrected DirectML inference and profile both failed at provider initialization with Windows error 80070057. A separate FP16 QNN DLC vocoder compile passed, but NPU-requested inference was pending. The 28-layer FP16 QNN talker step executed, yet changed CPU top-1 token 80 to 1720 with 8.57% logits relative L2 on a synthetic cache fixture; a separate ONNX CPU talker compile passed, with inference pending. The older TFLite vocoder target was rejected on Windows ARM; no complete stream exists. | Audit QNN vocoder and CPU talker controls, investigate DirectML failure, then qualify complete local streaming, memory and sustained behavior; [vocoder](../../../e2e_expansion_20260924/evidence/qwen_tts_vocoder_qualcomm/README.md) and [talker](../../../e2e_expansion_20260924/evidence/qwen_tts_talker_qualcomm/README.md) evidence. |
|
||||
| PC Snapdragon X Elite CRD / AI Hub | Qwen3-TTS 0.6B CustomVoice | COMPONENT_ONNX_CPU_VOCODER_PARITY_PROFILED; DIRECTML_RUNTIME_FAILED; TALKER_FP16_NUMERIC_FAIL; QNN_NPU_VOCODER_NUMERIC_MEASURED; PLACEMENT_PENDING; NOT_E2E_PROFILEABLE | The fresh ONNX vocoder inference on the exact Windows 11 ARM64 X Elite CRD requested CPU and returned an unsaturated waveform on the pinned fixture, matching local ONNX Runtime CPU with 8.09e-7 relative L2 and 121.84 dB SNR. Its 94-sample component profile had nearest-rank p50/p95 6.384/6.644 s and all 634 execution-detail rows on CPU; 5.917 s estimated time is the sample minimum. This is one component, without a listening-quality gate. Corrected DirectML inference and profile both failed at provider initialization with Windows error 80070057. A separate FP16 QNN DLC vocoder inference returned an unsaturated waveform at 1.138% relative L2 / 38.88 dB SNR versus local ONNX CPU on the same fixture; NPU node-placement profile is pending and no listening-quality tolerance is set. The 28-layer FP16 QNN talker step executed, yet changed CPU top-1 token 80 to 1720 with 8.57% logits relative L2 on a synthetic cache fixture; a separate ONNX CPU talker compile passed, with inference pending. The older TFLite vocoder target was rejected on Windows ARM; no complete stream exists. | Audit QNN vocoder placement and CPU talker control, investigate DirectML failure, then qualify complete local streaming, memory and sustained behavior; [vocoder](../../../e2e_expansion_20260924/evidence/qwen_tts_vocoder_qualcomm/README.md) and [talker](../../../e2e_expansion_20260924/evidence/qwen_tts_talker_qualcomm/README.md) evidence. |
|
||||
| PC Snapdragon X Elite CRD / AI Hub | Qwen3.8-27B | NOT_E2E_PROFILEABLE | No complete qualified target artifact and device-local stateful Omni backend. Hub jobs provide component execution, not an on-device generation/streaming controller; ADB has no attached device. | Supply and validate the missing matching artifact/backend, then run the complete specified workload. |
|
||||
| PC Snapdragon X Elite CRD / AI Hub | MiniCPM-o 4.5 | COMPONENT_SPEECH_HEAD_QNN_COMPILE_PASS; INFERENCE_PENDING; NOT_E2E_PROFILEABLE | The real 20-layer MiniCPM-o speech-head source compiled to FP16 QNN DLC for the exact X Elite CRD. Same-fixture NPU-requested inference was pending; no numerical output or placement profile exists. Thinker/encoders/vocoder and a device-local stateful Omni backend remain unqualified. | Audit the [X Elite speech-head inference](../../../e2e_expansion_20260924/evidence/minicpmo_tts_qualcomm/README.md), then validate the complete multimodal pipeline, state, quality, memory and sustained behavior. |
|
||||
| PC Snapdragon X Elite CRD / AI Hub | InternVLA-A1 | NOT_E2E_PROFILEABLE | No complete qualified target artifact and device-local stateful Omni backend. Hub jobs provide component execution, not an on-device generation/streaming controller; ADB has no attached device. | Supply and validate the missing matching artifact/backend, then run the complete specified workload. |
|
||||
@@ -104,7 +104,7 @@ NOT E2E means no complete device/model workload was verified. “Scoped E2E” i
|
||||
| Embedded SA8775P ADP / AI Hub | Spark-X2.5 | COMPONENT_NPU_INFERENCE_PROFILED; NOT_E2E_PROFILEABLE | A real Spark-X2.5-1.7B one-layer W8A16 QDQ attention artifact, compiled as QNN DLC, ran on the exact hosted SA8775P ADP (Android 14). Inference returned finite hidden/new-K/new-V tensors bitwise identical to the prior S24/S25 NPU outputs on the same fixture; the profile attributed all 73 execution-detail rows to NPU. Across 100 component samples, nearest-rank p50/p95 was 1.837/2.079 ms; the 1.749 ms estimated-time field is the minimum. Hidden/K/V relative L2 versus original FP32 ONNX CPU was 1.351%/0.377%/0.650%, without a next-token quality tolerance. There is no complete 28-layer prefill/decode/KV-ring/sampling loop, resident Omni controller or complete-request timing. | Qualify a complete embedded resident artifact/controller with state, token quality, memory admission/cancel and sustained power; [SA8775P NPU raw evidence](../../../e2e_expansion_20260923/evidence/spark_sa8775_full_attention/README.md). |
|
||||
| Embedded SA8775P ADP / AI Hub | Qwen3-TTS 0.6B CustomVoice | COMPONENT_CPU_VOCODER_PARITY_AND_PLACEMENT_PASS; QNN_NPU_VOCODER_COMPILE_PASS_INFERENCE_PENDING; COMPONENT_GPU_NUMERIC_FAIL; TALKER_FP16_NUMERIC_FAIL; NOT_E2E_PROFILEABLE | On the exact SA8775P, the same TFLite binary's CPU-requested Code2Wav inference matched original ONNX CPU within 4.13e-6 relative L2 / 107.67 dB SNR, without saturation; Its 100-sample CPU-requested component profile had p50/p95 5.384/5.534 s with all 738 execution-detail rows on CPU; the service estimated-time field 5.250 s is the sample minimum. The GPU-requested output saturated 96.63% of samples with 11.832 relative L2; its fresh 100-sample mixed CPU/GPU profile p50/p95 was 1.776/1.880 s, timing failed audio. Even with FP32-preserving GPU options, 84.89% of samples saturated with 11.389 relative L2. The 28-layer FP16 QNN talker step executed but changed CPU top-1 token 80 to 1720 with 9.43% logits relative L2. A TFLite CPU talker compile passed with same-fixture inference pending. A separate FP16 QNN DLC vocoder compile passed for SA8775P, with same-fixture NPU-requested inference pending. No complete pipeline exists. | Audit the pending SA FP16 QNN vocoder inference and talker CPU control, isolate GPU delegate and talker numerical failures, then establish listening quality, co-resident streaming, memory and sustained behavior; [vocoder](../../../e2e_expansion_20260924/evidence/qwen_tts_vocoder_qualcomm/README.md) and [talker](../../../e2e_expansion_20260924/evidence/qwen_tts_talker_qualcomm/README.md) evidence. |
|
||||
| Embedded SA8775P ADP / AI Hub | Qwen3.8-27B | NOT_E2E_PROFILEABLE | No complete qualified target artifact and device-local stateful Omni backend. Hub jobs provide component execution, not an on-device generation/streaming controller; ADB has no attached device. | Supply and validate the missing matching artifact/backend, then run the complete specified workload. |
|
||||
| Embedded SA8775P ADP / AI Hub | MiniCPM-o 4.5 | COMPONENT_SPEECH_HEAD_QNN_COMPILE_PASS; INFERENCE_PENDING; NOT_E2E_PROFILEABLE | The real 20-layer MiniCPM-o speech-head source compiled to FP16 QNN DLC for the exact SA8775P ADP. Same-fixture NPU-requested inference was pending, so output parity and actual placement are unknown. No complete thinker/encoders/vocoder bundle or device-local stateful controller exists. | Audit the [SA8775P speech-head inference](../../../e2e_expansion_20260924/evidence/minicpmo_tts_qualcomm/README.md), then qualify the complete multimodal pipeline and sustained behavior. |
|
||||
| Embedded SA8775P ADP / AI Hub | MiniCPM-o 4.5 | COMPONENT_SPEECH_HEAD_NUMERIC_MEASURED; PLACEMENT_PENDING; NOT_E2E_PROFILEABLE | The real 20-layer MiniCPM-o speech-head source compiled to FP16 QNN DLC for the exact SA8775P ADP. Same-fixture NPU-requested inference returned 41 finite tensors, preserved ONNX CPU top-1 speech token 1867, and had 0.3568% logits relative L2 and 1.225% maximum K/V relative L2. The requested NPU route awaits actual node-placement and timing profile; no complete thinker/encoders/vocoder bundle or device-local stateful controller exists. | Audit the [SA8775P speech-head placement profile](../../../e2e_expansion_20260924/evidence/minicpmo_tts_qualcomm/README.md), then qualify the complete multimodal pipeline and sustained behavior. |
|
||||
| Embedded SA8775P ADP / AI Hub | InternVLA-A1 | NOT_E2E_PROFILEABLE | No complete qualified target artifact and device-local stateful Omni backend. Hub jobs provide component execution, not an on-device generation/streaming controller; ADB has no attached device. | Supply and validate the missing matching artifact/backend, then run the complete specified workload. |
|
||||
| Embedded RB3 Gen 2 / QCS6490 / AI Hub | Spark-X2.5 | COMPONENT_CPU_PARITY_PASS; COMPONENT_NPU_NUMERIC_UNQUALIFIED; NOT_E2E_PROFILEABLE | On the exact hosted Dragonwing RB3 Gen 2 Vision Kit (QCS6490, QC Linux 1.6), the reused S24 W8A16 QNN DLC and a separately RB3-compiled W8A16 DLC each failed inference/profile at QnnModel_composeGraphsFromDlc: MODEL_GRAPH_ERROR. An FP32 ONNX one-layer CPU route passed with all 56 profile rows on CPU, source-relative hidden/K/V L2 of 8.73e-7/2.98e-7/3.37e-7 and 100-sample component p50/p95 19.304/21.113 ms. A calibrated W8A8 QNN DLC ran with all 61 rows on NPU and p50/p95 5.610/5.761 ms, but hidden/K/V relative L2 was 20.914%/1.702%/4.413% versus FP32 CPU, without a next-token quality gate. These are different precisions, not a qualified speedup; no complete 28-layer prefill/decode/KV-ring/sampling, local Omni controller or complete-request timing exists. | Improve representative W8A8 calibration or find another compatible quality-passing artifact; validate token parity, then full device-resident generation, admission/cancel and sustained power; [RB3 raw evidence](../../../e2e_expansion_20260923/evidence/spark_rb3_full_attention/README.md). |
|
||||
| Embedded RB3 Gen 2 / QCS6490 / AI Hub | Qwen3-TTS 0.6B CustomVoice | COMPONENT_MIXED_VOCODER_PROFILED_NUMERIC_UNQUALIFIED; NOT_E2E_PROFILEABLE | Fresh same-fixture TFLite GPU-requested vocoder inference returned a finite unsaturated waveform, but relative L2 versus original ONNX CPU was 15.73% (16.06 dB SNR), without a listening-quality tolerance; it is not a qualified audio path. The fresh 100-sample component profile had nearest-rank p50/p95 3.099/3.133 s and 403 CPU plus 335 GPU execution-detail rows. Its 3.070 s estimated-time field is the sample minimum, not median. The historical NPU talker build was rejected for floating-point inputs; no full model pipeline exists. | Establish waveform quality and a compatible integer talker artifact, then validate complete local execution, memory and sustained behavior; [vocoder evidence](../../../e2e_expansion_20260924/evidence/qwen_tts_vocoder_qualcomm/README.md). |
|
||||
|
||||
@@ -15,13 +15,13 @@ remain open; the 2026-09-22 audit remains the historical baseline.
|
||||
|
||||
The [Spark hosted Qualcomm component expansion](../../benchmarks/edge_harness/results/e2e_expansion_20260923/evidence/spark_s24_full_attention/README.md) advances M1 only to one real fixed-shape decoder attention layer. The same W8A16 QNN DLC ran on S24, Snapdragon X Elite CRD and SA8775P ADP NPUs with identical outputs on one pinned fixture; S25 had prior separate component evidence. On [RB3 Gen 2](../../benchmarks/edge_harness/results/e2e_expansion_20260923/evidence/spark_rb3_full_attention/README.md), the tested W8A16 graph failed QNN loading even after an exact-device compile. FP32 ONNX ran on CPU with near-source parity, while calibrated W8A8 ran on NPU but had 20.9% hidden-state relative L2 and remains numerically unqualified. These are component C results, not full M1: device-local 28-layer prefill/continuous decode, KV/ring, sampling, token quality, admission/cancellation, complete-request timing and sustained power/thermal gates remain open. AI Hub is the test facility, not a deployment dependency.
|
||||
|
||||
The [Qwen3-TTS Code2Wav Qualcomm follow-up](../../benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen_tts_vocoder_qualcomm/README.md) adds a retained same-fixture numerical check to M2. A fresh SA8775P GPU-requested output saturated 96.63% of samples and differs from original ONNX CPU by 11.832 relative L2, so that route fails the numerical gate despite profiling. The same binary's CPU-requested output on the exact SA device matched original ONNX CPU within 4.13e-6 relative L2; its 100-sample CPU-requested component profile passed placement (738/738 execution-detail rows on CPU) at nearest-rank p50/p95 5.384/5.534 s, still without a full stream. A FP32-preserving GPU-option control still saturated 84.89% of samples, so that option did not fix the tested route. RB3 TFLite GPU-requested inference returned an unsaturated but 15.73%-relative-L2 divergent waveform; its fresh 100-sample mixed CPU/GPU component p50/p95 was 3.099/3.133 s. X Elite ONNX CPU inference matched the local ONNX Runtime output with 8.09e-7 relative L2; its 94-sample component profile was p50/p95 6.384/6.644 s with all execution-detail rows on CPU. Corrected DirectML inference and profile both failed during provider initialization; separate FP16 QNN DLC compilation passed with NPU-requested inference pending. A separate SA8775P FP16 QNN DLC vocoder compile passed, with NPU-requested same-fixture inference pending. M2 still requires a quality-passing talker/predictor/vocoder set, device-local co-residency and handoff, playable streaming and sustained memory/power validation.
|
||||
The [Qwen3-TTS Code2Wav Qualcomm follow-up](../../benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen_tts_vocoder_qualcomm/README.md) adds a retained same-fixture numerical check to M2. A fresh SA8775P GPU-requested output saturated 96.63% of samples and differs from original ONNX CPU by 11.832 relative L2, so that route fails the numerical gate despite profiling. The same binary's CPU-requested output on the exact SA device matched original ONNX CPU within 4.13e-6 relative L2; its 100-sample CPU-requested component profile passed placement (738/738 execution-detail rows on CPU) at nearest-rank p50/p95 5.384/5.534 s, still without a full stream. A FP32-preserving GPU-option control still saturated 84.89% of samples, so that option did not fix the tested route. RB3 TFLite GPU-requested inference returned an unsaturated but 15.73%-relative-L2 divergent waveform; its fresh 100-sample mixed CPU/GPU component p50/p95 was 3.099/3.133 s. X Elite ONNX CPU inference matched the local ONNX Runtime output with 8.09e-7 relative L2; its 94-sample component profile was p50/p95 6.384/6.644 s with all execution-detail rows on CPU. Corrected DirectML inference and profile both failed during provider initialization; separate FP16 QNN DLC inference returned an unsaturated waveform at 1.138% relative L2 / 38.88 dB SNR versus local ONNX CPU, with placement profiling pending. A separate SA8775P FP16 QNN DLC vocoder compile passed, with NPU-requested same-fixture inference pending. M2 still requires a quality-passing talker/predictor/vocoder set, device-local co-residency and handoff, playable streaming and sustained memory/power validation.
|
||||
|
||||
The identical SA/RB3 TFLite vocoder binary also matched original ONNX CPU within 3.45e-6 relative L2 on a separate local HX370 x86-64 LiteRT 2.2.0 CPU control. Exact-SA CPU-requested parity narrows the failure to the GPU-requested route. The FP32-preserving GPU option did not fix it, so per-node placement and delegate-operation isolation remain open.
|
||||
|
||||
A separate [28-layer talker decode-step attempt](../../benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen_tts_talker_qualcomm/README.md) retains a synthetic fixed-cache fixture and CPU reference. Historical FP16 QNN context-binary inference executed on S24, S25, SA8775P and X Elite, but all four changed the CPU top-1 speech token with 8.57–9.43% logits relative L2. RB3 rejected that floating-point-input artifact. Explicit X Elite ONNX CPU and SA8775P TFLite CPU fallback compiles passed, with same-fixture inference pending. None advances M2 to numerical talker or full-stream qualification yet.
|
||||
|
||||
For M4, a [MiniCPM-o 4.5 speech-head study](../../benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/minicpmo_tts_qualcomm/README.md) recovered the historical 20-layer ONNX source and matching S24/S25 fixtures. Both FP16 context-binary outputs preserved the ONNX CPU top-1 speech token on one decode step, with logits relative L2 0.346%/0.360%. New FP16 QNN DLC compiles passed for SA8775P, X Elite and RB3, while same-fixture NPU-requested inference remained pending. This is component C numerical evidence only, not thinker/encoder/vocoder or complete multimodal support.
|
||||
For M4, a [MiniCPM-o 4.5 speech-head study](../../benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/minicpmo_tts_qualcomm/README.md) recovered the historical 20-layer ONNX source and matching S24/S25 fixtures. Both FP16 context-binary outputs preserved the ONNX CPU top-1 speech token on one decode step, with logits relative L2 0.346%/0.360%. New FP16 QNN DLC compiles passed for SA8775P, X Elite and RB3. SA8775P same-fixture NPU-requested inference preserved CPU top-1 speech token 1867 with 0.3568% logits relative L2 and 1.225% maximum K/V relative L2; placement profiling is pending. X Elite and RB3 inferences remain pending. This is component C numerical evidence only, not thinker/encoder/vocoder or complete multimodal support.
|
||||
|
||||
For M4, a [Galaxy S25 Cosmos encoder attempt](../../benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/internvla_cosmos_qualcomm/README.md) repackages the real Place_Markpen FP32 ONNX export after retaining two CPU parity fixtures. FLOAT16 QNN DLC inference returned a latent within 0.0524% relative L2 of the source on one synthetic pattern. Injecting that output into the real policy on local CPU changed synthetic actions by 0.742% relative L2, without a task tolerance. NPU placement profile was pending, and the policy did not run on S25. This is component/action-sensitivity evidence, not device-local M4 qualification.
|
||||
|
||||
|
||||
Reference in new issue
Block a user