Record pinned S25 and X Elite vocoder waveform evidence
This commit is contained in:
tzhouam committed
1 parent
203039e054
commit
4386562f9b
13 files changed
+291
-25
No files matched your search
@@ -31,8 +31,10 @@ ONNX CPU waveform at 1.89e-6 relative L2 on that fixture, without proving
|
||||
the source export's revision.
|
||||
A [fresh 25-frame export](evidence/qwen_tts_vocoder_qualcomm/s25/full_pinned_tflite/README.md)
|
||||
from that pinned snapshot matches local eager CPU at 1.99e-6 relative L2 and
|
||||
has compiled to S25 TFLite. Its GPU-requested inference and placement
|
||||
profile are pending, separate from the historical target's measurements.
|
||||
has compiled to S25 TFLite. Its GPU-requested inference returned an
|
||||
unsaturated waveform at 4.239% relative L2 versus local ONNX CPU, bitwise
|
||||
identical to the historical GPU output on this fixture. Its own placement
|
||||
profile remains pending; the historical timing is separate evidence.
|
||||
|
||||
The same pinned [two-frame graph on Galaxy S24](evidence/qwen_tts_vocoder_qualcomm/s24/tflite_short_chunk/README.md)
|
||||
matched local ONNX CPU at 5.75e-6 relative L2 when requested on CPU, while
|
||||
@@ -42,7 +44,9 @@ its GPU-requested waveform was fully saturated at 24.218 relative L2. The
|
||||
A FP32-preserving GPU control returned the same saturated waveform bitwise.
|
||||
A
|
||||
[Snapdragon X Elite FP16 QNN short graph](evidence/qwen_tts_vocoder_qualcomm/xelite/qnn_short_chunk/README.md)
|
||||
compiled, with NPU inference still pending. These are component-only results.
|
||||
compiled and returned an unsaturated NPU-requested waveform at 0.930%
|
||||
relative L2 versus local ONNX CPU on one fixture. Its placement profile
|
||||
remains pending. These are component-only results.
|
||||
|
||||
The [Qwen3-TTS vocoder study](evidence/qwen_tts_vocoder_qualcomm/README.md) reuses a retained fixed-shape Code2Wav export and historical Workbench input to test numerical behavior on SA8775P and other Qualcomm settings. The fresh SA8775P GPU-requested inference produces a saturated waveform, while the same binary's CPU-requested output on the exact SA device matches ONNX CPU within 4.13e-6 relative L2. Its 100-sample CPU-requested component profile had p50/p95 5.384/5.534 s with all 738 execution-detail rows on CPU. A FP32-preserving GPU-option control still saturated 84.89% of samples, so that option did not repair the tested path. RB3 GPU-requested execution produces an unsaturated but 15.73%-divergent waveform; its fresh 100-sample mixed CPU/GPU component p50/p95 was 3.099/3.133 s, not a usable TTS timing claim. On the exact RB3 device, the same binary requested on CPU matched original ONNX CPU within 4.13e-6 relative L2 on the retained fixture; its 47-sample component profile measured p50/p95 12.727/13.424 s with all 738 execution-detail rows on CPU. This is a major streaming-latency blocker. A single-fixture-calibrated RB3 INT8 QNN vocoder returned a finite but 66.93%-relative-L2 divergent waveform (3.49 dB SNR) versus ONNX CPU; its 100-sample component profile was p50/p95 1.971/1.992 s with all 728 execution-detail rows on NPU; these timings describe numerically unqualified audio. X Elite ONNX CPU output matches the local ONNX Runtime reference on the retained fixture; its 94-sample component p50/p95 was 6.384/6.644 s with all execution-detail rows on CPU. Corrected X Elite DirectML inference and profile both failed during provider initialization; separate FP16 QNN DLC inference returned an unsaturated waveform at 1.138% relative L2 / 38.88 dB SNR against original ONNX CPU; its placement profile timed out during QNN graph preparation without samples. A separate SA8775P FP16 QNN vocoder compile passed, but same-fixture NPU-requested inference failed after compilation without a waveform or placement samples. Fresh S24 and S25 CPU-requested TFLite vocoder inferences matched ONNX CPU within 4.04e-6 relative L2 on the retained fixture. S25 measured 100 component samples at p50/p95 4.418/4.641 s with all 738 execution-detail rows on CPU; S24 measured 100 at 4.826/4.958 s with all 738 rows on CPU. Separate explicit INT8 and W8A16 QDQ ONNX exports checked on local ORT CPU failed the same-fixture numerical gate at 246.97% and 331.83% waveform relative L2, respectively. S25 CPU talker inference preserved token 80 and its 100-sample component profile measured p50/p95 30.161/31.301 ms with all 1,800 rows on CPU. No complete on-device TTS stream or model-quality qualification is claimed.
|
||||
|
||||
|
||||
+16
-1
@@ -10,7 +10,22 @@ matched that historical ONNX CPU waveform at **1.89e-6 relative L2** on the
|
||||
same fixture, strengthening numerical continuity without proving the
|
||||
historical export revision.
|
||||
|
||||
On [Galaxy S24](s24/tflite_short_chunk/README.md), the same pinned two-frame TFLite source matched local ONNX CPU at 5.75e-6 relative L2 when requested on CPU, but its GPU-requested output saturated every sample and differed by 24.218 relative L2. The 100-sample CPU component p50/p95 was 3.433/3.873 s for only 0.16 s audio (median RTF 21.46), with all 738 rows on CPU. The mixed CPU/GPU component measured 0.601/0.895 s with 403 CPU and 335 GPU rows, but that timing describes numerically failed audio. A FP32-preserving delegate control remains pending; neither route is a complete S24 TTS stream.
|
||||
A [revision-pinned 25-frame export](s25/full_pinned_tflite/README.md) from
|
||||
CustomVoice revision `85e237c12c027371202489a0ec509ded67b5e4b5`
|
||||
matched its eager decoder at 1.99e-6 relative L2 on the retained synthetic
|
||||
fixture and compiled for S25 TFLite. Its GPU-requested inference returned a
|
||||
finite unsaturated waveform at 4.239% relative L2 versus local ONNX CPU,
|
||||
bitwise identical to the historical target's device output on the same fixture.
|
||||
Its own placement profile is still running; the historical target's latency
|
||||
is separate evidence.
|
||||
|
||||
The same pinned two-frame source [compiled for X Elite QNN](xelite/qnn_short_chunk/README.md)
|
||||
and its NPU-requested inference returned an unsaturated waveform at 0.930%
|
||||
relative L2 / 40.63 dB SNR versus local ONNX CPU. Its placement profile is
|
||||
still running; this is component numerical evidence, not verified NPU node
|
||||
placement or complete TTS.
|
||||
|
||||
On [Galaxy S24](s24/tflite_short_chunk/README.md), the same pinned two-frame TFLite source matched local ONNX CPU at 5.75e-6 relative L2 when requested on CPU, but its GPU-requested output saturated every sample and differed by 24.218 relative L2. The 100-sample CPU component p50/p95 was 3.433/3.873 s for only 0.16 s audio (median RTF 21.46), with all 738 rows on CPU. The mixed CPU/GPU component measured 0.601/0.895 s with 403 CPU and 335 GPU rows, but that timing describes numerically failed audio. An [FP32-preserving delegate control](s24/tflite_short_chunk_gpu_fp32/audit_report.json) returned the same saturated waveform bitwise, so that option did not fix this artifact. Neither route is a complete S24 TTS stream.
|
||||
|
||||
The historical Workbench source model `mqyer339n` is a 423,655,047-byte ONNX graph with one FP32 input `[1,512,97]` and one 48,000-sample output. The [manifest](artifact_manifest.json) pins its SHA256, the retained Workbench fixture dataset `d7m80jl32`, a local ONNX Runtime 1.29 CPU output, compiled target downloads, and the prior [S25 GPU output](s25_reference_output.npz). The source export script identifies the Qwen3-TTS 0.6B CustomVoice model family, but did not attest the exact checkpoint revision; these results must not be promoted to a revision-qualified release claim. Large source/target artifacts and the fixture remain outside Git under `/home/zhout/project/edge_infer/models/`.
|
||||
|
||||
|
||||
+10
-6
@@ -13,9 +13,13 @@ reproduces this graph with `--chunk-frames 25`.
|
||||
|
||||
The [source upload](upload_submission.json) and [one-sample input dataset](dataset_submission.json)
|
||||
were submitted to Qualcomm AI Hub Workbench. The [S25 TFLite compile](compile_report.json)
|
||||
succeeded; [GPU-requested inference](inference_submission.json) and a separate
|
||||
[placement profile](profile_submission.json) are pending on that target.
|
||||
Device numerical waveform quality, node placement,
|
||||
complete-stream handoff and sustained behavior remain unverified for this
|
||||
revision-pinned artifact. The [historical 25-frame target](../full_tflite_gpu_profile/README.md)
|
||||
is separate despite the local same-fixture numerical continuity.
|
||||
and [GPU-requested inference](inference_report.json) succeeded on that exact
|
||||
target. The [waveform audit](audit_report.json) measured **4.239% relative L2**
|
||||
and **27.46 dB SNR** versus local ONNX Runtime CPU, with no saturated samples.
|
||||
The new device waveform is [bitwise identical](historical_device_parity_report.json)
|
||||
to the [historical 25-frame target's](../full_tflite_gpu_profile/README.md)
|
||||
GPU output on this retained fixture. This supports numerical continuity on
|
||||
one input, not identical historical export provenance or listening quality.
|
||||
A separate [placement profile](profile_submission.json) is in progress; GPU
|
||||
node placement and latency for this revision-pinned target remain unverified.
|
||||
Complete-stream handoff and sustained behavior are also unverified.
|
||||
+78
@@ -0,0 +1,78 @@
|
||||
{
|
||||
"scope": "one 25-frame Qwen3-TTS vocoder component on hosted Samsung Galaxy S25; no complete stream",
|
||||
"status": "component_inference_numeric_measured",
|
||||
"device": {
|
||||
"name": "Samsung Galaxy S25",
|
||||
"os": "15",
|
||||
"attributes": [
|
||||
"os:android",
|
||||
"framework:tflite",
|
||||
"framework:onnx",
|
||||
"abi:aarch64-android",
|
||||
"vendor:samsung",
|
||||
"format:phone",
|
||||
"chipset:qualcomm-snapdragon-8-elite-for-galaxy",
|
||||
"chipset:sm8750-ac",
|
||||
"hexagon:v79",
|
||||
"soc-model:69",
|
||||
"htp-supports-fp16:true",
|
||||
"htp-supports-weight-sharing:true",
|
||||
"framework:qnn"
|
||||
]
|
||||
},
|
||||
"source_model_id": "mng2vkdrm",
|
||||
"target_model_id": "mn4o5j17q",
|
||||
"input_dataset_id": "d2nmpxd57",
|
||||
"jobs": {
|
||||
"compile": "jgddd1zzg",
|
||||
"inference": "jglyymx25"
|
||||
},
|
||||
"route": {
|
||||
"compile_options": "--target_runtime tflite",
|
||||
"run_options": "--compute_unit gpu"
|
||||
},
|
||||
"files": {
|
||||
"fixture": {
|
||||
"bytes": 129721,
|
||||
"sha256": "dfca9e8d2a724a568a31b6cd18ff6a52fdc205fd34e13cbf1cd75f0f7937e82d"
|
||||
},
|
||||
"ort_cpu": {
|
||||
"bytes": 179957,
|
||||
"sha256": "e6c1f321ed752e6c2dec3586e7e80cdf43f18487c25a74acc7eddfd61f4f9f98"
|
||||
},
|
||||
"eager_cpu": {
|
||||
"bytes": 179870,
|
||||
"sha256": "0b4ddaf66d1cb737a2a101db1c1e7fa5871e4a73164a607afe029197a854058c"
|
||||
},
|
||||
"device_output": {
|
||||
"bytes": 115235,
|
||||
"sha256": "4a26486d5e84a6e22d16f9b26f995b8fe645c29bbd70b103615ec35a0fcb10a0"
|
||||
}
|
||||
},
|
||||
"ort_cpu_vs_device": {
|
||||
"relative_l2": 0.042385301235348234,
|
||||
"snr_db": 27.45569451867911,
|
||||
"max_abs": 0.10591868683695793,
|
||||
"reference_rms": 0.0837343916262634,
|
||||
"output_rms": 0.08376787228276501,
|
||||
"output_saturated_fraction_abs_ge_0_999": 0.0
|
||||
},
|
||||
"eager_cpu_vs_device": {
|
||||
"relative_l2": 0.04238528582586735,
|
||||
"snr_db": 27.455697676497284,
|
||||
"max_abs": 0.10591860488057137,
|
||||
"reference_rms": 0.083734386044337,
|
||||
"output_rms": 0.08376787228276501,
|
||||
"output_saturated_fraction_abs_ge_0_999": 0.0
|
||||
},
|
||||
"gross_numeric_failure_screen": {
|
||||
"failed": false,
|
||||
"criteria": "relative L2 >= 1.0 or fraction |sample| >= 0.999 at least 0.1; this is not a speech-quality acceptance threshold"
|
||||
},
|
||||
"limits": [
|
||||
"One fixed-window waveform has no listening-quality tolerance.",
|
||||
"The 72-frame input history, talker, predictor and stage handoff did not run on the device.",
|
||||
"No persistent state, admission, cancellation or complete TTS stream was tested.",
|
||||
"Requested accelerator execution is not verified node placement without a profile."
|
||||
]
|
||||
}
|
||||
BIN
Binary file not shown.
+16
@@ -0,0 +1,16 @@
|
||||
{
|
||||
"scope": "one retained synthetic 25-frame fixture; historical and revision-pinned S25 GPU-requested TFLite outputs",
|
||||
"new_output_sha256": "4a26486d5e84a6e22d16f9b26f995b8fe645c29bbd70b103615ec35a0fcb10a0",
|
||||
"historical_output_sha256": "4a26486d5e84a6e22d16f9b26f995b8fe645c29bbd70b103615ec35a0fcb10a0",
|
||||
"output_shape": [
|
||||
1,
|
||||
48000
|
||||
],
|
||||
"dtype": "float32",
|
||||
"bitwise_equal": true,
|
||||
"maximum_absolute_difference": 0.0,
|
||||
"limits": [
|
||||
"Bitwise same output on one fixture does not prove identical graph provenance for historical export.",
|
||||
"No listening-quality gate or complete stream is established."
|
||||
]
|
||||
}
|
||||
+33
@@ -0,0 +1,33 @@
|
||||
{
|
||||
"timestamp_utc": "2026-09-24T06:34:28.494002+00:00",
|
||||
"qai_hub_version": "0.55.0",
|
||||
"facility": "Qualcomm AI Hub Workbench",
|
||||
"job_id": "jglyymx25",
|
||||
"url": "https://workbench.aihub.qualcomm.com/jobs/jglyymx25/",
|
||||
"status": "SUCCESS",
|
||||
"status_message": "",
|
||||
"options": "--compute_unit gpu",
|
||||
"hub_version": "aihub-2026.09.11.0",
|
||||
"device": {
|
||||
"name": "Samsung Galaxy S25",
|
||||
"os": "15",
|
||||
"attributes": [
|
||||
"os:android",
|
||||
"framework:tflite",
|
||||
"framework:onnx",
|
||||
"abi:aarch64-android",
|
||||
"vendor:samsung",
|
||||
"format:phone",
|
||||
"chipset:qualcomm-snapdragon-8-elite-for-galaxy",
|
||||
"chipset:sm8750-ac",
|
||||
"hexagon:v79",
|
||||
"soc-model:69",
|
||||
"htp-supports-fp16:true",
|
||||
"htp-supports-weight-sharing:true",
|
||||
"framework:qnn"
|
||||
]
|
||||
},
|
||||
"model_id": "mn4o5j17q",
|
||||
"input_dataset_id": "d2nmpxd57",
|
||||
"outputs": "device_output.npz"
|
||||
}
|
||||
+9
-6
@@ -2,11 +2,14 @@
|
||||
|
||||
The same pinned [Qwen3-TTS two-frame source and fixture](../../s25/qnn_short_chunk/README.md)
|
||||
compiled to [FP16 QNN DLC](compile_report.json) for Snapdragon X Elite CRD.
|
||||
An [NPU-requested inference job](inference_submission.json) is pending on
|
||||
the retained fixture. A separate [placement profile](profile_submission.json)
|
||||
is also pending for the same target and tensor shape. Compilation alone does
|
||||
not establish numerical output, NPU placement, timing, or complete TTS
|
||||
support. The earlier
|
||||
[NPU-requested inference](inference_report.json) succeeded on the retained
|
||||
fixture. The [audit](audit_report.json) measures **0.930% relative L2** and
|
||||
**40.63 dB SNR** against local ONNX Runtime CPU, with no saturated samples.
|
||||
This is one synthetic, two-frame waveform without a listening-quality gate.
|
||||
A separate [placement profile](profile_submission.json) remains in progress
|
||||
for the same target and tensor shape; the request alone does not verify NPU
|
||||
node placement or latency. The earlier
|
||||
[25-frame X Elite NPU attempt](../qnn_npu/inference_report.json) returned a
|
||||
waveform but its placement profile timed out during graph preparation; this
|
||||
shorter artifact tests a separate memory/graph-preparation boundary.
|
||||
shorter artifact tests a separate memory/graph-preparation boundary. Neither
|
||||
artifact establishes complete TTS support.
|
||||
+77
@@ -0,0 +1,77 @@
|
||||
{
|
||||
"scope": "one 2-frame Qwen3-TTS vocoder component on hosted Snapdragon X Elite CRD; no complete stream",
|
||||
"status": "component_inference_numeric_measured",
|
||||
"device": {
|
||||
"name": "Snapdragon X Elite CRD",
|
||||
"os": "11",
|
||||
"attributes": [
|
||||
"os:windows",
|
||||
"format:compute",
|
||||
"framework:onnx",
|
||||
"vendor:qualcomm",
|
||||
"abi:aarch64-windows",
|
||||
"chipset:qualcomm-snapdragon-x-elite",
|
||||
"chipset:sc8380xp",
|
||||
"hexagon:v73",
|
||||
"soc-model:60",
|
||||
"htp-supports-fp16:true",
|
||||
"htp-supports-weight-sharing:true",
|
||||
"framework:qnn"
|
||||
]
|
||||
},
|
||||
"source_model_id": "mng2vv4em",
|
||||
"target_model_id": "mqeyv2w7m",
|
||||
"input_dataset_id": "d7ovqp0w2",
|
||||
"jobs": {
|
||||
"compile": "j5m0m22qg",
|
||||
"inference": "jgdd3lxeg"
|
||||
},
|
||||
"route": {
|
||||
"compile_options": "--target_runtime qnn_dlc --qnn_options default_graph_htp_precision=FLOAT16",
|
||||
"run_options": "--compute_unit npu"
|
||||
},
|
||||
"files": {
|
||||
"fixture": {
|
||||
"bytes": 129614,
|
||||
"sha256": "77aea3554c51b9daf808748257ebfc5e0496172c4d34c8267288be984d8a0f55"
|
||||
},
|
||||
"ort_cpu": {
|
||||
"bytes": 14813,
|
||||
"sha256": "989307db317d77be054f684211ae0c6a2fd24d93bcd74307c597fed430ba2a37"
|
||||
},
|
||||
"eager_cpu": {
|
||||
"bytes": 14818,
|
||||
"sha256": "542113d24feeb4ff58b895ce7d96fc7aa5240b046a3a0d1d3922c7ab23ea0a7c"
|
||||
},
|
||||
"device_output": {
|
||||
"bytes": 3639,
|
||||
"sha256": "2f9c5f18535f85ecd016de8d5635bf9deb6c3ff1282de0ebc72af8ad9d7bfe8c"
|
||||
}
|
||||
},
|
||||
"ort_cpu_vs_device": {
|
||||
"relative_l2": 0.009296003183799334,
|
||||
"snr_db": 40.63407472423357,
|
||||
"max_abs": 0.002414429560303688,
|
||||
"reference_rms": 0.04131374650145444,
|
||||
"output_rms": 0.04128884867187232,
|
||||
"output_saturated_fraction_abs_ge_0_999": 0.0
|
||||
},
|
||||
"eager_cpu_vs_device": {
|
||||
"relative_l2": 0.009295756129736288,
|
||||
"snr_db": 40.63430556675796,
|
||||
"max_abs": 0.0024141762405633926,
|
||||
"reference_rms": 0.04131376905845152,
|
||||
"output_rms": 0.04128884867187232,
|
||||
"output_saturated_fraction_abs_ge_0_999": 0.0
|
||||
},
|
||||
"gross_numeric_failure_screen": {
|
||||
"failed": false,
|
||||
"criteria": "relative L2 >= 1.0 or fraction |sample| >= 0.999 at least 0.1; this is not a speech-quality acceptance threshold"
|
||||
},
|
||||
"limits": [
|
||||
"One fixed-window waveform has no listening-quality tolerance.",
|
||||
"The 72-frame input history, talker, predictor and stage handoff did not run on the device.",
|
||||
"No persistent state, admission, cancellation or complete TTS stream was tested.",
|
||||
"Requested accelerator execution is not verified node placement without a profile."
|
||||
]
|
||||
}
|
||||
BIN
Binary file not shown.
+32
@@ -0,0 +1,32 @@
|
||||
{
|
||||
"timestamp_utc": "2026-09-24T06:30:20.902478+00:00",
|
||||
"qai_hub_version": "0.55.0",
|
||||
"facility": "Qualcomm AI Hub Workbench",
|
||||
"job_id": "jgdd3lxeg",
|
||||
"url": "https://workbench.aihub.qualcomm.com/jobs/jgdd3lxeg/",
|
||||
"status": "SUCCESS",
|
||||
"status_message": "",
|
||||
"options": "--compute_unit npu",
|
||||
"hub_version": "aihub-2026.09.11.0",
|
||||
"device": {
|
||||
"name": "Snapdragon X Elite CRD",
|
||||
"os": "11",
|
||||
"attributes": [
|
||||
"os:windows",
|
||||
"format:compute",
|
||||
"framework:onnx",
|
||||
"vendor:qualcomm",
|
||||
"abi:aarch64-windows",
|
||||
"chipset:qualcomm-snapdragon-x-elite",
|
||||
"chipset:sc8380xp",
|
||||
"hexagon:v73",
|
||||
"soc-model:60",
|
||||
"htp-supports-fp16:true",
|
||||
"htp-supports-weight-sharing:true",
|
||||
"framework:qnn"
|
||||
]
|
||||
},
|
||||
"model_id": "mqeyv2w7m",
|
||||
"input_dataset_id": "d7ovqp0w2",
|
||||
"outputs": "device_output.npz"
|
||||
}
|
||||
@@ -49,8 +49,8 @@ NOT E2E means no complete device/model workload was verified. “Scoped E2E” i
|
||||
| PC HX370 + Radeon 890M / Windows worker | scoped E2E: Omni Spark text | scoped E2E: Omni Radeon talker/codec + CPU predictor | NOT E2E | scoped E2E: Omni Radeon+CPU image+audio → text+speech (native-storage 20-run profile) | synthetic policy pass: Omni CPU+Radeon Cosmos (WSL/native Windows) |
|
||||
| PC HX370 + AMD NPU / Windows worker | NOT E2E (output-head component passed) | NOT E2E | NOT E2E | NOT E2E | NOT E2E |
|
||||
| PC CPU+iGPU+NPU +/- discrete GPU, joint model execution | NOT E2E | NOT E2E | NOT E2E | NOT E2E | NOT E2E |
|
||||
| PC Snapdragon X Elite CRD / AI Hub | NOT E2E (one Spark attention layer on NPU) | NOT E2E (CPU talker/vocoder steps; QNN vocoder numeric) | NOT E2E | NOT E2E (NPU speech-head step profiled) | NOT E2E (Cosmos NPU encoder profiled) |
|
||||
| Mobile Galaxy S25 / Snapdragon 8 Elite for Galaxy | NOT E2E (one Spark attention layer on NPU) | NOT E2E (FP16 talker token mismatch; CPU talker + vocoder profiled; 25-frame and two-frame QNN vocoder memory failures; 25-frame TFLite CPU/GPU vocoder component RTF 0.420; short chunk RTF 3.49) | NOT E2E | NOT E2E (NPU speech head step profiled) | NOT E2E (Cosmos NPU encoder profiled) |
|
||||
| PC Snapdragon X Elite CRD / AI Hub | NOT E2E (one Spark attention layer on NPU) | NOT E2E (CPU talker/vocoder steps; two-frame QNN vocoder waveform measured, placement pending) | NOT E2E | NOT E2E (NPU speech-head step profiled) | NOT E2E (Cosmos NPU encoder profiled) |
|
||||
| Mobile Galaxy S25 / Snapdragon 8 Elite for Galaxy | NOT E2E (one Spark attention layer on NPU) | NOT E2E (FP16 talker token mismatch; CPU talker + vocoder profiled; 25-frame and two-frame QNN vocoder memory failures; historical 25-frame TFLite mixed vocoder component RTF 0.420; pinned 25-frame GPU waveform measured, placement pending; short chunk RTF 3.49) | NOT E2E | NOT E2E (NPU speech head step profiled) | NOT E2E (Cosmos NPU encoder profiled) |
|
||||
| Mobile Galaxy S24 / Snapdragon 8 Gen 3 | NOT E2E (one Spark attention layer on NPU) | NOT E2E (FP16 talker mismatch; short CPU vocoder RTF 21.46; mixed CPU/GPU waveform saturated) | NOT E2E | NOT E2E (NPU speech head step profiled) | NOT E2E (Cosmos encoder numeric and NPU profiled) |
|
||||
| Embedded SA8775P ADP / AI Hub | NOT E2E (one Spark attention layer on NPU) | NOT E2E (CPU talker/vocoder steps; NPU vocoder failed) | NOT E2E | NOT E2E (NPU speech-head step profiled) | NOT E2E (Cosmos NPU encoder profiled) |
|
||||
| Embedded RB3 Gen 2 / QCS6490 / AI Hub | NOT E2E (Spark CPU layer passed; W8A8 NPU quality open) | NOT E2E (CPU talker + vocoder profiled; INT8 NPU vocoder fails numeric gate) | NOT E2E | NOT E2E (CPU speech-head step profiled) | NOT E2E (Cosmos CPU numeric and profiled) |
|
||||
@@ -95,17 +95,17 @@ NOT E2E means no complete device/model workload was verified. “Scoped E2E” i
|
||||
| PC CPU+iGPU+NPU +/- discrete GPU, joint model execution | MiniCPM-o 4.5 | NOT_E2E_PROFILEABLE | No qualifying same-model multi-accelerator deployment/artifact plan or overlap/recovery evidence. Individual routes are not a joint pipeline. | Supply and validate the missing matching artifact/backend, then run the complete specified workload. |
|
||||
| PC CPU+iGPU+NPU +/- discrete GPU, joint model execution | InternVLA-A1 | NOT_E2E_PROFILEABLE | No qualifying same-model multi-accelerator deployment/artifact plan or overlap/recovery evidence. Individual routes are not a joint pipeline. | Supply and validate the missing matching artifact/backend, then run the complete specified workload. |
|
||||
| PC Snapdragon X Elite CRD / AI Hub | Spark-X2.5 | COMPONENT_NPU_INFERENCE_PROFILED; NOT_E2E_PROFILEABLE | A real Spark-X2.5-1.7B one-layer W8A16 QDQ attention artifact, compiled as QNN DLC, ran on the exact hosted Snapdragon X Elite CRD (Windows 11 ARM64). Inference returned finite hidden/new-K/new-V tensors bitwise identical to the prior S24/S25 NPU outputs on the same fixture; the profile attributed all 73 execution-detail rows to NPU. Across 100 component samples, nearest-rank p50/p95 was 1.433/1.606 ms; the 1.346 ms estimated-time field is the minimum. Hidden/K/V relative L2 versus original FP32 ONNX CPU was 1.351%/0.377%/0.650%, without a next-token quality tolerance. There is no complete 28-layer prefill/decode/KV-ring/sampling loop, resident Omni controller or complete-request timing. | Qualify a complete X Elite resident artifact/controller with state, token quality, memory admission/cancel and sustained power; [X Elite NPU raw evidence](../../../e2e_expansion_20260923/evidence/spark_xelite_full_attention/README.md). |
|
||||
| PC Snapdragon X Elite CRD / AI Hub | Qwen3-TTS 0.6B CustomVoice | COMPONENT_ONNX_CPU_VOCODER_PARITY_PROFILED; DIRECTML_RUNTIME_FAILED; TALKER_FP16_NUMERIC_FAIL; QNN_NPU_VOCODER_NUMERIC_MEASURED_PROFILE_TIMEOUT; TALKER_ONNX_CPU_NUMERIC_PROFILED; NOT_E2E_PROFILEABLE | The fresh ONNX vocoder inference on the exact Windows 11 ARM64 X Elite CRD requested CPU and returned an unsaturated waveform on the pinned fixture, matching local ONNX Runtime CPU with 8.09e-7 relative L2 and 121.84 dB SNR. Its 94-sample component profile had nearest-rank p50/p95 6.384/6.644 s and all 634 execution-detail rows on CPU; 5.917 s estimated time is the sample minimum. This is one component, without a listening-quality gate. Corrected DirectML inference and profile both failed at provider initialization with Windows error 80070057. A separate FP16 QNN DLC vocoder inference returned an unsaturated waveform at 1.138% relative L2 / 38.88 dB SNR versus local ONNX CPU on the same fixture; the NPU node-placement profile timed out during graph preparation with no timing or per-node samples, and no listening-quality tolerance is set. The 28-layer FP16 QNN talker step executed, yet changed CPU top-1 token 80 to 1720 with 8.57% logits relative L2 on a synthetic cache fixture; a separate ONNX CPU-requested talker step preserved CPU token 80 with logits relative L2 2.07e-5 and maximum K/V relative L2 2.18e-5. Its 100-sample component profile had p50/p95 191.414/309.428 ms and all 1,744 execution-detail rows on CPU. The older TFLite vocoder target was rejected on Windows ARM; no complete stream exists. | Reduce or partition the QNN vocoder profile graph and audit CPU talker placement; investigate DirectML failure, then qualify complete local streaming, memory and sustained behavior; [vocoder](../../../e2e_expansion_20260924/evidence/qwen_tts_vocoder_qualcomm/README.md) and [talker](../../../e2e_expansion_20260924/evidence/qwen_tts_talker_qualcomm/README.md) evidence. |
|
||||
| PC Snapdragon X Elite CRD / AI Hub | Qwen3-TTS 0.6B CustomVoice | COMPONENT_ONNX_CPU_VOCODER_PARITY_PROFILED; DIRECTML_RUNTIME_FAILED; TALKER_FP16_NUMERIC_FAIL; QNN_NPU_VOCODER_NUMERIC_MEASURED_PROFILE_TIMEOUT; SHORT_QNN_VOCODER_NUMERIC_MEASURED_PLACEMENT_PENDING; TALKER_ONNX_CPU_NUMERIC_PROFILED; NOT_E2E_PROFILEABLE | The fresh ONNX vocoder inference on the exact Windows 11 ARM64 X Elite CRD requested CPU and returned an unsaturated waveform on the pinned fixture, matching local ONNX Runtime CPU with 8.09e-7 relative L2 and 121.84 dB SNR. Its 94-sample component profile had nearest-rank p50/p95 6.384/6.644 s and all 634 execution-detail rows on CPU; 5.917 s estimated time is the sample minimum. This is one component, without a listening-quality gate. Corrected DirectML inference and profile both failed at provider initialization with Windows error 80070057. A separate FP16 QNN DLC vocoder inference returned an unsaturated waveform at 1.138% relative L2 / 38.88 dB SNR versus local ONNX CPU on the same fixture; the NPU node-placement profile timed out during graph preparation with no timing or per-node samples, and no listening-quality tolerance is set. A new [two-frame QNN vocoder candidate](../../../e2e_expansion_20260924/evidence/qwen_tts_vocoder_qualcomm/xelite/qnn_short_chunk/README.md) compiled for X Elite and returned an unsaturated waveform at 0.930% relative L2 / 40.63 dB SNR versus pinned local ONNX CPU on one synthetic fixture. Its NPU-requested placement profile remains in progress; inference request alone does not verify NPU nodes or latency. The 28-layer FP16 QNN talker step executed, yet changed CPU top-1 token 80 to 1720 with 8.57% logits relative L2 on a synthetic cache fixture; a separate ONNX CPU-requested talker step preserved CPU token 80 with logits relative L2 2.07e-5 and maximum K/V relative L2 2.18e-5. Its 100-sample component profile had p50/p95 191.414/309.428 ms and all 1,744 execution-detail rows on CPU. The older TFLite vocoder target was rejected on Windows ARM; no complete stream exists. | Complete the short QNN vocoder placement profile and audit CPU talker placement; investigate DirectML failure, then qualify complete local streaming, memory and sustained behavior; [vocoder](../../../e2e_expansion_20260924/evidence/qwen_tts_vocoder_qualcomm/README.md) and [talker](../../../e2e_expansion_20260924/evidence/qwen_tts_talker_qualcomm/README.md) evidence. |
|
||||
| PC Snapdragon X Elite CRD / AI Hub | Qwen3.8-27B | NOT_E2E_PROFILEABLE | No complete qualified target artifact and device-local stateful Omni backend. Hub jobs provide component execution, not an on-device generation/streaming controller; ADB has no attached device. | Supply and validate the missing matching artifact/backend, then run the complete specified workload. |
|
||||
| PC Snapdragon X Elite CRD / AI Hub | MiniCPM-o 4.5 | COMPONENT_SPEECH_HEAD_NPU_NUMERIC_PROFILED; NOT_E2E_PROFILEABLE | The real 20-layer MiniCPM-o speech-head FP16 QNN DLC ran on the exact X Elite CRD. Same-fixture inference returned 41 finite tensors, preserved ONNX CPU top-1 speech token 1867, and had 0.3170% logits relative L2 and 1.221% maximum K/V relative L2. Its 100-sample component profile had nearest-rank p50/p95 9.905/11.037 ms, with all 944 execution-detail rows on NPU. Reported inference peak memory was 79,208,448 bytes and first-load peak 1,081,573,376 bytes. Thinker/encoders/vocoder and a device-local stateful Omni backend remain unqualified. | Validate the complete multimodal pipeline, state, quality, co-resident memory and sustained behavior; [X Elite raw evidence](../../../e2e_expansion_20260924/evidence/minicpmo_tts_qualcomm/README.md). |
|
||||
| PC Snapdragon X Elite CRD / AI Hub | InternVLA-A1 | COMPONENT_COSMOS_NPU_NUMERIC_PROFILED; NOT_E2E_PROFILEABLE | The real Place_Markpen Cosmos FP16 QNN image encoder returned a finite FP32 latent for the six-frame synthetic pattern on exact X Elite CRD. Against the pinned source, relative L2 was 0.0608%, SNR 64.32 dB and cosine 0.999999815. The 100-sample six-frame encoder profile had p50/p95 82.939/83.837 ms, with all 121 execution-detail rows on NPU. Reported component inference peak memory was 24,059,904 bytes and first-load peak 692,891,648 bytes. No real-observation action tolerance, complete policy or local stateful Omni backend exists. | Validate real observations, action units/timing, state, admission, cancellation and sustained behavior; [X Elite Cosmos raw evidence](../../../e2e_expansion_20260924/evidence/internvla_cosmos_qualcomm/README.md). |
|
||||
| Mobile Galaxy S25 / Snapdragon 8 Elite for Galaxy | Spark-X2.5 | COMPONENT_NPU_INFERENCE_PROFILED; NOT_E2E_PROFILEABLE | The real 1.7B checkpoint one-layer W8A16 full-attention decoder artifact executed for a fixed 1,024-token cache bucket. Same-fixture outputs matched S24; the hosted S25 profile had 100 component samples at nearest-rank p50/p95 0.957/1.028 ms with all 73 execution-detail rows on NPU. FP32-source hidden-state relative L2 was 1.351% without a downstream token-quality gate. No full 28-layer prefill/decode, KV/ring, sampling or resident controller ran on S25. | Implement and validate local 128-token generation, 512/1024 transitions, cancellation, resident memory and sustained power; [S25/S24 component evidence](../../../e2e_expansion_20260923/evidence/spark_s24_full_attention/README.md). |
|
||||
| Mobile Galaxy S25 / Snapdragon 8 Elite for Galaxy | Qwen3-TTS 0.6B CustomVoice | TALKER_FP16_NUMERIC_FAIL; CPU_TALKER_AND_VOCODER_NUMERIC_PROFILED; QNN_VOCODER_MEMORY_FAIL; FULL_TFLITE_MIXED_COMPONENT_PROFILED; SHORT_TFLITE_MIXED_COMPONENT_PROFILED; NOT_E2E_PROFILEABLE | Fresh FP16 predictor inference passed; its one output exactly matches the same compiled artifact/dataset reference. The same historical S25 TFLite vocoder target requested on CPU returned a finite unsaturated waveform at 4.04e-6 relative L2 / 107.86 dB SNR versus original ONNX CPU on the retained fixture. Its 100-sample component profile measured nearest-rank p50/p95 4.418/4.641 s, with all 738 execution-detail rows on CPU and reported inference peak memory 867,893,248 bytes. A separate 28-layer TFLite CPU-requested talker step returned 57 finite outputs, preserved ONNX CPU top-1 speech token 80, and had logits relative L2 3.33e-5 and maximum new-cache relative L2 3.41e-5. Its 100-sample component profile measured p50/p95 30.161/31.301 ms, with all 1,800 execution-detail rows on CPU and reported inference peak memory 1,929,318,400 bytes. The FP16 QNN talker step executed on the synthetic cache fixture but changed CPU top-1 speech token 80 to 412 with 8.99% logits relative L2 and maximum new-K/V relative L2 11.71%. A separately compiled FP16 QNN DLC talker also changed top-1 to 1720 with 8.77% logits relative L2 and 11.27% maximum new-cache relative L2. Both tested NPU talker artifacts are numerically unqualified. A fresh 25-frame FP16 QNN DLC vocoder compiled for S25, but its NPU-requested inference failed when Workbench reported device memory limits; it returned no waveform or profile samples. A [two-frame short-window export](../../../e2e_expansion_20260924/evidence/qwen_tts_vocoder_qualcomm/s25/qnn_short_chunk/README.md) matched local ONNX CPU at 2.96e-6 relative L2, then its separately compiled S25 FP16 QNN DLC also failed NPU-requested inference with the same reported device-memory limit. Its local HX370 WSL CPU component p50/p95 was 1.365/1.453 s to produce 0.16 s audio, not an S25 estimate. A separate [two-frame TFLite GPU-requested inference](../../../e2e_expansion_20260924/evidence/qwen_tts_vocoder_qualcomm/s25/tflite_short_chunk/README.md) returned finite unsaturated audio at 2.325% relative L2 / 32.67 dB SNR versus local ONNX CPU on one fixture; its 100-sample placement profile measured p50/p95 0.559/1.052 s for 0.16 s audio, with 403 CPU and 335 GPU execution-detail rows and 379,121,664 bytes reported peak. The mixed CPU/GPU short component median RTF is 3.49. A separate [25-frame TFLite GPU-requested profile](../../../e2e_expansion_20260924/evidence/qwen_tts_vocoder_qualcomm/s25/full_tflite_gpu_profile/README.md) of the historical target measured 100 component samples at p50/p95 0.839/1.157 s for 2 s audio, with 403 CPU and 335 GPU rows, reported 374,591,488-byte peak and median component RTF 0.420. Its retained GPU waveform differed from ONNX CPU by 4.24% relative L2 on one fixture. A pinned CustomVoice eager 25-frame decoder matched the historical ONNX CPU waveform at 1.89e-6 relative L2 on that fixture, but the historical export revision and listening quality remain unverified. A fresh [revision-pinned 25-frame TFLite export](../../../e2e_expansion_20260924/evidence/qwen_tts_vocoder_qualcomm/s25/full_pinned_tflite/README.md) matched local eager CPU at 1.99e-6 relative L2 and compiled for S25; its GPU inference and placement profile are pending. No complete co-resident NPU/GPU stream exists. | Improve and revalidate the [talker artifact](../../../e2e_expansion_20260924/evidence/qwen_tts_talker_qualcomm/README.md), then qualify the [device-local predictor/vocoder pipeline](../../../e2e_expansion_20260924/evidence/qwen_tts_vocoder_qualcomm/README.md), output quality, recovery and 30-minute thermal behavior. |
|
||||
| Mobile Galaxy S25 / Snapdragon 8 Elite for Galaxy | Qwen3-TTS 0.6B CustomVoice | TALKER_FP16_NUMERIC_FAIL; CPU_TALKER_AND_VOCODER_NUMERIC_PROFILED; QNN_VOCODER_MEMORY_FAIL; FULL_TFLITE_MIXED_COMPONENT_PROFILED; PINNED_FULL_TFLITE_GPU_NUMERIC_MEASURED_PLACEMENT_PENDING; SHORT_TFLITE_MIXED_COMPONENT_PROFILED; NOT_E2E_PROFILEABLE | Fresh FP16 predictor inference passed; its one output exactly matches the same compiled artifact/dataset reference. The same historical S25 TFLite vocoder target requested on CPU returned a finite unsaturated waveform at 4.04e-6 relative L2 / 107.86 dB SNR versus original ONNX CPU on the retained fixture. Its 100-sample component profile measured nearest-rank p50/p95 4.418/4.641 s, with all 738 execution-detail rows on CPU and reported inference peak memory 867,893,248 bytes. A separate 28-layer TFLite CPU-requested talker step returned 57 finite outputs, preserved ONNX CPU top-1 speech token 80, and had logits relative L2 3.33e-5 and maximum new-cache relative L2 3.41e-5. Its 100-sample component profile measured p50/p95 30.161/31.301 ms, with all 1,800 execution-detail rows on CPU and reported inference peak memory 1,929,318,400 bytes. The FP16 QNN talker step executed on the synthetic cache fixture but changed CPU top-1 speech token 80 to 412 with 8.99% logits relative L2 and maximum new-K/V relative L2 11.71%. A separately compiled FP16 QNN DLC talker also changed top-1 to 1720 with 8.77% logits relative L2 and 11.27% maximum new-cache relative L2. Both tested NPU talker artifacts are numerically unqualified. A fresh 25-frame FP16 QNN DLC vocoder compiled for S25, but its NPU-requested inference failed when Workbench reported device memory limits; it returned no waveform or profile samples. A [two-frame short-window export](../../../e2e_expansion_20260924/evidence/qwen_tts_vocoder_qualcomm/s25/qnn_short_chunk/README.md) matched local ONNX CPU at 2.96e-6 relative L2, then its separately compiled S25 FP16 QNN DLC also failed NPU-requested inference with the same reported device-memory limit. Its local HX370 WSL CPU component p50/p95 was 1.365/1.453 s to produce 0.16 s audio, not an S25 estimate. A separate [two-frame TFLite GPU-requested inference](../../../e2e_expansion_20260924/evidence/qwen_tts_vocoder_qualcomm/s25/tflite_short_chunk/README.md) returned finite unsaturated audio at 2.325% relative L2 / 32.67 dB SNR versus local ONNX CPU on one fixture; its 100-sample placement profile measured p50/p95 0.559/1.052 s for 0.16 s audio, with 403 CPU and 335 GPU execution-detail rows and 379,121,664 bytes reported peak. The mixed CPU/GPU short component median RTF is 3.49. A separate [25-frame TFLite GPU-requested profile](../../../e2e_expansion_20260924/evidence/qwen_tts_vocoder_qualcomm/s25/full_tflite_gpu_profile/README.md) of the historical target measured 100 component samples at p50/p95 0.839/1.157 s for 2 s audio, with 403 CPU and 335 GPU rows, reported 374,591,488-byte peak and median component RTF 0.420. Its retained GPU waveform differed from ONNX CPU by 4.24% relative L2 on one fixture. A pinned CustomVoice eager 25-frame decoder matched the historical ONNX CPU waveform at 1.89e-6 relative L2 on that fixture, but the historical export revision and listening quality remain unverified. A fresh [revision-pinned 25-frame TFLite export](../../../e2e_expansion_20260924/evidence/qwen_tts_vocoder_qualcomm/s25/full_pinned_tflite/README.md) matched local eager CPU at 1.99e-6 relative L2 and compiled for S25. Its GPU-requested inference returned an unsaturated waveform at 4.239% relative L2 / 27.46 dB SNR versus pinned local ONNX CPU, bitwise identical to the historical GPU output on this one fixture. Its own placement profile remains pending; the historical target's timing does not transfer automatically. No complete co-resident NPU/GPU stream exists. | Improve and revalidate the [talker artifact](../../../e2e_expansion_20260924/evidence/qwen_tts_talker_qualcomm/README.md), then qualify the [device-local predictor/vocoder pipeline](../../../e2e_expansion_20260924/evidence/qwen_tts_vocoder_qualcomm/README.md), output quality, recovery and 30-minute thermal behavior. |
|
||||
| Mobile Galaxy S25 / Snapdragon 8 Elite for Galaxy | Qwen3.8-27B | NOT_E2E_PROFILEABLE | No complete qualified target artifact and device-local stateful Omni backend. Hub jobs provide component execution, not an on-device generation/streaming controller; ADB has no attached device. | Supply and validate the missing matching artifact/backend, then run the complete specified workload. |
|
||||
| Mobile Galaxy S25 / Snapdragon 8 Elite for Galaxy | MiniCPM-o 4.5 | COMPONENT_FP16_SPEECH_HEAD_NPU_NUMERIC_PROFILED; NOT_E2E_PROFILEABLE | Fresh FP16 speech-head inference passed; all 41 outputs exactly matched the same artifact/dataset reference. Against the recovered original ONNX CPU source on the same fixture, logits relative L2 was 0.3605%, maximum new-K/V relative L2 1.311%, and top-1 speech token 1867 matched. Historical FP16 parity passed; tested W8A16 top-1 parity failed. The 100-sample component profile had p50/p95 7.197/7.615 ms, with all 1,029 execution-detail rows on NPU and reported inference peak memory 185,991,168 bytes. Generated-audio quality and qualified thinker/encoders/vocoder remain unverified. | Qualify a complete mobile checkpoint/export and the full multimodal/stateful pipeline; [speech-head evidence](../../../e2e_expansion_20260924/evidence/minicpmo_tts_qualcomm/README.md). |
|
||||
| Mobile Galaxy S25 / Snapdragon 8 Elite for Galaxy | InternVLA-A1 | COMPONENT_COSMOS_NPU_NUMERIC_PROFILED; SYNTHETIC_ACTION_SENSITIVITY_MEASURED; NOT_E2E_PROFILEABLE | The real Place_Markpen Cosmos image encoder FP32 ONNX export matched its source on two six-frame fixtures. Galaxy S25 FP16 QNN DLC inference returned a finite latent within 0.0524% relative L2 / 65.61 dB SNR of the pinned pattern source. Injecting this device latent into the same real policy on local CPU changed synthetic `[1,50,32]` actions by 0.742% relative L2; the CPU source repeated bitwise identically. The 100-sample six-frame encoder profile had nearest-rank p50/p95 49.665/51.382 ms, all 121 execution-detail rows on NPU, and a reported inference peak of 158,232,576 bytes. No real-observation task tolerance exists, and the action policy was not resident on S25. ADB has no attached device. | Extend the [S25 encoder component](../../../e2e_expansion_20260924/evidence/internvla_cosmos_qualcomm/README.md), then validate real action quality, device-local state, admission, cancellation and sustained behavior. |
|
||||
| Mobile Galaxy S24 / Snapdragon 8 Gen 3 | Spark-X2.5 | COMPONENT_NPU_INFERENCE_PROFILED; NOT_E2E_PROFILEABLE | The real 1.7B checkpoint's fixed-shape full-attention decoder **one-layer** W8A16 QDQ artifact compiled as QNN DLC and ran on an exact hosted Galaxy S24 (Android 14). A same-dataset inference returned finite hidden/new-K/new-V outputs identical to the prior S25 NPU outputs; the profile attributed all 73 execution-detail rows to NPU. Across 100 component samples, nearest-rank p50/p95 was 1.109/1.205 ms; the 1.098 ms `estimated_inference_time` field is the minimum, not a median. Relative L2 versus original FP32 ONNX CPU was 1.351%/0.377%/0.650% for hidden/K/V; no token-quality tolerance is known. This is one cache bucket, not 28-layer prefill/decode, KV-ring, sampling, a resident local controller or complete-request timing. ADB has no attached device. | Qualify a complete device-resident artifact/controller with state and token-quality gates, 128-token generation across 512/1024 context transitions, admission/cancel and sustained memory/power; [S24 NPU raw evidence](../../../e2e_expansion_20260923/evidence/spark_s24_full_attention/README.md). |
|
||||
| Mobile Galaxy S24 / Snapdragon 8 Gen 3 | Qwen3-TTS 0.6B CustomVoice | TALKER_FP16_NUMERIC_FAIL; SHORT_GPU_VOCODER_NUMERIC_FAIL_PROFILED; CPU_TALKER_AND_VOCODER_NUMERIC_PROFILED; NOT_E2E_PROFILEABLE | A TFLite vocoder target compiled for exact S24 returned a finite, unsaturated CPU-requested waveform matching original ONNX CPU within 4.04e-6 relative L2 / 107.86 dB SNR on the retained fixture. Its 100-sample component profile measured p50/p95 4.826/4.958 s, all 738 execution-detail rows on CPU, and reported inference peak memory 851,140,608 bytes. A separate TFLite CPU-requested 28-layer talker step returned 57 finite outputs, preserved ONNX CPU top-1 speech token 80, and had logits relative L2 3.33e-5 and maximum new-cache relative L2 3.41e-5. Its 100-sample component profile measured p50/p95 36.976/43.264 ms with all 1,800 execution-detail rows on CPU and reported inference peak memory 1,922,400,256 bytes. The FP16 QNN talker step executed on the same synthetic cache fixture, but changed CPU top-1 speech token 80 to 1720 with 9.33% logits relative L2 and maximum new-K/V relative L2 11.23%. This differs from an earlier FP16 decoder parity test (39.13 dB SNR) on a different component/fixture; the tested INT8 decoder parity failed with negative SNR. A [pinned two-frame S24 TFLite attempt](../../../e2e_expansion_20260924/evidence/qwen_tts_vocoder_qualcomm/s24/tflite_short_chunk/README.md) preserved the waveform on a CPU-requested route at 5.75e-6 relative L2, but its GPU-requested output was fully saturated at 24.218 relative L2 / −27.68 dB SNR versus local ONNX CPU. The short CPU route measured 100 component samples at p50/p95 3.433/3.873 s for 0.16 s audio, with all 738 node rows on CPU and median component RTF 21.46. The GPU-requested route measured 100 samples at 0.601/0.895 s with 403 CPU and 335 GPU rows, but its waveform failed the gross numerical gate and the timing is not usable TTS performance. A FP32-preserving GPU control () returned a waveform bitwise identical to the fully saturated default GPU output, so that option did not repair this artifact. No full local TTS stream exists. | Investigate the [FP16 NPU talker numerical difference](../../../e2e_expansion_20260924/evidence/qwen_tts_talker_qualcomm/README.md), then validate the [complete local pipeline](../../../e2e_expansion_20260924/evidence/qwen_tts_vocoder_qualcomm/README.md) and a quality-passing artifact set. |
|
||||
| Mobile Galaxy S24 / Snapdragon 8 Gen 3 | Qwen3-TTS 0.6B CustomVoice | TALKER_FP16_NUMERIC_FAIL; SHORT_GPU_VOCODER_NUMERIC_FAIL_PROFILED; CPU_TALKER_AND_VOCODER_NUMERIC_PROFILED; NOT_E2E_PROFILEABLE | A TFLite vocoder target compiled for exact S24 returned a finite, unsaturated CPU-requested waveform matching original ONNX CPU within 4.04e-6 relative L2 / 107.86 dB SNR on the retained fixture. Its 100-sample component profile measured p50/p95 4.826/4.958 s, all 738 execution-detail rows on CPU, and reported inference peak memory 851,140,608 bytes. A separate TFLite CPU-requested 28-layer talker step returned 57 finite outputs, preserved ONNX CPU top-1 speech token 80, and had logits relative L2 3.33e-5 and maximum new-cache relative L2 3.41e-5. Its 100-sample component profile measured p50/p95 36.976/43.264 ms with all 1,800 execution-detail rows on CPU and reported inference peak memory 1,922,400,256 bytes. The FP16 QNN talker step executed on the same synthetic cache fixture, but changed CPU top-1 speech token 80 to 1720 with 9.33% logits relative L2 and maximum new-K/V relative L2 11.23%. This differs from an earlier FP16 decoder parity test (39.13 dB SNR) on a different component/fixture; the tested INT8 decoder parity failed with negative SNR. A [pinned two-frame S24 TFLite attempt](../../../e2e_expansion_20260924/evidence/qwen_tts_vocoder_qualcomm/s24/tflite_short_chunk/README.md) preserved the waveform on a CPU-requested route at 5.75e-6 relative L2, but its GPU-requested output was fully saturated at 24.218 relative L2 / −27.68 dB SNR versus local ONNX CPU. The short CPU route measured 100 component samples at p50/p95 3.433/3.873 s for 0.16 s audio, with all 738 node rows on CPU and median component RTF 21.46. The GPU-requested route measured 100 samples at 0.601/0.895 s with 403 CPU and 335 GPU rows, but its waveform failed the gross numerical gate and the timing is not usable TTS performance. A [FP32-preserving GPU control](../../../e2e_expansion_20260924/evidence/qwen_tts_vocoder_qualcomm/s24/tflite_short_chunk_gpu_fp32/audit_report.json) returned a waveform bitwise identical to the fully saturated default GPU output, so that option did not repair this artifact. No full local TTS stream exists. | Investigate the [FP16 NPU talker numerical difference](../../../e2e_expansion_20260924/evidence/qwen_tts_talker_qualcomm/README.md), then validate the [complete local pipeline](../../../e2e_expansion_20260924/evidence/qwen_tts_vocoder_qualcomm/README.md) and a quality-passing artifact set. |
|
||||
| Mobile Galaxy S24 / Snapdragon 8 Gen 3 | Qwen3.8-27B | NOT_E2E_PROFILEABLE | No complete qualified target artifact and device-local stateful Omni backend. Hub jobs provide component execution, not an on-device generation/streaming controller; ADB has no attached device. | Supply and validate the missing matching artifact/backend, then run the complete specified workload. |
|
||||
| Mobile Galaxy S24 / Snapdragon 8 Gen 3 | MiniCPM-o 4.5 | COMPONENT_FP16_SPEECH_HEAD_NPU_NUMERIC_PROFILED; NOT_E2E_PROFILEABLE | The retained FP16 speech-head job used a dataset with identical tensors to the recovered S25 fixture. Against original ONNX CPU, logits relative L2 was 0.3459%, maximum new-K/V relative L2 1.235%, and top-1 speech token 1867 matched. Earlier parity reported 49.22 dB SNR/top-1 match; tested W8A16 top-1 failed. The 100-sample component profile had p50/p95 8.537/9.006 ms, with all 1,029 execution-detail rows on NPU and reported inference peak memory 165,433,344 bytes. Generated-audio quality and the rest of the model remain unqualified. | Qualify thinker/encoders/vocoder and stateful local integration; [speech-head evidence](../../../e2e_expansion_20260924/evidence/minicpmo_tts_qualcomm/README.md). |
|
||||
| Mobile Galaxy S24 / Snapdragon 8 Gen 3 | InternVLA-A1 | COMPONENT_COSMOS_NPU_NUMERIC_PROFILED; NOT_E2E_PROFILEABLE | The real Place_Markpen Cosmos FP16 QNN image encoder returned a finite six-frame synthetic-pattern latent on exact hosted S24 at 0.0613% relative L2 / 64.25 dB SNR versus pinned FP32 source. Its 100-sample six-frame component profile had nearest-rank p50/p95 64.314/67.472 ms, all 121 execution-detail rows on NPU, reported inference peak memory 309,661,696 bytes and first-load peak 712,884,224 bytes. Injecting its latent into the real policy on local CPU changed synthetic actions by 0.631% relative L2, without a task tolerance. No real-observation action tolerance, complete local policy or stateful controller exists. ADB has no attached device. | Validate real observations, action units/timing, state, admission and cancellation on-device; [S24 Cosmos raw evidence](../../../e2e_expansion_20260924/evidence/internvla_cosmos_qualcomm/README.md). |
|
||||
|
||||
@@ -45,9 +45,11 @@ waveform at 1.89e-6 relative L2 on the same fixture, narrowing the model
|
||||
continuity question without proving the historical export revision.
|
||||
A [fresh revision-pinned 25-frame export](../../benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen_tts_vocoder_qualcomm/s25/full_pinned_tflite/README.md)
|
||||
now matches local eager CPU at 1.99e-6 relative L2 and has compiled for S25
|
||||
TFLite. Its GPU-requested numerical inference and node-placement profile are
|
||||
pending. This avoids promoting the historical artifact's timing to a
|
||||
revision-qualified deployment claim.
|
||||
TFLite. Its GPU-requested inference returned an unsaturated waveform at
|
||||
4.239% relative L2 versus pinned local ONNX CPU, bitwise identical to the
|
||||
historical target's output on the same fixture. The new target's own
|
||||
node-placement profile is pending, so the historical timing does not yet
|
||||
qualify this revision-pinned artifact.
|
||||
On [Galaxy S24](../../benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen_tts_vocoder_qualcomm/s24/tflite_short_chunk/README.md),
|
||||
the same pinned two-frame TFLite source passed CPU-requested waveform parity
|
||||
at 5.75e-6 relative L2, but the GPU-requested waveform saturated completely
|
||||
@@ -56,7 +58,9 @@ at 24.218 relative L2. The 100-sample short CPU component p50/p95 was
|
||||
measured 0.601/0.895 s but failed the waveform gate. A FP32-preserving GPU
|
||||
option returned the same saturated waveform bitwise, so it did not repair
|
||||
the tested delegate artifact. A [Snapdragon X Elite short QNN graph](../../benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen_tts_vocoder_qualcomm/xelite/qnn_short_chunk/README.md)
|
||||
compiled, with NPU-requested inference pending. Neither changes M2 support
|
||||
compiled and returned an unsaturated NPU-requested waveform at 0.930%
|
||||
relative L2 versus local ONNX CPU on one synthetic fixture. Its node-placement
|
||||
profile is pending. Neither changes M2 support
|
||||
depth before full stream and quality gates.
|
||||
|
||||
The [Spark hosted Qualcomm component expansion](../../benchmarks/edge_harness/results/e2e_expansion_20260923/evidence/spark_s24_full_attention/README.md) advances M1 only to one real fixed-shape decoder attention layer. The same W8A16 QNN DLC ran on S24, Snapdragon X Elite CRD and SA8775P ADP NPUs with identical outputs on one pinned fixture; S25 had prior separate component evidence. On [RB3 Gen 2](../../benchmarks/edge_harness/results/e2e_expansion_20260923/evidence/spark_rb3_full_attention/README.md), the tested W8A16 graph failed QNN loading even after an exact-device compile. FP32 ONNX ran on CPU with near-source parity, while calibrated W8A8 ran on NPU but had 20.9% hidden-state relative L2 and remains numerically unqualified. These are component C results, not full M1: device-local 28-layer prefill/continuous decode, KV/ring, sampling, token quality, admission/cancellation, complete-request timing and sustained power/thermal gates remain open. AI Hub is the test facility, not a deployment dependency.
|
||||
|
||||
Reference in new issue
Block a user