From 327a5dc66f455c221586b493b75f59d1b4864d6f Mon Sep 17 00:00:00 2001 From: tzhouam Date: Thu, 24 Sep 2026 20:34:04 +0800 Subject: [PATCH] Record InternVLA FP32 AMD NPU compiler failure in matrix --- .../results/e2e_expansion_20260924/README.md | 2 + .../.gitattributes | 2 + .../internvla_cosmos_amd_npu_fp32/README.md | 11 ++++ .../cpu_control.json | 52 ++++++++++++++++++ .../npu_probe.log | Bin 0 -> 1104 bytes ...probe_profile_2026-09-24_20-32-05_385.json | 0 .../evidence/summary/README.md | 6 +- 7 files changed, 71 insertions(+), 2 deletions(-) create mode 100644 benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/internvla_cosmos_amd_npu_fp32/.gitattributes create mode 100644 benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/internvla_cosmos_amd_npu_fp32/README.md create mode 100644 benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/internvla_cosmos_amd_npu_fp32/cpu_control.json create mode 100644 benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/internvla_cosmos_amd_npu_fp32/npu_probe.log create mode 100644 benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/internvla_cosmos_amd_npu_fp32/npu_probe_profile_2026-09-24_20-32-05_385.json diff --git a/benchmarks/edge_harness/results/e2e_expansion_20260924/README.md b/benchmarks/edge_harness/results/e2e_expansion_20260924/README.md index d9f3d9b2b..e50675094 100644 --- a/benchmarks/edge_harness/results/e2e_expansion_20260924/README.md +++ b/benchmarks/edge_harness/results/e2e_expansion_20260924/README.md @@ -127,6 +127,8 @@ The [talker decode-step attempt](evidence/qwen_tts_talker_qualcomm/README.md) pi A separate [HX370 AMD NPU MiniCPM-o speech-head attempt](evidence/minicpmo_amd_npu_speech_head/README.md) ran the real fixed-cache 20-layer ONNX step as one VitisAI NPU node. The local ORT CPU output was bitwise identical to the retained reference; NPU preserved top speech token 1867 but differed 2.172% on logits and up to 4.353% on new-cache tensors, failing the provisional component gate. The 420.34 s session build and one cold 39.22 ms step do not profile or qualify a complete MiniCPM-o stream. +A separate [HX370 AMD NPU InternVLA FP32 source attempt](evidence/internvla_cosmos_amd_npu_fp32/README.md) reproduced the pinned Cosmos encoder on ORT CPU within 2.29e-6 relative L2 on two fixtures. The unchanged FP32 graph then aborted during VitisAI session creation at a native `SmallVector` assertion. The empty profile supplies no NPU placement or latency evidence, and the action policy did not run. This adds an FP32 compiler failure to the earlier A16W8 artifact failures without establishing general model incompatibility. + The [MiniCPM-o speech-head study](evidence/minicpmo_tts_qualcomm/README.md) recovered the real 20-layer ONNX source and a matching retained fixture. S24 and S25 FP16 context-binary outputs preserved the ONNX CPU top-1 speech token with 0.346%/0.360% logits relative L2 on one step. Their 100-sample component profiles had p50/p95 8.537/9.006 ms and 7.197/7.615 ms, with all 1,029 execution-detail rows on NPU for each. FP16 QNN DLC compiles passed for SA8775P, X Elite and RB3. SA8775P NPU-requested inference preserved CPU top-1 token 1867 with 0.3568% logits relative L2; its 100-sample profile measured p50/p95 11.901/13.038 ms, all 944 execution-detail rows on NPU. X Elite same-fixture inference also preserved token 1867 with 0.3170% logits relative L2; its 100-sample profile measured p50/p95 9.905/11.037 ms, all 944 execution-detail rows on NPU. RB3 FP16 QNN inference failed at graph load despite compile success; an explicit TFLite CPU fallback same-fixture inference preserved top-1 token 1867 with logits relative L2 1.26e-6 versus ONNX CPU; its 100-sample CPU-requested profile measured p50/p95 59.518/71.347 ms with all 1,008 execution-detail rows on CPU. The complete multimodal model remains unqualified on those devices. The [native Windows Qwen3-TTS sustained analysis](evidence/qwen_tts_sustained_drift/README.md) finds a sharp playback throughput collapse around minute 26–27 in the historical 30-minute RTX 5090 Laptop stream. The last 17 serial requests all had simulated underruns with median RTF 1.331; the separate WSL run did not show that behavior. The revised local profiler recorded a completed current-source 30-minute repeat with device-wide NVML telemetry. The final 119 requests had median RTF 0.185 and three brief simulated underruns, so the prior collapse did not recur. The historical evidence and uncontrolled repeat conditions cannot identify its cause; underrun-free native Windows playback remains unqualified. diff --git a/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/internvla_cosmos_amd_npu_fp32/.gitattributes b/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/internvla_cosmos_amd_npu_fp32/.gitattributes new file mode 100644 index 000000000..c79c0d5cc --- /dev/null +++ b/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/internvla_cosmos_amd_npu_fp32/.gitattributes @@ -0,0 +1,2 @@ +*.log binary -eol +*profile*.json binary -eol diff --git a/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/internvla_cosmos_amd_npu_fp32/README.md b/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/internvla_cosmos_amd_npu_fp32/README.md new file mode 100644 index 000000000..5078a2a01 --- /dev/null +++ b/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/internvla_cosmos_amd_npu_fp32/README.md @@ -0,0 +1,11 @@ +# InternVLA Cosmos FP32 encoder on HX370 AMD NPU (2026-09-24) + +**Disposition: CPU parity passed; VitisAI compiler assertion; no NPU inference; NOT E2E.** This is the real Place_Markpen Cosmos CI8x8 image-encoder component, not the action policy or a device-local Omni request. The attempted candidate is the unchanged FP32 ONNX export, distinct from the earlier A16W8 QDQ candidates in the [prior study](../../../e2e_expansion_20260923/evidence/internvla_cosmos_amd_npu/README.md). + +The 502,735-byte ONNX graph has SHA-256 `fee79864e393df35475ba243e14b4995c5651aa33e7e182cd1fda726425668b7`; its 124,649,472-byte external weight data has SHA-256 `b9bc3411c10f05daec2ff977698ee100979ef756ed7f0d685275403517558b58`. The fixed `[6,3,256,256]` ramp and pattern fixtures have SHA-256 `dcbaf1391ca914d4bbccc1cb384daf05666e771d9cdae90c4f069ec8fcf05368`. These artifacts are retained outside Git under `/home/zhout/project/edge_infer/models/InternVLA-cosmos-onnx/`. + +On HX370 native Windows 11 build 26200, AMD NPU driver `32.0.203.329`, ONNX Runtime `1.30.0`, ONNX `1.22.0`, and VitisAI EP `1.8.63.0`, the [CPU control](cpu_control.json) returned `[6,16,32,32]` latents within `8.737078e-7` and `2.286713e-6` relative L2 of the retained source on ramp and pattern. The CPU-control report calls the candidate `quantized` because the probe's candidate argument has that generic name; the file passed here is the **same FP32 source**, with no QDQ conversion. + +With the same graph and fixtures, VitisAI session creation aborted at a native LLVM `SmallVector.h:291` assertion (`idx < size()`), process exit code **3**. The [raw mixed-encoding log](npu_probe.log), SHA-256 `0f48cf84b5b528d193931a99bb5e594bf792d7040cded1b5f0d8bc79ed1e2de5`, records the failure. No inference JSON was produced; the only emitted [profile file](npu_probe_profile_2026-09-24_20-32-05_385.json) is zero bytes, with no execution event or placement evidence. [AMD's VitisAI documentation](https://ryzenai.docs.amd.com/projects/WinML/en/latest/model_support.html) describes automatic BF16 handling of FP32 models; this run did not reach a profile from which internal precision can be confirmed. + +The [probe](../../../../experiments/probe_internvla_cosmos_amd_npu.py) checked exact model, weights, fixture and prior-export hashes before the provider attempt. This result constrains this FP32 export with EP `1.8.63.0`; it does not show that the encoder or model is generally unsupported on AMD NPU. Next isolate the first failing operator or subgraph, preserve CPU latent parity through a supported rewrite or partition, and verify actual NPU node execution and policy-action tolerance before revisiting the complete-device cell. diff --git a/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/internvla_cosmos_amd_npu_fp32/cpu_control.json b/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/internvla_cosmos_amd_npu_fp32/cpu_control.json new file mode 100644 index 000000000..aca179900 --- /dev/null +++ b/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/internvla_cosmos_amd_npu_fp32/cpu_control.json @@ -0,0 +1,52 @@ +{ + "scope": "real InternVLA Cosmos CI8x8 fixed-shape encoder component only; no action-policy E2E claim", + "hardware": "Ryzen AI 9 HX 370 AMD NPU", + "os": "Windows-11-10.0.26200-SP0", + "onnx": "1.22.0", + "onnxruntime": "1.30.0", + "source_sha256": "fee79864e393df35475ba243e14b4995c5651aa33e7e182cd1fda726425668b7", + "external_weights_sha256": "b9bc3411c10f05daec2ff977698ee100979ef756ed7f0d685275403517558b58", + "fixtures_sha256": "dcbaf1391ca914d4bbccc1cb384daf05666e771d9cdae90c4f069ec8fcf05368", + "input_shape": [ + 6, + 3, + 256, + 256 + ], + "output_shape": [ + 6, + 16, + 32, + 32 + ], + "quantization": "FP32 source, VitisAI automatic BF16 candidate", + "quantized_op_types": [ + "Conv", + "MatMul" + ], + "per_channel": false, + "max_relative_l2_gate": 0.001, + "source_cpu_parity": { + "ramp": { + "max_abs": 1.5974044799804688e-05, + "relative_l2": 8.73707802280739e-07 + }, + "pattern": { + "max_abs": 2.8371810913085938e-05, + "relative_l2": 2.2867125923021193e-06 + } + }, + "quantized_sha256": "fee79864e393df35475ba243e14b4995c5651aa33e7e182cd1fda726425668b7", + "quantized_bytes": 502735, + "quantized_cpu_parity": { + "ramp": { + "max_abs": 1.5974044799804688e-05, + "relative_l2": 8.73707802280739e-07 + }, + "pattern": { + "max_abs": 2.8371810913085938e-05, + "relative_l2": 2.2867125923021193e-06 + } + }, + "status": "passed_cpu_component" +} diff --git a/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/internvla_cosmos_amd_npu_fp32/npu_probe.log b/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/internvla_cosmos_amd_npu_fp32/npu_probe.log new file mode 100644 index 0000000000000000000000000000000000000000..cf472e707262af18c09f27df86cc022a5ada90e3 GIT binary patch literal 1104 zcmeHF(Q3jl6ur-a|By$8O>6C(mc0}^L>LaLY%dPUYOEPGDNU?(U%yi=b1xh0UBcz& z-gD16Nyuvu&8D-7&z^LWNR_a6vC+9;Qzh+0>m(I5&}uVj*ddqJ3dO8u^F{a=Ma{<4 zb=-k7ay{lce&6+-yH0=DbM9R>=zE^Sa*;@5MZQ`e72nBtEltL4yzRtU=6jzi@CGC*^LM&~; z$l-qrwMxs<5sVb80K4kuaQc8+4D5m|!Bgm^vZss7K8Ph}XwT>H1jsS [6,16,32,32] ONNX; ORT CPU matched the source on two fixtures with relative L2 8.74e-7/2.29e-6. A16W8 Conv+MatMul QDQ changed encoder output by 11.77%/5.03% on CPU and the VitisAI compiler emitted a missing-scale fatal assertion. MatMul-only A16W8 retained near-source CPU parity, but its original and ORT-preprocessed graphs each hit a native SmallVector compiler assertion. No VitisAI node executed, so this is not NPU component placement or complete InternVLA policy support. Failures are specific to these artifacts and EP 1.8.63.0. | Produce a VitisAI-supported whole-encoder or coarse substage artifact with representative calibration and numerical/policy tolerances; then verify actual NPU nodes, same-device handoff, full-policy behavior and memory/power before changing this disposition. [raw export and failure evidence](../../../e2e_expansion_20260923/evidence/internvla_cosmos_amd_npu/README.md). | +| PC HX370 + AMD NPU / Windows worker | InternVLA-A1 | COMPONENT_EXPORT_CPU_PASS; FP32_AND_A16W8_NPU_COMPILER_FAILED; NOT_E2E_PROFILEABLE | The real Place_Markpen Cosmos CI8x8 encoder exported to fixed FP32 [6,3,256,256] -> [6,16,32,32] ONNX; ORT CPU matched the source on two fixtures with relative L2 8.74e-7/2.29e-6. A16W8 Conv+MatMul QDQ changed encoder output by 11.77%/5.03% on CPU and the VitisAI compiler emitted a missing-scale fatal assertion. MatMul-only A16W8 retained near-source CPU parity, but its original and ORT-preprocessed graphs each hit a native SmallVector compiler assertion. A fresh attempt with the unchanged FP32 source graph also passed CPU parity on both fixtures, then aborted in VitisAI session creation at SmallVector.h:291 (exit code 3). Its only profile file is empty. No VitisAI node executed, so this is not NPU component placement or complete InternVLA policy support. Failures are specific to these artifacts and EP 1.8.63.0. | Isolate the compiler-failing operator or partition and preserve CPU parity through a supported rewrite; then verify actual NPU nodes, policy-action tolerance, same-device handoff, full-policy behavior and memory/power before changing this disposition. [A16W8 export and failure evidence](../../../e2e_expansion_20260923/evidence/internvla_cosmos_amd_npu/README.md); [FP32 source compiler-failure evidence](../../../e2e_expansion_20260924/evidence/internvla_cosmos_amd_npu_fp32/README.md). | | PC CPU+iGPU+NPU +/- discrete GPU, joint model execution | Spark-X2.5 | NOT_E2E_PROFILEABLE | No qualifying same-model multi-accelerator deployment/artifact plan or overlap/recovery evidence. Individual routes are not a joint pipeline. | Supply and validate the missing matching artifact/backend, then run the complete specified workload. | | PC CPU+iGPU+NPU +/- discrete GPU, joint model execution | Qwen3-TTS 0.6B CustomVoice | NOT_E2E_PROFILEABLE | No qualifying same-model multi-accelerator deployment/artifact plan or overlap/recovery evidence. Individual routes are not a joint pipeline. | Supply and validate the missing matching artifact/backend, then run the complete specified workload. | | PC CPU+iGPU+NPU +/- discrete GPU, joint model execution | Qwen3.8-27B | NOT_E2E_PROFILEABLE | No qualifying same-model multi-accelerator deployment/artifact plan or overlap/recovery evidence. Individual routes are not a joint pipeline. | Supply and validate the missing matching artifact/backend, then run the complete specified workload. |