diff --git a/benchmarks/edge_harness/results/e2e_expansion_20260924/README.md b/benchmarks/edge_harness/results/e2e_expansion_20260924/README.md index 937d8f2ca..c20ca075b 100644 --- a/benchmarks/edge_harness/results/e2e_expansion_20260924/README.md +++ b/benchmarks/edge_harness/results/e2e_expansion_20260924/README.md @@ -10,4 +10,4 @@ The [MiniCPM-o speech-head study](evidence/minicpmo_tts_qualcomm/README.md) reco The [native Windows Qwen3-TTS sustained analysis](evidence/qwen_tts_sustained_drift/README.md) finds a sharp playback throughput collapse around minute 26–27 in the historical 30-minute RTX 5090 Laptop stream. The last 17 serial requests all had simulated underruns with median RTF 1.331; the separate WSL run did not show that behavior. A revised local profiler now records timestamped device-wide NVML clocks, power, temperature, throttling and utilization to investigate the cause in a repeat run. The historical evidence cannot identify the cause or qualify native Windows sustained playback. -The [InternVLA Cosmos S25 attempt](evidence/internvla_cosmos_qualcomm/README.md) repackages the real policy's image encoder from its parity-tested ONNX export. FLOAT16 QNN DLC inference returned a latent within 0.0524% relative L2 of the source on one pattern. Injecting it into the real policy on local CPU changed synthetic actions by 0.742% relative L2, without a task tolerance. S25 six-frame component placement profile returned 100 samples at p50/p95 49.665/51.382 ms, with all 121 execution-detail rows on NPU. The same encoder compiled to FP16 QNN DLC for X Elite CRD; same-fixture NPU-requested inference returned a finite latent at 0.0608% relative L2 / 64.32 dB SNR versus the source, with its 100-sample six-frame profile at p50/p95 82.939/83.837 ms and all 121 execution-detail rows on NPU. An SA8775P FP16 QNN DLC inference returned a finite same-fixture latent at 0.0613% relative L2 / 64.25 dB SNR versus source; placement profiling is pending. Device-local action-policy support remains unqualified. The checkpoint-named Place_Markpen dataset returned HTTP 404, and the official vLLM-Omni example archive in InternData-A1 returned HTTP 403 gated-access denial under the current authenticated account; [real-observation access evidence](evidence/internvla_real_observation_access/README.md) records the blocker without treating synthetic actions as task validation. +The [InternVLA Cosmos S25 attempt](evidence/internvla_cosmos_qualcomm/README.md) repackages the real policy's image encoder from its parity-tested ONNX export. FLOAT16 QNN DLC inference returned a latent within 0.0524% relative L2 of the source on one pattern. Injecting it into the real policy on local CPU changed synthetic actions by 0.742% relative L2, without a task tolerance. S25 six-frame component placement profile returned 100 samples at p50/p95 49.665/51.382 ms, with all 121 execution-detail rows on NPU. The same encoder compiled to FP16 QNN DLC for X Elite CRD; same-fixture NPU-requested inference returned a finite latent at 0.0608% relative L2 / 64.32 dB SNR versus the source, with its 100-sample six-frame profile at p50/p95 82.939/83.837 ms and all 121 execution-detail rows on NPU. An SA8775P FP16 QNN DLC inference returned a finite same-fixture latent at 0.0613% relative L2 / 64.25 dB SNR versus source; its 100-sample six-frame profile measured p50/p95 116.021/116.327 ms with all 121 execution-detail rows on NPU. Device-local action-policy support remains unqualified. The checkpoint-named Place_Markpen dataset returned HTTP 404, and the official vLLM-Omni example archive in InternData-A1 returned HTTP 403 gated-access denial under the current authenticated account; [real-observation access evidence](evidence/internvla_real_observation_access/README.md) records the blocker without treating synthetic actions as task validation. diff --git a/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/internvla_cosmos_qualcomm/README.md b/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/internvla_cosmos_qualcomm/README.md index 77d437feb..718d38ce4 100644 --- a/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/internvla_cosmos_qualcomm/README.md +++ b/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/internvla_cosmos_qualcomm/README.md @@ -6,8 +6,8 @@ Workbench source model `mn4okj3rq` was [submitted for Galaxy S25 QNN DLC compila The same source model also [compiled to FP16 QNN DLC on Snapdragon X Elite CRD](xelite/compile_report.json), producing target `mmxj0w8kq` for Windows ARM. [Same-fixture NPU-requested inference](xelite/inference_report.json) on dataset `d26qo8p57` returned a finite FP32 `[6,16,32,32]` latent. Its [audit](xelite/audit_report.json) measured **0.0608% relative L2 / 64.32 dB SNR** versus the pinned source pattern, with cosine 0.999999815. The [placement profile](xelite/profile_report.json) returned 100 six-frame component samples at nearest-rank p50/p95 **82.939/83.837 ms**, with all **121 execution-detail rows on NPU**. The 82.446 ms estimated-time field is the sample minimum. Reported inference peak memory was 24,059,904 bytes and first-load peak 692,891,648 bytes. This is component evidence only; neither action quality nor a complete device-local policy was tested. -The same source also [compiled to FP16 QNN DLC on SA8775P ADP](sa8775p/compile_report.json), producing target mn750wz8m for Android. [Same-fixture NPU-requested inference](sa8775p/inference_report.json) using dataset d26qo8p57 returned a finite FP32 [6,16,32,32] latent. Its [audit](sa8775p/audit_report.json) measured **0.0613% relative L2 / 64.25 dB SNR** versus the pinned source pattern, with cosine 0.999999812. Actual NPU node assignment and timing await the [submitted placement profile](sa8775p/profile_submission.json); no complete local policy follows. +The same source also [compiled to FP16 QNN DLC on SA8775P ADP](sa8775p/compile_report.json), producing target mn750wz8m for Android. [Same-fixture NPU-requested inference](sa8775p/inference_report.json) using dataset d26qo8p57 returned a finite FP32 [6,16,32,32] latent. Its [audit](sa8775p/audit_report.json) measured **0.0613% relative L2 / 64.25 dB SNR** versus the pinned source pattern, with cosine 0.999999812. The [placement profile](sa8775p/profile_report.json) returned 100 six-frame component samples at nearest-rank p50/p95 **116.021/116.327 ms**, with all **121 execution-detail rows on NPU**. The 114.779 ms estimated-time field is the sample minimum. Reported inference peak memory was 116,924,416 bytes and first-load peak 659,845,120 bytes. No complete local policy follows. The [generic collector](../../../../experiments/collect_aihub_component.py) preserves terminal jobs, and the [Cosmos auditor](../../../../experiments/audit_internvla_cosmos_qualcomm.py) checks source/fixture hashes, target/device/dataset identity, latent shape and finite numerical difference against the pinned source reference. A separate [synthetic action sensitivity check](s25/action_sensitivity.json) injected either the pinned source or S25 latent into the same real Place_Markpen policy on **local CPU**, with zero state/noise. The [reproducer](../../../../experiments/probe_internvla_hosted_latent_actions.py) returned finite `[1,50,32]` actions; hosted-vs-source action relative L2 was **0.742%**, maximum absolute difference **0.0115**, cosine **0.999973**. Repeating the source policy gave bitwise identical actions. This is a cross-device numerical handoff experiment, not a device-local policy or robot-task pass; no real-observation action tolerance is known. -Next: audit SA8775P placement, test representative real observations and action tolerances, then build and validate a device-local stateful policy. Full multimodal observation processing, admission, cancellation and sustained profiling remain open. +Next: test representative real observations and action tolerances, then build and validate a device-local stateful policy. Full multimodal observation processing, admission, cancellation and sustained profiling remain open. diff --git a/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/internvla_cosmos_qualcomm/sa8775p/audit_report.json b/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/internvla_cosmos_qualcomm/sa8775p/audit_report.json index 5a179e9e2..a03894c72 100644 --- a/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/internvla_cosmos_qualcomm/sa8775p/audit_report.json +++ b/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/internvla_cosmos_qualcomm/sa8775p/audit_report.json @@ -1,6 +1,6 @@ { "scope": "one six-frame Cosmos image encoder on synthetic pattern; no complete InternVLA policy", - "status": "component_inference_numeric_measured", + "status": "component_inference_numeric_and_placement_measured", "device": { "name": "SA8775P ADP", "os": "14", @@ -48,7 +48,22 @@ }, "limits": [ "The fixture is synthetic; no real-observation or action-quality tolerance is established.", - "The action policy, state lifecycle, admission and complete request were not run on this device.", - "Requested NPU is not actual per-node placement without a device profile." - ] + "The action policy, state lifecycle, admission and complete request were not run on this device." + ], + "profile": { + "job_id": "jpe7z7ov5", + "sample_count": 100, + "sample_min_us": 114779, + "nearest_rank_p50_us": 116021, + "nearest_rank_p95_us": 116327, + "reported_estimated_inference_time_us": 114779, + "reported_peak_memory_bytes": 116924416, + "reported_first_load_time_us": 105214319, + "reported_warm_load_time_us": 469698, + "reported_first_load_peak_memory_bytes": 659845120, + "reported_warm_load_peak_memory_bytes": 268201984, + "compute_unit_row_counts": { + "NPU": 121 + } + } } diff --git a/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/internvla_cosmos_qualcomm/sa8775p/profile_report.json b/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/internvla_cosmos_qualcomm/sa8775p/profile_report.json new file mode 100644 index 000000000..4cbbff046 --- /dev/null +++ b/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/internvla_cosmos_qualcomm/sa8775p/profile_report.json @@ -0,0 +1,1043 @@ +{ + "timestamp_utc": "2026-09-24T01:32:20.796053+00:00", + "qai_hub_version": "0.55.0", + "facility": "Qualcomm AI Hub Workbench", + "job_id": "jpe7z7ov5", + "url": "https://workbench.aihub.qualcomm.com/jobs/jpe7z7ov5/", + "status": "SUCCESS", + "status_message": "", + "options": "--compute_unit npu", + "hub_version": "aihub-2026.09.11.0", + "device": { + "name": "SA8775P ADP", + "os": "14", + "attributes": [ + "os:android", + "framework:tflite", + "framework:onnx", + "vendor:qualcomm", + "abi:aarch64-android", + "format:auto", + "chipset:qualcomm-sa8775p", + "hexagon:v73", + "soc-model:52", + "htp-supports-fp16:true", + "htp-supports-weight-sharing:true", + "framework:qnn" + ] + }, + "model_id": "mn750wz8m", + "shapes": { + "pixels": [ + [ + 6, + 3, + 256, + 256 + ], + "float32" + ] + }, + "profile": { + "execution_summary": { + "estimated_inference_time": 114779, + "estimated_inference_peak_memory": 116924416, + "first_load_time": 105214319, + "first_load_peak_memory": 659845120, + "warm_load_time": 469698, + "warm_load_peak_memory": 268201984, + "compile_time": 0, + "compile_peak_memory": 0, + "compile_memory_increase_range": [ + 0, + 0 + ], + "compile_memory_peak_range": [ + 0, + 0 + ], + "first_load_memory_increase_range": [ + 120332640, + 126118352 + ], + "first_load_memory_peak_range": [ + 563167232, + 568952944 + ], + "warm_load_memory_increase_range": [ + 18446744073210102704, + 8454944 + ], + "warm_load_memory_peak_range": [ + 29622272, + 537526128 + ], + "inference_memory_increase_range": [ + 18446744073203195296, + 601968 + ], + "inference_memory_peak_range": [ + 520192, + 507478480 + ], + "all_compile_times": [], + "all_first_load_times": [ + 105214319 + ], + "all_warm_load_times": [ + 469698 + ], + "all_inference_times": [ + 117259, + 116376, + 114952, + 116301, + 116180, + 114997, + 116048, + 116271, + 115179, + 114999, + 116143, + 115963, + 114831, + 116171, + 116025, + 114864, + 116294, + 116014, + 114855, + 115966, + 116033, + 115994, + 115068, + 116196, + 116289, + 114937, + 116286, + 116180, + 115040, + 116442, + 116314, + 114902, + 116215, + 115980, + 114833, + 116042, + 116145, + 114886, + 115981, + 116081, + 114856, + 116118, + 116076, + 115906, + 114944, + 116166, + 116013, + 114857, + 116577, + 116130, + 114935, + 116278, + 116220, + 115191, + 116299, + 116195, + 114995, + 116018, + 116104, + 114866, + 115972, + 116107, + 114780, + 116037, + 116218, + 114950, + 115510, + 116101, + 115855, + 114822, + 115982, + 116021, + 114912, + 116049, + 116191, + 114938, + 116310, + 116149, + 114957, + 116246, + 116109, + 115008, + 116261, + 116129, + 114779, + 115928, + 116347, + 114950, + 116043, + 116083, + 115841, + 114819, + 116026, + 116064, + 114840, + 115952, + 116208, + 114937, + 115924, + 116327 + ] + }, + "execution_detail": [ + { + "name": "Input", + "type": "Input", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 362711 + }, + { + "name": "pixels_hoist_0231", + "type": "pixels", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 1785946 + }, + { + "name": "node_pad", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 3328941 + }, + { + "name": "node_Conv_29", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 1746768 + }, + { + "name": "node_Conv_30", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 1717906 + }, + { + "name": "node_Conv_31", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 458617 + }, + { + "name": "node_Conv_32", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 455564 + }, + { + "name": "node_Conv_33", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 437239 + }, + { + "name": "node_Conv_34", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 434896 + }, + { + "name": "node_cat", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 852972 + }, + { + "name": "node_Mul_1", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 1168100 + }, + { + "name": "node_pad_1", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 1209119 + }, + { + "name": "node_Conv_35", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 485457 + }, + { + "name": "node_Conv_36", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 499277 + }, + { + "name": "node_Conv_37", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 131954 + }, + { + "name": "node_Conv_38", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 129080 + }, + { + "name": "node_Conv_39", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 130104 + }, + { + "name": "node_Conv_40", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 130785 + }, + { + "name": "node_cat_1", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 297226 + }, + { + "name": "node_Mul_3", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 649764 + }, + { + "name": "node_conv2d_12", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 1170461 + }, + { + "name": "node_group_norm", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 3768608 + }, + { + "name": "node_sigmoid", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 0 + }, + { + "name": "node_mul_2", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 4246763 + }, + { + "name": "node_conv2d_13", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 0 + }, + { + "name": "node_group_norm_1", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 11768492 + }, + { + "name": "node_sigmoid_1", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 0 + }, + { + "name": "node_mul_3", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 8698080 + }, + { + "name": "node_conv2d_14", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 7628537 + }, + { + "name": "node_conv2d_15", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 406259 + }, + { + "name": "node_add", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 926061 + }, + { + "name": "node_group_norm_2", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 6026493 + }, + { + "name": "node_sigmoid_2", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 0 + }, + { + "name": "node_mul_4", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 7633971 + }, + { + "name": "node_conv2d_16", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 0 + }, + { + "name": "node_group_norm_3", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 16337516 + }, + { + "name": "node_sigmoid_3", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 0 + }, + { + "name": "node_mul_5", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 7642809 + }, + { + "name": "node_conv2d_17", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 7064641 + }, + { + "name": "node_add_1", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 201743 + }, + { + "name": "node_conv2d_18", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 2741473 + }, + { + "name": "node_group_norm_4", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 1895744 + }, + { + "name": "node_sigmoid_4", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 0 + }, + { + "name": "node_mul_6", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 2049452 + }, + { + "name": "node_conv2d_19", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 0 + }, + { + "name": "node_group_norm_5", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 8258801 + }, + { + "name": "node_sigmoid_5", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 0 + }, + { + "name": "node_mul_7", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 4101584 + }, + { + "name": "node_conv2d_20", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 7126121 + }, + { + "name": "node_conv2d_21", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 418427 + }, + { + "name": "node_add_2", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 684683 + }, + { + "name": "node_group_norm_6", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 3296987 + }, + { + "name": "node_sigmoid_6", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 0 + }, + { + "name": "node_mul_8", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 4046897 + }, + { + "name": "node_conv2d_22", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 0 + }, + { + "name": "node_group_norm_7", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 12283087 + }, + { + "name": "node_sigmoid_7", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 0 + }, + { + "name": "node_mul_9", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 5141446 + }, + { + "name": "node_conv2d_23", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 7430450 + }, + { + "name": "node_add_3", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 486121 + }, + { + "name": "node_group_norm_8", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 3285038 + }, + { + "name": "node_sigmoid_8", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 0 + }, + { + "name": "node_mul_10", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 4058070 + }, + { + "name": "node_conv2d_24", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 0 + }, + { + "name": "node_group_norm_9", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 12402041 + }, + { + "name": "node_sigmoid_9", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 0 + }, + { + "name": "node_mul_11", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 5081707 + }, + { + "name": "node_conv2d_25", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 7483431 + }, + { + "name": "node_add_4", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 581440 + }, + { + "name": "node_group_norm_10", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 3311044 + }, + { + "name": "node_sigmoid_10", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 0 + }, + { + "name": "node_mul_12", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 4086564 + }, + { + "name": "node_conv2d_26", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 0 + }, + { + "name": "node_group_norm_11", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 12335640 + }, + { + "name": "node_sigmoid_11", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 0 + }, + { + "name": "node_mul_13", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 5138106 + }, + { + "name": "node_conv2d_27", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 7424789 + }, + { + "name": "node_add_5", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 563260 + }, + { + "name": "node_group_norm_12", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 3299046 + }, + { + "name": "node_sigmoid_12", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 0 + }, + { + "name": "node_mul_14", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 4102658 + }, + { + "name": "node_conv2d_28", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 0 + }, + { + "name": "node_group_norm_13", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 12286844 + }, + { + "name": "node_sigmoid_13", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 0 + }, + { + "name": "node_mul_15", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 5137952 + }, + { + "name": "node_conv2d_29", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 7425569 + }, + { + "name": "node_add_6", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 470950 + }, + { + "name": "node_group_norm_14", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 3618957 + }, + { + "name": "node_conv2d_30", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 797494 + }, + { + "name": "node_conv2d_31", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 799787 + }, + { + "name": "node_conv2d_32", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 789052 + }, + { + "name": "conv2d_32_0312", + "type": "conv2d", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 1259191 + }, + { + "name": "node_view_4", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 794022 + }, + { + "name": "node_view_5", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 0 + }, + { + "name": "view_5_012", + "type": "view", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 0 + }, + { + "name": "node_bmm", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 4975108 + }, + { + "name": "node_mul_16", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 0 + }, + { + "name": "node_softmax", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 6254182 + }, + { + "name": "node_view_6", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 692032 + }, + { + "name": "node_permute_1", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 0 + }, + { + "name": "node_bmm_1", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 9984557 + }, + { + "name": "node_view_7", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 7175222 + }, + { + "name": "view_7_0231", + "type": "view", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 1064463 + }, + { + "name": "node_conv2d_33", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 800968 + }, + { + "name": "node_add_7", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 442715 + }, + { + "name": "node_group_norm_15", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 3641149 + }, + { + "name": "node_sigmoid_14", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 0 + }, + { + "name": "node_mul_17", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 4331396 + }, + { + "name": "node_conv2d_34", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 0 + }, + { + "name": "node_group_norm_16", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 12374050 + }, + { + "name": "node_sigmoid_15", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 0 + }, + { + "name": "node_mul_18", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 5146232 + }, + { + "name": "node_conv2d_35", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 7377965 + }, + { + "name": "node_add_8", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 0 + }, + { + "name": "node_group_norm_17", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 3644864 + }, + { + "name": "node_sigmoid_16", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 0 + }, + { + "name": "node_mul_19", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 3757927 + }, + { + "name": "node_conv2d_36", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 436847 + }, + { + "name": "node_conv2d_37", + "type": "node", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 18755 + }, + { + "name": "output_0_0231", + "type": "output", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 88590 + }, + { + "name": "Output", + "type": "Output", + "compute_unit": "NPU", + "execution_time": 0, + "execution_cycles": 23789 + } + ] + } +} diff --git a/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen_tts_sustained_drift/README.md b/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen_tts_sustained_drift/README.md index ba4c65d02..eec592dc1 100644 --- a/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen_tts_sustained_drift/README.md +++ b/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen_tts_sustained_drift/README.md @@ -15,4 +15,6 @@ The reproducible [analysis script](../../../../experiments/analyze_tts_sustained The Windows stream changed regime around minute 26–27. In its last window, all 17 requests had RTF above real time at the median and an underrun at first-audio playback. A small jitter buffer cannot cure this late sustained throughput deficit. The recorded profile sampled memory but not a timestamped GPU clock, temperature, power or throttling timeline, so its cause is unknown. The updated [local TTS profiler](../../../../profile_local_tts.py) can sample NVML and host telemetry with `--gpu-telemetry-interval-s 1`; repeat the Windows run with that option and correlate the device-wide samples with request timestamps. GPU telemetry is whole-device and cannot attribute power or utilization to the model alone. +The first instrumented retry [failed before model startup](windows_startup_failures/unc_cwd_report.json). A second retry from a native Windows working directory still imported an older editable Omni checkout; its [runtime log](windows_startup_failures/stale_checkout.log.gz) shows the stage worker trying the POSIX-only `os.fchmod` path on Windows, and its [report](windows_startup_failures/stale_checkout_report.json) has zero requests. The current fork source already has a Windows path for that lock. With the current fork first on `PYTHONPATH` and native `C:\Users\zhout\w2` as working directory, a [new smoke report](windows_current_source_smoke/report.json) completed startup in 69.38 s, one warmup, a slow-consumer request, cancellation acknowledgement and an after-cancel request. Its [request events](windows_current_source_smoke/requests.jsonl) and 71 [device-wide telemetry samples](windows_current_source_smoke/gpu_telemetry.jsonl) are retained. This smoke validates the measurement path, not 30-minute playback. A matching sustained repeat is running separately and will supersede this startup-only evidence when terminal. + This evidence does not qualify uninterrupted native Windows playback, speech quality, restart after cancellation, or mobile/embedded TTS. It does not establish thermal throttling or an Omni software regression as the cause. diff --git a/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen_tts_sustained_drift/windows_current_source_smoke/gpu_telemetry.jsonl b/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen_tts_sustained_drift/windows_current_source_smoke/gpu_telemetry.jsonl new file mode 100644 index 000000000..56a18a0c7 --- /dev/null +++ b/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen_tts_sustained_drift/windows_current_source_smoke/gpu_telemetry.jsonl @@ -0,0 +1,71 @@ +{"unix": 1790213199.4832008, "gpu_temperature_c": 55, "gpu_power_mw": 20639, "gpu_clock_sm_mhz": 795, "gpu_clock_memory_mhz": 405, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 6, "gpu_memory_used_bytes": 3570622464, "host_available_bytes": 29135712256, "host_cpu_utilization_pct": 0.0} +{"unix": 1790213200.5007305, "gpu_temperature_c": 55, "gpu_power_mw": 19386, "gpu_clock_sm_mhz": 757, "gpu_clock_memory_mhz": 405, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 5, "gpu_memory_used_bytes": 3570622464, "host_available_bytes": 29094313984, "host_cpu_utilization_pct": 8.0} +{"unix": 1790213201.5018215, "gpu_temperature_c": 54, "gpu_power_mw": 19568, "gpu_clock_sm_mhz": 787, "gpu_clock_memory_mhz": 405, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 5, "gpu_memory_used_bytes": 3570622464, "host_available_bytes": 29095579648, "host_cpu_utilization_pct": 9.1} +{"unix": 1790213202.5088906, "gpu_temperature_c": 54, "gpu_power_mw": 21677, "gpu_clock_sm_mhz": 780, "gpu_clock_memory_mhz": 405, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 5, "gpu_memory_used_bytes": 3570622464, "host_available_bytes": 29093814272, "host_cpu_utilization_pct": 9.9} +{"unix": 1790213203.517305, "gpu_temperature_c": 54, "gpu_power_mw": 20605, "gpu_clock_sm_mhz": 840, "gpu_clock_memory_mhz": 405, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 5, "gpu_memory_used_bytes": 3570622464, "host_available_bytes": 29100527616, "host_cpu_utilization_pct": 6.8} +{"unix": 1790213204.6159062, "gpu_temperature_c": 54, "gpu_power_mw": 22439, "gpu_clock_sm_mhz": 802, "gpu_clock_memory_mhz": 405, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 10, "gpu_memory_used_bytes": 3570622464, "host_available_bytes": 29043507200, "host_cpu_utilization_pct": 12.5} +{"unix": 1790213205.6573763, "gpu_temperature_c": 54, "gpu_power_mw": 21371, "gpu_clock_sm_mhz": 810, "gpu_clock_memory_mhz": 405, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 10, "gpu_memory_used_bytes": 3570622464, "host_available_bytes": 29044187136, "host_cpu_utilization_pct": 13.4} +{"unix": 1790213206.7041438, "gpu_temperature_c": 54, "gpu_power_mw": 19150, "gpu_clock_sm_mhz": 832, "gpu_clock_memory_mhz": 405, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 5, "gpu_memory_used_bytes": 3570622464, "host_available_bytes": 28750356480, "host_cpu_utilization_pct": 15.1} +{"unix": 1790213207.7247946, "gpu_temperature_c": 54, "gpu_power_mw": 18296, "gpu_clock_sm_mhz": 787, "gpu_clock_memory_mhz": 405, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 5, "gpu_memory_used_bytes": 3570622464, "host_available_bytes": 28349960192, "host_cpu_utilization_pct": 16.5} +{"unix": 1790213208.7327693, "gpu_temperature_c": 54, "gpu_power_mw": 19546, "gpu_clock_sm_mhz": 772, "gpu_clock_memory_mhz": 405, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 5, "gpu_memory_used_bytes": 3570622464, "host_available_bytes": 28251353088, "host_cpu_utilization_pct": 21.5} +{"unix": 1790213209.7421014, "gpu_temperature_c": 55, "gpu_power_mw": 22072, "gpu_clock_sm_mhz": 915, "gpu_clock_memory_mhz": 405, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 13, "gpu_memory_used_bytes": 3575996416, "host_available_bytes": 28110458880, "host_cpu_utilization_pct": 27.0} +{"unix": 1790213210.7809682, "gpu_temperature_c": 54, "gpu_power_mw": 20365, "gpu_clock_sm_mhz": 765, "gpu_clock_memory_mhz": 405, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 5, "gpu_memory_used_bytes": 3575996416, "host_available_bytes": 28011036672, "host_cpu_utilization_pct": 18.4} +{"unix": 1790213211.8100326, "gpu_temperature_c": 55, "gpu_power_mw": 20358, "gpu_clock_sm_mhz": 825, "gpu_clock_memory_mhz": 405, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 5, "gpu_memory_used_bytes": 3601588224, "host_available_bytes": 28017262592, "host_cpu_utilization_pct": 23.1} +{"unix": 1790213212.8141804, "gpu_temperature_c": 55, "gpu_power_mw": 21057, "gpu_clock_sm_mhz": 967, "gpu_clock_memory_mhz": 405, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 11, "gpu_memory_used_bytes": 3603095552, "host_available_bytes": 27432693760, "host_cpu_utilization_pct": 37.0} +{"unix": 1790213213.8234646, "gpu_temperature_c": 55, "gpu_power_mw": 22226, "gpu_clock_sm_mhz": 787, "gpu_clock_memory_mhz": 405, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 13, "gpu_memory_used_bytes": 3646390272, "host_available_bytes": 26278273024, "host_cpu_utilization_pct": 33.7} +{"unix": 1790213214.8353975, "gpu_temperature_c": 55, "gpu_power_mw": 27358, "gpu_clock_sm_mhz": 742, "gpu_clock_memory_mhz": 405, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 5, "gpu_memory_used_bytes": 3641016320, "host_available_bytes": 26117877760, "host_cpu_utilization_pct": 30.0} +{"unix": 1790213215.8378766, "gpu_temperature_c": 55, "gpu_power_mw": 22878, "gpu_clock_sm_mhz": 990, "gpu_clock_memory_mhz": 810, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 78, "gpu_memory_used_bytes": 3642261504, "host_available_bytes": 26117926912, "host_cpu_utilization_pct": 26.0} +{"unix": 1790213216.858904, "gpu_temperature_c": 56, "gpu_power_mw": 26852, "gpu_clock_sm_mhz": 1072, "gpu_clock_memory_mhz": 9001, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 59, "gpu_memory_used_bytes": 3663233024, "host_available_bytes": 26057072640, "host_cpu_utilization_pct": 28.4} +{"unix": 1790213217.9053948, "gpu_temperature_c": 56, "gpu_power_mw": 31184, "gpu_clock_sm_mhz": 1852, "gpu_clock_memory_mhz": 9001, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 3, "gpu_memory_used_bytes": 3659890688, "host_available_bytes": 25986555904, "host_cpu_utilization_pct": 22.3} +{"unix": 1790213218.9857621, "gpu_temperature_c": 56, "gpu_power_mw": 35164, "gpu_clock_sm_mhz": 1995, "gpu_clock_memory_mhz": 9001, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 25, "gpu_memory_used_bytes": 3613818880, "host_available_bytes": 25973886976, "host_cpu_utilization_pct": 35.8} +{"unix": 1790213220.041663, "gpu_temperature_c": 56, "gpu_power_mw": 31785, "gpu_clock_sm_mhz": 1957, "gpu_clock_memory_mhz": 9001, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 20, "gpu_memory_used_bytes": 3627712512, "host_available_bytes": 25889935360, "host_cpu_utilization_pct": 24.4} +{"unix": 1790213221.083642, "gpu_temperature_c": 56, "gpu_power_mw": 28363, "gpu_clock_sm_mhz": 780, "gpu_clock_memory_mhz": 810, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 14, "gpu_memory_used_bytes": 3660677120, "host_available_bytes": 25836867584, "host_cpu_utilization_pct": 26.7} +{"unix": 1790213222.1192544, "gpu_temperature_c": 56, "gpu_power_mw": 22691, "gpu_clock_sm_mhz": 705, "gpu_clock_memory_mhz": 810, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 4, "gpu_memory_used_bytes": 3660677120, "host_available_bytes": 25782816768, "host_cpu_utilization_pct": 20.5} +{"unix": 1790213223.163932, "gpu_temperature_c": 56, "gpu_power_mw": 19904, "gpu_clock_sm_mhz": 802, "gpu_clock_memory_mhz": 405, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 6, "gpu_memory_used_bytes": 3660677120, "host_available_bytes": 25722703872, "host_cpu_utilization_pct": 22.2} +{"unix": 1790213224.195849, "gpu_temperature_c": 56, "gpu_power_mw": 22283, "gpu_clock_sm_mhz": 885, "gpu_clock_memory_mhz": 405, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 11, "gpu_memory_used_bytes": 3660677120, "host_available_bytes": 25714479104, "host_cpu_utilization_pct": 21.6} +{"unix": 1790213225.260356, "gpu_temperature_c": 56, "gpu_power_mw": 23106, "gpu_clock_sm_mhz": 885, "gpu_clock_memory_mhz": 405, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 11, "gpu_memory_used_bytes": 3660677120, "host_available_bytes": 25688887296, "host_cpu_utilization_pct": 18.0} +{"unix": 1790213226.2967477, "gpu_temperature_c": 56, "gpu_power_mw": 22306, "gpu_clock_sm_mhz": 870, "gpu_clock_memory_mhz": 405, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 9, "gpu_memory_used_bytes": 3660677120, "host_available_bytes": 25670811648, "host_cpu_utilization_pct": 20.6} +{"unix": 1790213227.3697987, "gpu_temperature_c": 56, "gpu_power_mw": 22520, "gpu_clock_sm_mhz": 870, "gpu_clock_memory_mhz": 405, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 10, "gpu_memory_used_bytes": 3660677120, "host_available_bytes": 25707016192, "host_cpu_utilization_pct": 29.5} +{"unix": 1790213228.4130785, "gpu_temperature_c": 56, "gpu_power_mw": 21084, "gpu_clock_sm_mhz": 795, "gpu_clock_memory_mhz": 405, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 7, "gpu_memory_used_bytes": 3660677120, "host_available_bytes": 25708683264, "host_cpu_utilization_pct": 26.6} +{"unix": 1790213229.4297194, "gpu_temperature_c": 56, "gpu_power_mw": 18147, "gpu_clock_sm_mhz": 847, "gpu_clock_memory_mhz": 405, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 10, "gpu_memory_used_bytes": 3660677120, "host_available_bytes": 25703862272, "host_cpu_utilization_pct": 20.7} +{"unix": 1790213230.502961, "gpu_temperature_c": 56, "gpu_power_mw": 20550, "gpu_clock_sm_mhz": 787, "gpu_clock_memory_mhz": 405, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 5, "gpu_memory_used_bytes": 3677454336, "host_available_bytes": 25680760832, "host_cpu_utilization_pct": 17.8} +{"unix": 1790213231.5587397, "gpu_temperature_c": 56, "gpu_power_mw": 26204, "gpu_clock_sm_mhz": 1860, "gpu_clock_memory_mhz": 9001, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 3, "gpu_memory_used_bytes": 4202303488, "host_available_bytes": 25402011648, "host_cpu_utilization_pct": 32.5} +{"unix": 1790213232.6357276, "gpu_temperature_c": 57, "gpu_power_mw": 40989, "gpu_clock_sm_mhz": 2085, "gpu_clock_memory_mhz": 9001, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 16, "gpu_memory_used_bytes": 6348148736, "host_available_bytes": 23917826048, "host_cpu_utilization_pct": 34.7} +{"unix": 1790213233.7703621, "gpu_temperature_c": 57, "gpu_power_mw": 33395, "gpu_clock_sm_mhz": 1935, "gpu_clock_memory_mhz": 9001, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 3, "gpu_memory_used_bytes": 6359617536, "host_available_bytes": 24904699904, "host_cpu_utilization_pct": 37.7} +{"unix": 1790213234.9093769, "gpu_temperature_c": 56, "gpu_power_mw": 22149, "gpu_clock_sm_mhz": 1725, "gpu_clock_memory_mhz": 9001, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 1, "gpu_memory_used_bytes": 6208557056, "host_available_bytes": 24895795200, "host_cpu_utilization_pct": 14.6} +{"unix": 1790213236.0116951, "gpu_temperature_c": 56, "gpu_power_mw": 21971, "gpu_clock_sm_mhz": 1717, "gpu_clock_memory_mhz": 9001, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 1, "gpu_memory_used_bytes": 6171332608, "host_available_bytes": 24919101440, "host_cpu_utilization_pct": 13.6} +{"unix": 1790213237.1057372, "gpu_temperature_c": 56, "gpu_power_mw": 23349, "gpu_clock_sm_mhz": 1792, "gpu_clock_memory_mhz": 9001, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 4, "gpu_memory_used_bytes": 6613344256, "host_available_bytes": 24639811584, "host_cpu_utilization_pct": 23.4} +{"unix": 1790213238.2306685, "gpu_temperature_c": 56, "gpu_power_mw": 27090, "gpu_clock_sm_mhz": 1852, "gpu_clock_memory_mhz": 9001, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 3, "gpu_memory_used_bytes": 7309598720, "host_available_bytes": 23933394944, "host_cpu_utilization_pct": 32.7} +{"unix": 1790213239.3501265, "gpu_temperature_c": 58, "gpu_power_mw": 51753, "gpu_clock_sm_mhz": 2287, "gpu_clock_memory_mhz": 9001, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 23, "gpu_memory_used_bytes": 9271558144, "host_available_bytes": 23618150400, "host_cpu_utilization_pct": 24.8} +{"unix": 1790213240.3548572, "gpu_temperature_c": 57, "gpu_power_mw": 32660, "gpu_clock_sm_mhz": 1717, "gpu_clock_memory_mhz": 14001, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 1, "gpu_memory_used_bytes": 9871343616, "host_available_bytes": 23575003136, "host_cpu_utilization_pct": 15.1} +{"unix": 1790213241.3781776, "gpu_temperature_c": 57, "gpu_power_mw": 29482, "gpu_clock_sm_mhz": 1717, "gpu_clock_memory_mhz": 9001, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 1, "gpu_memory_used_bytes": 9871343616, "host_available_bytes": 23580078080, "host_cpu_utilization_pct": 17.9} +{"unix": 1790213242.3898137, "gpu_temperature_c": 56, "gpu_power_mw": 21525, "gpu_clock_sm_mhz": 1717, "gpu_clock_memory_mhz": 9001, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 1, "gpu_memory_used_bytes": 9871343616, "host_available_bytes": 23433216000, "host_cpu_utilization_pct": 23.1} +{"unix": 1790213243.3960268, "gpu_temperature_c": 56, "gpu_power_mw": 22147, "gpu_clock_sm_mhz": 1717, "gpu_clock_memory_mhz": 9001, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 1, "gpu_memory_used_bytes": 9877635072, "host_available_bytes": 23356010496, "host_cpu_utilization_pct": 18.9} +{"unix": 1790213244.4523852, "gpu_temperature_c": 56, "gpu_power_mw": 22588, "gpu_clock_sm_mhz": 495, "gpu_clock_memory_mhz": 810, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 3, "gpu_memory_used_bytes": 9866813440, "host_available_bytes": 23321116672, "host_cpu_utilization_pct": 28.0} +{"unix": 1790213245.5123162, "gpu_temperature_c": 56, "gpu_power_mw": 22293, "gpu_clock_sm_mhz": 892, "gpu_clock_memory_mhz": 405, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 7, "gpu_memory_used_bytes": 9868910592, "host_available_bytes": 23358095360, "host_cpu_utilization_pct": 23.2} +{"unix": 1790213246.5181894, "gpu_temperature_c": 56, "gpu_power_mw": 47864, "gpu_clock_sm_mhz": 592, "gpu_clock_memory_mhz": 9001, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 1, "gpu_memory_used_bytes": 14103060480, "host_available_bytes": 23275102208, "host_cpu_utilization_pct": 19.9} +{"unix": 1790213247.5792825, "gpu_temperature_c": 57, "gpu_power_mw": 38254, "gpu_clock_sm_mhz": 1672, "gpu_clock_memory_mhz": 9001, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 10, "gpu_memory_used_bytes": 14151294976, "host_available_bytes": 23046995968, "host_cpu_utilization_pct": 19.4} +{"unix": 1790213248.6410213, "gpu_temperature_c": 56, "gpu_power_mw": 19941, "gpu_clock_sm_mhz": 442, "gpu_clock_memory_mhz": 9001, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 1, "gpu_memory_used_bytes": 14153392128, "host_available_bytes": 23047921664, "host_cpu_utilization_pct": 10.7} +{"unix": 1790213249.655187, "gpu_temperature_c": 56, "gpu_power_mw": 20953, "gpu_clock_sm_mhz": 495, "gpu_clock_memory_mhz": 810, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 1, "gpu_memory_used_bytes": 14146314240, "host_available_bytes": 23050997760, "host_cpu_utilization_pct": 21.0} +{"unix": 1790213250.6775556, "gpu_temperature_c": 56, "gpu_power_mw": 20395, "gpu_clock_sm_mhz": 510, "gpu_clock_memory_mhz": 810, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 2, "gpu_memory_used_bytes": 14146314240, "host_available_bytes": 23054508032, "host_cpu_utilization_pct": 14.3} +{"unix": 1790213251.7742314, "gpu_temperature_c": 56, "gpu_power_mw": 15782, "gpu_clock_sm_mhz": 472, "gpu_clock_memory_mhz": 405, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 2, "gpu_memory_used_bytes": 14146314240, "host_available_bytes": 23066025984, "host_cpu_utilization_pct": 12.8} +{"unix": 1790213252.7879412, "gpu_temperature_c": 55, "gpu_power_mw": 15875, "gpu_clock_sm_mhz": 480, "gpu_clock_memory_mhz": 405, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 3, "gpu_memory_used_bytes": 14146314240, "host_available_bytes": 23091122176, "host_cpu_utilization_pct": 12.3} +{"unix": 1790213253.832425, "gpu_temperature_c": 55, "gpu_power_mw": 14752, "gpu_clock_sm_mhz": 570, "gpu_clock_memory_mhz": 405, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 5, "gpu_memory_used_bytes": 14149591040, "host_available_bytes": 23080730624, "host_cpu_utilization_pct": 14.9} +{"unix": 1790213254.936086, "gpu_temperature_c": 55, "gpu_power_mw": 17915, "gpu_clock_sm_mhz": 562, "gpu_clock_memory_mhz": 405, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 5, "gpu_memory_used_bytes": 14149828608, "host_available_bytes": 23076450304, "host_cpu_utilization_pct": 16.0} +{"unix": 1790213255.9399188, "gpu_temperature_c": 55, "gpu_power_mw": 16551, "gpu_clock_sm_mhz": 532, "gpu_clock_memory_mhz": 405, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 4, "gpu_memory_used_bytes": 14149828608, "host_available_bytes": 23093075968, "host_cpu_utilization_pct": 14.4} +{"unix": 1790213256.955558, "gpu_temperature_c": 55, "gpu_power_mw": 16688, "gpu_clock_sm_mhz": 540, "gpu_clock_memory_mhz": 405, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 3, "gpu_memory_used_bytes": 14183448576, "host_available_bytes": 22961127424, "host_cpu_utilization_pct": 16.6} +{"unix": 1790213257.9702487, "gpu_temperature_c": 55, "gpu_power_mw": 16743, "gpu_clock_sm_mhz": 480, "gpu_clock_memory_mhz": 405, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 3, "gpu_memory_used_bytes": 14183448576, "host_available_bytes": 22828597248, "host_cpu_utilization_pct": 17.0} +{"unix": 1790213259.0051956, "gpu_temperature_c": 55, "gpu_power_mw": 19517, "gpu_clock_sm_mhz": 645, "gpu_clock_memory_mhz": 405, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 5, "gpu_memory_used_bytes": 14183448576, "host_available_bytes": 22954094592, "host_cpu_utilization_pct": 20.2} +{"unix": 1790213260.1248965, "gpu_temperature_c": 55, "gpu_power_mw": 18757, "gpu_clock_sm_mhz": 630, "gpu_clock_memory_mhz": 405, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 5, "gpu_memory_used_bytes": 14183448576, "host_available_bytes": 22931677184, "host_cpu_utilization_pct": 12.4} +{"unix": 1790213261.1317537, "gpu_temperature_c": 55, "gpu_power_mw": 23817, "gpu_clock_sm_mhz": 982, "gpu_clock_memory_mhz": 405, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 6, "gpu_memory_used_bytes": 14185545728, "host_available_bytes": 22937169920, "host_cpu_utilization_pct": 12.1} +{"unix": 1790213262.190852, "gpu_temperature_c": 55, "gpu_power_mw": 17946, "gpu_clock_sm_mhz": 570, "gpu_clock_memory_mhz": 405, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 4, "gpu_memory_used_bytes": 14185545728, "host_available_bytes": 22944727040, "host_cpu_utilization_pct": 12.5} +{"unix": 1790213263.2524807, "gpu_temperature_c": 55, "gpu_power_mw": 18175, "gpu_clock_sm_mhz": 615, "gpu_clock_memory_mhz": 405, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 3, "gpu_memory_used_bytes": 14185545728, "host_available_bytes": 22910607360, "host_cpu_utilization_pct": 20.8} +{"unix": 1790213264.2618542, "gpu_temperature_c": 58, "gpu_power_mw": 49257, "gpu_clock_sm_mhz": 1912, "gpu_clock_memory_mhz": 9001, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 31, "gpu_memory_used_bytes": 14233780224, "host_available_bytes": 22751629312, "host_cpu_utilization_pct": 17.0} +{"unix": 1790213265.2645442, "gpu_temperature_c": 55, "gpu_power_mw": 20959, "gpu_clock_sm_mhz": 510, "gpu_clock_memory_mhz": 9001, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 1, "gpu_memory_used_bytes": 14233780224, "host_available_bytes": 22716305408, "host_cpu_utilization_pct": 24.4} +{"unix": 1790213266.3272357, "gpu_temperature_c": 55, "gpu_power_mw": 30621, "gpu_clock_sm_mhz": 742, "gpu_clock_memory_mhz": 9001, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 3, "gpu_memory_used_bytes": 14236147712, "host_available_bytes": 22694354944, "host_cpu_utilization_pct": 15.5} +{"unix": 1790213267.8191726, "gpu_temperature_c": 55, "gpu_power_mw": 29425, "gpu_clock_sm_mhz": 615, "gpu_clock_memory_mhz": 9001, "gpu_throttle_reasons": 0, "gpu_utilization_pct": 3, "gpu_memory_used_bytes": 14243291136, "host_available_bytes": 22747131904, "host_cpu_utilization_pct": 23.9} +{"unix": 1790213270.0743954, "gpu_temperature_c": 59, "gpu_power_mw": 65943, "gpu_clock_sm_mhz": 2212, "gpu_clock_memory_mhz": 10820, "gpu_throttle_reasons": 4, "gpu_utilization_pct": 87, "gpu_memory_used_bytes": 14247550976, "host_available_bytes": 22630240256, "host_cpu_utilization_pct": 27.7} +{"unix": 1790213271.0953062, "gpu_temperature_c": 58, "gpu_power_mw": 91914, "gpu_clock_sm_mhz": 1762, "gpu_clock_memory_mhz": 14001, "gpu_throttle_reasons": 4, "gpu_utilization_pct": 87, "gpu_memory_used_bytes": 14247550976, "host_available_bytes": 22642855936, "host_cpu_utilization_pct": 35.7} +{"unix": 1790213272.1024942, "gpu_temperature_c": 59, "gpu_power_mw": 103590, "gpu_clock_sm_mhz": 1815, "gpu_clock_memory_mhz": 14001, "gpu_throttle_reasons": 4, "gpu_utilization_pct": 87, "gpu_memory_used_bytes": 14242496512, "host_available_bytes": 22641094656, "host_cpu_utilization_pct": 29.1} +{"unix": 1790213273.1253207, "gpu_temperature_c": 57, "gpu_power_mw": 29238, "gpu_clock_sm_mhz": 472, "gpu_clock_memory_mhz": 10820, "gpu_throttle_reasons": 4, "gpu_utilization_pct": 1, "gpu_memory_used_bytes": 14244593664, "host_available_bytes": 22686302208, "host_cpu_utilization_pct": 53.5} +{"unix": 1790213274.1981916, "gpu_temperature_c": 57, "gpu_power_mw": 27460, "gpu_clock_sm_mhz": 660, "gpu_clock_memory_mhz": 14001, "gpu_throttle_reasons": 4, "gpu_utilization_pct": 2, "gpu_memory_used_bytes": 3666354176, "host_available_bytes": 26011693056, "host_cpu_utilization_pct": 21.9} diff --git a/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen_tts_sustained_drift/windows_current_source_smoke/report.json b/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen_tts_sustained_drift/windows_current_source_smoke/report.json new file mode 100644 index 000000000..45e58ba00 --- /dev/null +++ b/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen_tts_sustained_drift/windows_current_source_smoke/report.json @@ -0,0 +1,302 @@ +{ + "status": "completed", + "argv": [ + "\\\\wsl.localhost\\Ubuntu\\home\\zhout\\project\\edge_infer\\vllm-omni-edge\\benchmarks\\edge_harness\\profile_local_tts.py", + "--model", + "C:\\Users\\zhout\\w2\\models\\Qwen3-TTS-0.6B-85e237c", + "--out", + "\\\\wsl.localhost\\Ubuntu\\home\\zhout\\project\\edge_infer\\analysis\\experiments\\tts_windows_repo_smoke_detached_20260924", + "--repeats", + "0", + "--length-band", + "medium", + "--concurrency", + "1", + "--sustained-seconds", + "0", + "--gpu-telemetry-interval-s", + "1" + ], + "python": "C:\\Users\\zhout\\w2\\omni029venv\\Scripts\\python.exe", + "cwd": "C:\\Users\\zhout\\w2", + "loaded_omni_path": "\\\\wsl.localhost\\Ubuntu\\home\\zhout\\project\\edge_infer\\vllm-omni-edge\\vllm_omni\\__init__.py", + "host_environment": { + "VLLM_CUDART_SO_PATH": "C:\\Users\\zhout\\w2\\omni029venv\\Lib\\site-packages\\torch\\lib\\cudart64_13.dll", + "VLLM_CACHE_ROOT": "C:\\Users\\zhout\\c29", + "PYTHONUTF8": "1" + }, + "platform": "Windows-11-10.0.26200-SP0", + "runtime": { + "python": "3.12.10", + "platform": "Windows-11-10.0.26200-SP0", + "torch": "2.13.0+cu130", + "vllm": "0.29.0", + "vllm_omni": "0.29.0rc2.dev71+g00510ce38.d20260922", + "vllm_omni_file": "\\\\wsl.localhost\\Ubuntu\\home\\zhout\\project\\edge_infer\\vllm-omni-edge\\vllm_omni\\__init__.py", + "vllm_file": "C:\\Users\\zhout\\w2\\omni029venv\\Lib\\site-packages\\vllm\\__init__.py", + "checkouts": {} + }, + "settings": { + "model": "C:\\Users\\zhout\\w2\\models\\Qwen3-TTS-0.6B-85e237c", + "out": "\\\\wsl.localhost\\Ubuntu\\home\\zhout\\project\\edge_infer\\analysis\\experiments\\tts_windows_repo_smoke_detached_20260924", + "repeats": 0, + "sustained_seconds": 0.0, + "gpu_telemetry_interval_s": 1.0, + "length_band": "medium", + "concurrency": 1 + }, + "start_unix": 1790213199.4230263, + "requests": 3, + "cache_condition": "existing disk/JIT caches; not cold disk", + "quality_gate": "finite PCM checks only; ASR/speaker/listening reference gates pending", + "playback_scope": "arrival-based simulated playback; no sound-device measurement", + "terminal_audio": "DELTA output required; terminal audio is included once in complete playback metrics and archived separately", + "prompts": { + "short": { + "prompt_token_ids": [ + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0 + ], + "additional_information": { + "task_type": [ + "CustomVoice" + ], + "text": [ + "Hello, this is a speech test." + ], + "language": [ + "English" + ], + "speaker": [ + "Vivian" + ], + "global_request_id": [ + "after_cancel-short-1-3-8424a630" + ] + } + }, + "medium": { + "prompt_token_ids": [ + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0 + ], + "additional_information": { + "task_type": [ + "CustomVoice" + ], + "text": [ + "The garden has trees, flowers, and a small pond. We walk along the path and listen to the birds." + ], + "language": [ + "English" + ], + "speaker": [ + "Vivian" + ], + "global_request_id": [ + "slow_consumer-medium-1-2-9ec16a39" + ] + } + }, + "long": { + "prompt_token_ids": [ + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0 + ], + "additional_information": { + "task_type": [ + "CustomVoice" + ], + "text": [ + "Today we are testing a local speech system. It should speak clearly and keep the audio flowing smoothly. Today we are testing a local speech system. It should speak clearly and keep the audio flowing smoothly. Today we are testing a local speech system. It should speak clearly and keep the audio flowing smoothly. Today we are testing a local speech system. It should speak clearly and keep the audio flowing smoothly. " + ], + "language": [ + "English" + ], + "speaker": [ + "Vivian" + ], + "global_request_id": [ + "cancel-probe-a9894041" + ] + } + } + }, + "startup_s": 69.3754797000438, + "sampling_params": [ + "SamplingParams(n=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.05, temperature=0.9, top_p=1.0, top_k=50, min_p=0.0, seed=None, stop=[], stop_token_ids=[2150], bad_words=[], thinking_token_budget=None, include_stop_str_in_output=False, ignore_eos=False, max_tokens=4096, min_tokens=2, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, structured_outputs=None, extra_args=None)", + "SamplingParams(n=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=0.0, top_p=1.0, top_k=0, min_p=0.0, seed=None, stop=[], stop_token_ids=[], bad_words=[], thinking_token_budget=None, include_stop_str_in_output=False, ignore_eos=False, max_tokens=65536, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, structured_outputs=None, extra_args=None)" + ], + "memory_after_load": { + "samples": 189, + "interval_s": 0.25, + "note": "peaks are lower bounds: anything shorter than interval_s is missed. nvml_device_used is the whole card including other processes; tree_pss includes a share of file-backed pages. These three counters are different accountings and must not be summed.", + "nvml_device_used_peak": 14243291136, + "nvml_device_used_min": 3566690304, + "tree_rss_peak": 6339624960, + "tree_pss_peak": 0, + "host_available_min": 22691536896 + }, + "sustained_wall_s": 0.0, + "cancel": { + "abort_ack_s": 0.007381099974736571, + "scope": "abort acknowledgement and subsequent request; late-event gate not verified" + }, + "memory": { + "samples": 202, + "interval_s": 0.25, + "note": "peaks are lower bounds: anything shorter than interval_s is missed. nvml_device_used is the whole card including other processes; tree_pss includes a share of file-backed pages. These three counters are different accountings and must not be summed.", + "nvml_device_used_peak": 14247550976, + "nvml_device_used_min": 3566690304, + "tree_rss_peak": 6505013248, + "tree_pss_peak": 0, + "host_available_min": 22616309760 + }, + "gpu_telemetry": { + "path": "gpu_telemetry.jsonl", + "samples": 71, + "interval_s": 1.0, + "error": null, + "scope": "whole GPU/host; not attributable to this process; sampled values may miss peaks" + }, + "end_unix": 1790213274.3247113 +} diff --git a/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen_tts_sustained_drift/windows_current_source_smoke/requests.jsonl b/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen_tts_sustained_drift/windows_current_source_smoke/requests.jsonl new file mode 100644 index 000000000..9e891c2b0 --- /dev/null +++ b/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen_tts_sustained_drift/windows_current_source_smoke/requests.jsonl @@ -0,0 +1,3 @@ +{"request_id": "warmup-medium-1-1", "phase": "warmup", "length_band": "medium", "submitted_unix": 1790213269.0747316, "finished_unix": 1790213271.118368, "concurrency": 1, "sr": 24000, "chunks_all": [{"idx": 0, "t_ms": 797.8608999401331, "samples": 3840, "sr": 24000, "request_id": "warmup-medium-1-1", "terminal": false, "sha256": "aeb469aa6da8e9d4476a94dbcb92a11a6050c1c91e81318fbbb7facdc8bd9f95", "finite": true}, {"idx": 1, "t_ms": 801.3278000289574, "samples": 7680, "sr": 24000, "request_id": "warmup-medium-1-1", "terminal": false, "sha256": "b5414113d4ca9d7b99de9160cdd651f685adb51b2d013cb46e62c13ca044512b", "finite": true}, {"idx": 2, "t_ms": 801.6816999297589, "samples": 15360, "sr": 24000, "request_id": "warmup-medium-1-1", "terminal": false, "sha256": "519adb0aeb2cf628c6ba6c45f64ddb08280d1d332487c57b262bdc9bdba306ab", "finite": true}, {"idx": 3, "t_ms": 1015.4290000209585, "samples": 30720, "sr": 24000, "request_id": "warmup-medium-1-1", "terminal": false, "sha256": "b533a446dc5bc4740437f0363da1ceb070e7cee387744320364166bd992b49ed", "finite": true}, {"idx": 4, "t_ms": 1309.366500005126, "samples": 48000, "sr": 24000, "request_id": "warmup-medium-1-1", "terminal": false, "sha256": "dcdf91dc053e6d19c242fb1dc140fac5fb880f497f5a2f062d60bbbee39408e3", "finite": true}, {"idx": 5, "t_ms": 1684.077400015667, "samples": 48000, "sr": 24000, "request_id": "warmup-medium-1-1", "terminal": false, "sha256": "351ad07df82c837a95cfd1c22cf553636b23ad47cb5c4a5bb42ffa106e30a6b1", "finite": true}, {"idx": 6, "t_ms": 2043.140499969013, "samples": 32640, "sr": 24000, "request_id": "warmup-medium-1-1", "terminal": true, "sha256": "5ba83aea7ab71c94851ae824f78c4e04282adbba74b93bc5d8726fe4a956b21d", "finite": true}], "finished": true, "terminal_audio_s": 1.36, "prefix_metrics": {"ttfa_ms": 797.8608999401331, "playback_start_ms": 797.8608999401331, "stall_at_ttfa_ms": 0, "n_chunks": 6, "streamed_audio_s": 6.4, "tail_audio_s": 0.0, "total_wall_s": 2.0438366999151185, "rtf_streamed": 0.263137093752448, "rtf_total": 0.31934948436173727, "gap_ms": [3.4669000888243318, 0.3538999008014798, 213.74730009119958, 293.9374999841675, 374.7109000105411]}, "ttfa_ms": 797.8608999401331, "playback_start_ms": 797.8608999401331, "stall_at_ttfa_ms": 0, "n_chunks": 7, "streamed_audio_s": 7.76, "tail_audio_s": 0.0, "total_wall_s": 2.0438366999151185, "rtf_streamed": 0.2632913015423986, "rtf_total": 0.2633810180302988, "gap_ms": [3.4669000888243318, 0.3538999008014798, 213.74730009119958, 293.9374999841675, 374.7109000105411, 359.06309995334595], "underruns": []} +{"request_id": "slow_consumer-medium-1-2", "phase": "slow_consumer", "length_band": "medium", "submitted_unix": 1790213271.1528645, "finished_unix": 1790213272.9485905, "concurrency": 1, "sr": 24000, "chunks_all": [{"idx": 0, "t_ms": 89.14759999606758, "samples": 3840, "sr": 24000, "request_id": "slow_consumer-medium-1-2", "terminal": false, "sha256": "ed4c0ef2343900c8dcb8703ada282010c959e96285cd578d164289c1ff0cee18", "finite": true}, {"idx": 1, "t_ms": 349.19750003609806, "samples": 7680, "sr": 24000, "request_id": "slow_consumer-medium-1-2", "terminal": false, "sha256": "4781bf3b3378db46727b07b9660389974750af34bf1583948e8ce007c5aeab8e", "finite": true}, {"idx": 2, "t_ms": 526.2864000396803, "samples": 15360, "sr": 24000, "request_id": "slow_consumer-medium-1-2", "terminal": false, "sha256": "42a30cdc018a3b20313a0039361a4d65c84dbb5f955a516a132723e53ff31045", "finite": true}, {"idx": 3, "t_ms": 751.6142000677064, "samples": 30720, "sr": 24000, "request_id": "slow_consumer-medium-1-2", "terminal": false, "sha256": "b6b9de0c46a33182b3316894ec734ecc96b920d4e398a1ef71facb1855b8f028", "finite": true}, {"idx": 4, "t_ms": 933.0112000461668, "samples": 48000, "sr": 24000, "request_id": "slow_consumer-medium-1-2", "terminal": false, "sha256": "115d7794ee94e84f075ded1cb7849872c10a3bd3f7d0482a71d65a5f816aafe1", "finite": true}, {"idx": 5, "t_ms": 1188.879600027576, "samples": 48000, "sr": 24000, "request_id": "slow_consumer-medium-1-2", "terminal": false, "sha256": "ef98b9f64d29bbe3ad3e4f47e0c31fe544f8eab4e0e8c3d6df4ffd5c3ba8e809", "finite": true}, {"idx": 6, "t_ms": 1606.7554000765085, "samples": 44160, "sr": 24000, "request_id": "slow_consumer-medium-1-2", "terminal": true, "sha256": "13374cff5bc5c52ba5f0d402d507e91c44200f0b593fa7501e27d75098654e38", "finite": true}], "finished": true, "terminal_audio_s": 1.84, "prefix_metrics": {"ttfa_ms": 89.14759999606758, "playback_start_ms": 189.19750003609806, "stall_at_ttfa_ms": 100.04990004003048, "n_chunks": 6, "streamed_audio_s": 6.4, "tail_audio_s": 0.0, "total_wall_s": 1.795640700031072, "rtf_streamed": 0.18576243750430876, "rtf_total": 0.280568859379855, "gap_ms": [260.0499000400305, 177.08890000358224, 225.3278000280261, 181.39699997846037, 255.8683999814093]}, "ttfa_ms": 89.14759999606758, "playback_start_ms": 189.19750003609806, "stall_at_ttfa_ms": 100.04990004003048, "n_chunks": 7, "streamed_audio_s": 8.24, "tail_audio_s": 0.0, "total_wall_s": 1.795640700031072, "rtf_streamed": 0.1949945873879258, "rtf_total": 0.2179175606833825, "gap_ms": [260.0499000400305, 177.08890000358224, 225.3278000280261, 181.39699997846037, 255.8683999814093, 417.87580004893243], "underruns": [[1, 100.04990004003048]]} +{"request_id": "after_cancel-short-1-3", "phase": "after_cancel", "length_band": "short", "submitted_unix": 1790213273.20442, "finished_unix": 1790213273.7606058, "concurrency": 1, "sr": 24000, "chunks_all": [{"idx": 0, "t_ms": 102.57940005976707, "samples": 3840, "sr": 24000, "request_id": "after_cancel-short-1-3", "terminal": false, "sha256": "c17a50ea811d252330c91d07a282a7e1fb43a94faf3f39481d10dffc6a1b5a93", "finite": true}, {"idx": 1, "t_ms": 163.28460001386702, "samples": 7680, "sr": 24000, "request_id": "after_cancel-short-1-3", "terminal": false, "sha256": "3dfd02017107564eb916b8a1ba9fd341f64186cdd63d7197f8124168f48acbc2", "finite": true}, {"idx": 2, "t_ms": 414.2688000574708, "samples": 15360, "sr": 24000, "request_id": "after_cancel-short-1-3", "terminal": false, "sha256": "4ccb566d55d1491c4a17fbe353487047334929b1a30850e69cb69df285b95fcd", "finite": true}, {"idx": 3, "t_ms": 460.2598000783473, "samples": 30720, "sr": 24000, "request_id": "after_cancel-short-1-3", "terminal": false, "sha256": "67c181988c188cf233128fc9b4ad207c1b571d65b0cafc71da1a68d0428805cb", "finite": true}, {"idx": 4, "t_ms": 555.4441000567749, "samples": 13440, "sr": 24000, "request_id": "after_cancel-short-1-3", "terminal": true, "sha256": "676b453f924e4b7cf74abe1ef9c9364b6b14402b8a5877ef342c1c250688c6dc", "finite": true}], "finished": true, "terminal_audio_s": 0.56, "prefix_metrics": {"ttfa_ms": 102.57940005976707, "playback_start_ms": 102.57940005976707, "stall_at_ttfa_ms": 0, "n_chunks": 4, "streamed_audio_s": 2.4, "tail_audio_s": 0.0, "total_wall_s": 0.5560630999971181, "rtf_streamed": 0.1917749166993114, "rtf_total": 0.23169295833213255, "gap_ms": [60.70519995409995, 250.98420004360378, 45.99100002087653]}, "ttfa_ms": 102.57940005976707, "playback_start_ms": 102.57940005976707, "stall_at_ttfa_ms": 0, "n_chunks": 5, "streamed_audio_s": 2.96, "tail_audio_s": 0.0, "total_wall_s": 0.5560630999971181, "rtf_streamed": 0.1876500338029645, "rtf_total": 0.1878591554044318, "gap_ms": [60.70519995409995, 250.98420004360378, 45.99100002087653, 95.18429997842759], "underruns": []} diff --git a/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen_tts_sustained_drift/windows_startup_failures/stale_checkout.log.gz b/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen_tts_sustained_drift/windows_startup_failures/stale_checkout.log.gz new file mode 100644 index 000000000..47ce80072 Binary files /dev/null and b/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen_tts_sustained_drift/windows_startup_failures/stale_checkout.log.gz differ diff --git a/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen_tts_sustained_drift/windows_startup_failures/stale_checkout_report.json b/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen_tts_sustained_drift/windows_startup_failures/stale_checkout_report.json new file mode 100644 index 000000000..6689509a2 --- /dev/null +++ b/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen_tts_sustained_drift/windows_startup_failures/stale_checkout_report.json @@ -0,0 +1,273 @@ +{ + "status": "failed", + "argv": [ + "\\\\wsl.localhost\\Ubuntu\\home\\zhout\\project\\edge_infer\\vllm-omni-edge\\benchmarks\\edge_harness\\profile_local_tts.py", + "--model", + "C:\\Users\\zhout\\w2\\models\\Qwen3-TTS-0.6B-85e237c", + "--out", + "\\\\wsl.localhost\\Ubuntu\\home\\zhout\\project\\edge_infer\\analysis\\experiments\\tts_windows_nativecwd_smoke_20260924", + "--repeats", + "0", + "--length-band", + "medium", + "--concurrency", + "1", + "--sustained-seconds", + "0", + "--gpu-telemetry-interval-s", + "1" + ], + "python": "C:\\Users\\zhout\\w2\\omni029venv\\Scripts\\python.exe", + "host_environment": { + "VLLM_CUDART_SO_PATH": "C:\\Users\\zhout\\w2\\omni029venv\\Lib\\site-packages\\torch\\lib\\cudart64_13.dll", + "VLLM_CACHE_ROOT": "C:\\Users\\zhout\\c29", + "PYTHONUTF8": "1" + }, + "platform": "Windows-11-10.0.26200-SP0", + "runtime": { + "python": "3.12.10", + "platform": "Windows-11-10.0.26200-SP0", + "torch": "2.13.0+cu130", + "vllm": "0.29.0", + "vllm_omni": "0.29.0", + "vllm_omni_file": "C:\\Users\\zhout\\w2\\vllm-omni-src-0.29.0\\vllm_omni\\__init__.py", + "vllm_file": "C:\\Users\\zhout\\w2\\omni029venv\\Lib\\site-packages\\vllm\\__init__.py", + "checkouts": {} + }, + "settings": { + "model": "C:\\Users\\zhout\\w2\\models\\Qwen3-TTS-0.6B-85e237c", + "out": "\\\\wsl.localhost\\Ubuntu\\home\\zhout\\project\\edge_infer\\analysis\\experiments\\tts_windows_nativecwd_smoke_20260924", + "repeats": 0, + "sustained_seconds": 0.0, + "gpu_telemetry_interval_s": 1.0, + "length_band": "medium", + "concurrency": 1 + }, + "start_unix": 1790212654.1643922, + "requests": 0, + "cache_condition": "existing disk/JIT caches; not cold disk", + "quality_gate": "finite PCM checks only; ASR/speaker/listening reference gates pending", + "playback_scope": "arrival-based simulated playback; no sound-device measurement", + "terminal_audio": "DELTA output required; terminal audio is included once in complete playback metrics and archived separately", + "prompts": { + "short": { + "prompt_token_ids": [ + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0 + ], + "additional_information": { + "task_type": [ + "CustomVoice" + ], + "text": [ + "Hello, this is a speech test." + ], + "language": [ + "English" + ], + "speaker": [ + "Vivian" + ] + } + }, + "medium": { + "prompt_token_ids": [ + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0 + ], + "additional_information": { + "task_type": [ + "CustomVoice" + ], + "text": [ + "The garden has trees, flowers, and a small pond. We walk along the path and listen to the birds." + ], + "language": [ + "English" + ], + "speaker": [ + "Vivian" + ] + } + }, + "long": { + "prompt_token_ids": [ + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0 + ], + "additional_information": { + "task_type": [ + "CustomVoice" + ], + "text": [ + "Today we are testing a local speech system. It should speak clearly and keep the audio flowing smoothly. Today we are testing a local speech system. It should speak clearly and keep the audio flowing smoothly. Today we are testing a local speech system. It should speak clearly and keep the audio flowing smoothly. Today we are testing a local speech system. It should speak clearly and keep the audio flowing smoothly. " + ], + "language": [ + "English" + ], + "speaker": [ + "Vivian" + ] + } + } + }, + "error": "RuntimeError(\"Orchestrator initialization failed: Engine core initialization failed. See root cause above. Failed proc(s): {'StageEngineCoreProc_stage0_replica0': 1}\")", + "traceback": "Traceback (most recent call last):\n File \"C:\\Users\\zhout\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\threading.py\", line 1075, in _bootstrap_inner\n self.run()\n File \"C:\\Users\\zhout\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\threading.py\", line 1012, in run\n self._target(*self._args, **self._kwargs)\n File \"C:\\Users\\zhout\\w2\\vllm-omni-src-0.29.0\\vllm_omni\\engine\\async_omni_engine.py\", line 462, in _bootstrap_orchestrator\n loop.run_until_complete(_run_orchestrator())\n File \"C:\\Users\\zhout\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\asyncio\\base_events.py\", line 691, in run_until_complete\n return future.result()\n ^^^^^^^^^^^^^^^\n File \"C:\\Users\\zhout\\w2\\vllm-omni-src-0.29.0\\vllm_omni\\engine\\async_omni_engine.py\", line 419, in _run_orchestrator\n self._initialize_stages(stage_init_timeout)\n File \"C:\\Users\\zhout\\w2\\vllm-omni-src-0.29.0\\vllm_omni\\engine\\async_omni_engine.py\", line 345, in _initialize_stages\n self._initialize_stages_impl(stage_init_timeout)\n File \"C:\\Users\\zhout\\w2\\vllm-omni-src-0.29.0\\vllm_omni\\engine\\async_omni_engine.py\", line 366, in _initialize_stages_impl\n self._runtime.initialize()\n File \"C:\\Users\\zhout\\w2\\vllm-omni-src-0.29.0\\vllm_omni\\engine\\stage_runtime.py\", line 264, in initialize\n raise exc\n File \"C:\\Users\\zhout\\w2\\vllm-omni-src-0.29.0\\vllm_omni\\engine\\stage_runtime.py\", line 245, in initialize\n initialized_clients = self._initialize_stage_replicas(stage_plans, self._stage_init_timeout)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"C:\\Users\\zhout\\w2\\vllm-omni-src-0.29.0\\vllm_omni\\engine\\stage_runtime.py\", line 517, in _initialize_stage_replicas\n raise primary_exc\n File \"C:\\Users\\zhout\\w2\\vllm-omni-src-0.29.0\\vllm_omni\\engine\\stage_runtime.py\", line 478, in _init_group\n client = self._initialize_replica(\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"C:\\Users\\zhout\\w2\\vllm-omni-src-0.29.0\\vllm_omni\\engine\\stage_runtime.py\", line 743, in _initialize_replica\n return self._initialize_local_llm_replica(plan, stage_init_timeout)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"C:\\Users\\zhout\\w2\\vllm-omni-src-0.29.0\\vllm_omni\\engine\\stage_runtime.py\", line 849, in _initialize_local_llm_replica\n launch_stack.close()\n File \"C:\\Users\\zhout\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\contextlib.py\", line 618, in close\n self.__exit__(None, None, None)\n File \"C:\\Users\\zhout\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\contextlib.py\", line 610, in __exit__\n raise exc_details[1]\n File \"C:\\Users\\zhout\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\contextlib.py\", line 595, in __exit__\n if cb(*exc_details):\n ^^^^^^^^^^^^^^^^\n File \"C:\\Users\\zhout\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\contextlib.py\", line 144, in __exit__\n next(self.gen)\n File \"C:\\Users\\zhout\\w2\\vllm-omni-src-0.29.0\\vllm_omni\\engine\\stage_engine_startup.py\", line 1211, in launch_stage_replica\n wait_for_engine_startup(\n File \"C:\\Users\\zhout\\w2\\omni029venv\\Lib\\site-packages\\vllm\\v1\\engine\\utils.py\", line 1307, in wait_for_engine_startup\n raise RuntimeError(\nRuntimeError: Engine core initialization failed. See root cause above. Failed proc(s): {'StageEngineCoreProc_stage0_replica0': 1}\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n File \"\\\\wsl.localhost\\Ubuntu\\home\\zhout\\project\\edge_infer\\vllm-omni-edge\\benchmarks\\edge_harness\\profile_local_tts.py\", line 86, in run\n omni = AsyncOmni(model=args.model, deploy_profile=\"edge\", stage_init_timeout=180)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"C:\\Users\\zhout\\w2\\vllm-omni-src-0.29.0\\vllm_omni\\entrypoints\\async_omni.py\", line 156, in __init__\n OmniBase.__init__(self, model=model, **kwargs)\n File \"C:\\Users\\zhout\\w2\\vllm-omni-src-0.29.0\\vllm_omni\\entrypoints\\omni_base.py\", line 223, in __init__\n self.engine = AsyncOmniEngine(\n ^^^^^^^^^^^^^^^^\n File \"C:\\Users\\zhout\\w2\\vllm-omni-src-0.29.0\\vllm_omni\\engine\\async_omni_engine.py\", line 300, in __init__\n self._wait_for_orchestrator_init(startup_future, startup_timeout)\n File \"C:\\Users\\zhout\\w2\\vllm-omni-src-0.29.0\\vllm_omni\\engine\\async_omni_engine.py\", line 533, in _wait_for_orchestrator_init\n startup_future.result(\n File \"C:\\Users\\zhout\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\concurrent\\futures\\_base.py\", line 456, in result\n return self.__get_result()\n ^^^^^^^^^^^^^^^^^^^\n File \"C:\\Users\\zhout\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\concurrent\\futures\\_base.py\", line 401, in __get_result\n raise self._exception\nRuntimeError: Orchestrator initialization failed: Engine core initialization failed. See root cause above. Failed proc(s): {'StageEngineCoreProc_stage0_replica0': 1}\n", + "memory": { + "samples": 72, + "interval_s": 0.25, + "note": "peaks are lower bounds: anything shorter than interval_s is missed. nvml_device_used is the whole card including other processes; tree_pss includes a share of file-backed pages. These three counters are different accountings and must not be summed.", + "nvml_device_used_peak": 3275038720, + "nvml_device_used_min": 3275038720, + "tree_rss_peak": 3044057088, + "tree_pss_peak": 0, + "host_available_min": 28737007616 + }, + "gpu_telemetry": { + "path": "gpu_telemetry.jsonl", + "samples": 24, + "interval_s": 1.0, + "error": null, + "scope": "whole GPU/host; not attributable to this process; sampled values may miss peaks" + }, + "end_unix": 1790212678.0856693 +} diff --git a/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen_tts_sustained_drift/windows_startup_failures/unc_cwd_report.json b/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen_tts_sustained_drift/windows_startup_failures/unc_cwd_report.json new file mode 100644 index 000000000..fd5e0ee31 --- /dev/null +++ b/benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen_tts_sustained_drift/windows_startup_failures/unc_cwd_report.json @@ -0,0 +1,273 @@ +{ + "status": "failed", + "argv": [ + "\\\\wsl.localhost\\Ubuntu\\home\\zhout\\project\\edge_infer\\vllm-omni-edge\\benchmarks\\edge_harness\\profile_local_tts.py", + "--model", + "C:\\Users\\zhout\\w2\\models\\Qwen3-TTS-0.6B-85e237c", + "--out", + "\\\\wsl.localhost\\Ubuntu\\home\\zhout\\project\\edge_infer\\analysis\\experiments\\tts_windows_telemetry_20260924", + "--repeats", + "1", + "--length-band", + "medium", + "--concurrency", + "1", + "--sustained-seconds", + "1800", + "--gpu-telemetry-interval-s", + "1" + ], + "python": "C:\\Users\\zhout\\w2\\omni029venv\\Scripts\\python.exe", + "host_environment": { + "VLLM_CUDART_SO_PATH": "C:\\Users\\zhout\\w2\\omni029venv\\Lib\\site-packages\\torch\\lib\\cudart64_13.dll", + "VLLM_CACHE_ROOT": "C:\\Users\\zhout\\c29", + "PYTHONUTF8": "1" + }, + "platform": "Windows-11-10.0.26200-SP0", + "runtime": { + "python": "3.12.10", + "platform": "Windows-11-10.0.26200-SP0", + "torch": "2.13.0+cu130", + "vllm": "0.29.0", + "vllm_omni": "0.29.0", + "vllm_omni_file": "C:\\Users\\zhout\\w2\\vllm-omni-src-0.29.0\\vllm_omni\\__init__.py", + "vllm_file": "C:\\Users\\zhout\\w2\\omni029venv\\Lib\\site-packages\\vllm\\__init__.py", + "checkouts": {} + }, + "settings": { + "model": "C:\\Users\\zhout\\w2\\models\\Qwen3-TTS-0.6B-85e237c", + "out": "\\\\wsl.localhost\\Ubuntu\\home\\zhout\\project\\edge_infer\\analysis\\experiments\\tts_windows_telemetry_20260924", + "repeats": 1, + "sustained_seconds": 1800.0, + "gpu_telemetry_interval_s": 1.0, + "length_band": "medium", + "concurrency": 1 + }, + "start_unix": 1790211568.3760028, + "requests": 0, + "cache_condition": "existing disk/JIT caches; not cold disk", + "quality_gate": "finite PCM checks only; ASR/speaker/listening reference gates pending", + "playback_scope": "arrival-based simulated playback; no sound-device measurement", + "terminal_audio": "DELTA output required; terminal audio is included once in complete playback metrics and archived separately", + "prompts": { + "short": { + "prompt_token_ids": [ + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0 + ], + "additional_information": { + "task_type": [ + "CustomVoice" + ], + "text": [ + "Hello, this is a speech test." + ], + "language": [ + "English" + ], + "speaker": [ + "Vivian" + ] + } + }, + "medium": { + "prompt_token_ids": [ + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0 + ], + "additional_information": { + "task_type": [ + "CustomVoice" + ], + "text": [ + "The garden has trees, flowers, and a small pond. We walk along the path and listen to the birds." + ], + "language": [ + "English" + ], + "speaker": [ + "Vivian" + ] + } + }, + "long": { + "prompt_token_ids": [ + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0 + ], + "additional_information": { + "task_type": [ + "CustomVoice" + ], + "text": [ + "Today we are testing a local speech system. It should speak clearly and keep the audio flowing smoothly. Today we are testing a local speech system. It should speak clearly and keep the audio flowing smoothly. Today we are testing a local speech system. It should speak clearly and keep the audio flowing smoothly. Today we are testing a local speech system. It should speak clearly and keep the audio flowing smoothly. " + ], + "language": [ + "English" + ], + "speaker": [ + "Vivian" + ] + } + } + }, + "error": "RuntimeError(\"Orchestrator initialization failed: Engine core initialization failed. See root cause above. Failed proc(s): {'StageEngineCoreProc_stage1_replica0': 1}\")", + "traceback": "Traceback (most recent call last):\n File \"C:\\Users\\zhout\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\threading.py\", line 1075, in _bootstrap_inner\n self.run()\n File \"C:\\Users\\zhout\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\threading.py\", line 1012, in run\n self._target(*self._args, **self._kwargs)\n File \"C:\\Users\\zhout\\w2\\vllm-omni-src-0.29.0\\vllm_omni\\engine\\async_omni_engine.py\", line 462, in _bootstrap_orchestrator\n loop.run_until_complete(_run_orchestrator())\n File \"C:\\Users\\zhout\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\asyncio\\base_events.py\", line 691, in run_until_complete\n return future.result()\n ^^^^^^^^^^^^^^^\n File \"C:\\Users\\zhout\\w2\\vllm-omni-src-0.29.0\\vllm_omni\\engine\\async_omni_engine.py\", line 419, in _run_orchestrator\n self._initialize_stages(stage_init_timeout)\n File \"C:\\Users\\zhout\\w2\\vllm-omni-src-0.29.0\\vllm_omni\\engine\\async_omni_engine.py\", line 345, in _initialize_stages\n self._initialize_stages_impl(stage_init_timeout)\n File \"C:\\Users\\zhout\\w2\\vllm-omni-src-0.29.0\\vllm_omni\\engine\\async_omni_engine.py\", line 366, in _initialize_stages_impl\n self._runtime.initialize()\n File \"C:\\Users\\zhout\\w2\\vllm-omni-src-0.29.0\\vllm_omni\\engine\\stage_runtime.py\", line 264, in initialize\n raise exc\n File \"C:\\Users\\zhout\\w2\\vllm-omni-src-0.29.0\\vllm_omni\\engine\\stage_runtime.py\", line 245, in initialize\n initialized_clients = self._initialize_stage_replicas(stage_plans, self._stage_init_timeout)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"C:\\Users\\zhout\\w2\\vllm-omni-src-0.29.0\\vllm_omni\\engine\\stage_runtime.py\", line 517, in _initialize_stage_replicas\n raise primary_exc\n File \"C:\\Users\\zhout\\w2\\vllm-omni-src-0.29.0\\vllm_omni\\engine\\stage_runtime.py\", line 478, in _init_group\n client = self._initialize_replica(\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"C:\\Users\\zhout\\w2\\vllm-omni-src-0.29.0\\vllm_omni\\engine\\stage_runtime.py\", line 743, in _initialize_replica\n return self._initialize_local_llm_replica(plan, stage_init_timeout)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"C:\\Users\\zhout\\w2\\vllm-omni-src-0.29.0\\vllm_omni\\engine\\stage_runtime.py\", line 849, in _initialize_local_llm_replica\n launch_stack.close()\n File \"C:\\Users\\zhout\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\contextlib.py\", line 618, in close\n self.__exit__(None, None, None)\n File \"C:\\Users\\zhout\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\contextlib.py\", line 610, in __exit__\n raise exc_details[1]\n File \"C:\\Users\\zhout\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\contextlib.py\", line 595, in __exit__\n if cb(*exc_details):\n ^^^^^^^^^^^^^^^^\n File \"C:\\Users\\zhout\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\contextlib.py\", line 144, in __exit__\n next(self.gen)\n File \"C:\\Users\\zhout\\w2\\vllm-omni-src-0.29.0\\vllm_omni\\engine\\stage_engine_startup.py\", line 1211, in launch_stage_replica\n wait_for_engine_startup(\n File \"C:\\Users\\zhout\\w2\\omni029venv\\Lib\\site-packages\\vllm\\v1\\engine\\utils.py\", line 1307, in wait_for_engine_startup\n raise RuntimeError(\nRuntimeError: Engine core initialization failed. See root cause above. Failed proc(s): {'StageEngineCoreProc_stage1_replica0': 1}\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n File \"\\\\wsl.localhost\\Ubuntu\\home\\zhout\\project\\edge_infer\\vllm-omni-edge\\benchmarks\\edge_harness\\profile_local_tts.py\", line 86, in run\n omni = AsyncOmni(model=args.model, deploy_profile=\"edge\", stage_init_timeout=180)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"C:\\Users\\zhout\\w2\\vllm-omni-src-0.29.0\\vllm_omni\\entrypoints\\async_omni.py\", line 156, in __init__\n OmniBase.__init__(self, model=model, **kwargs)\n File \"C:\\Users\\zhout\\w2\\vllm-omni-src-0.29.0\\vllm_omni\\entrypoints\\omni_base.py\", line 223, in __init__\n self.engine = AsyncOmniEngine(\n ^^^^^^^^^^^^^^^^\n File \"C:\\Users\\zhout\\w2\\vllm-omni-src-0.29.0\\vllm_omni\\engine\\async_omni_engine.py\", line 300, in __init__\n self._wait_for_orchestrator_init(startup_future, startup_timeout)\n File \"C:\\Users\\zhout\\w2\\vllm-omni-src-0.29.0\\vllm_omni\\engine\\async_omni_engine.py\", line 533, in _wait_for_orchestrator_init\n startup_future.result(\n File \"C:\\Users\\zhout\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\concurrent\\futures\\_base.py\", line 456, in result\n return self.__get_result()\n ^^^^^^^^^^^^^^^^^^^\n File \"C:\\Users\\zhout\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\concurrent\\futures\\_base.py\", line 401, in __get_result\n raise self._exception\nRuntimeError: Orchestrator initialization failed: Engine core initialization failed. See root cause above. Failed proc(s): {'StageEngineCoreProc_stage1_replica0': 1}\n", + "memory": { + "samples": 2681, + "interval_s": 0.25, + "note": "peaks are lower bounds: anything shorter than interval_s is missed. nvml_device_used is the whole card including other processes; tree_pss includes a share of file-backed pages. These three counters are different accountings and must not be summed.", + "nvml_device_used_peak": 3278577664, + "nvml_device_used_min": 3262193664, + "tree_rss_peak": 3045744640, + "tree_pss_peak": 0, + "host_available_min": 26667917312 + }, + "gpu_telemetry": { + "path": "gpu_telemetry.jsonl", + "samples": 893, + "interval_s": 1.0, + "error": null, + "scope": "whole GPU/host; not attributable to this process; sampled values may miss peaks" + }, + "end_unix": 1790212492.9256425 +} diff --git a/benchmarks/edge_harness/results/e2e_profiling_20260922/evidence/summary/README.md b/benchmarks/edge_harness/results/e2e_profiling_20260922/evidence/summary/README.md index 97697969d..dce26e54c 100644 --- a/benchmarks/edge_harness/results/e2e_profiling_20260922/evidence/summary/README.md +++ b/benchmarks/edge_harness/results/e2e_profiling_20260922/evidence/summary/README.md @@ -18,7 +18,7 @@ The identical SA/RB3 TFLite binary matched original ONNX CPU within 3.45e-6 rela A separate [Qwen3-TTS talker decode-step attempt](../../../e2e_expansion_20260924/evidence/qwen_tts_talker_qualcomm/README.md) recovered a 28-layer ONNX export and synthetic fixed-cache fixture. Historical FP16 QNN context-binary inference executed on S24, S25, SA8775P and X Elite, but all four changed the CPU reference top-1 speech token with 8.57–9.43% logits relative L2; they are numerically unqualified on this fixture. An RB3 compile rejected floating-point input. Separate CPU-requested talker steps on X Elite ONNX and SA8775P TFLite preserved CPU token 80 with logits relative L2 2.07e-5 and 3.33e-5; X Elite had 100 component samples at p50/p95 191.414/309.428 ms with all 1,744 execution-detail rows on CPU; SA had 100 at 52.635/76.925 ms with all 1,800 rows on CPU. No complete talker loop or TTS stream follows. -A [Galaxy S25 InternVLA Cosmos attempt](../../../e2e_expansion_20260924/evidence/internvla_cosmos_qualcomm/README.md) packaged the real Place_Markpen image encoder into one ONNX file after two CPU parity fixtures. FLOAT16 QNN DLC inference returned a latent within 0.0524% relative L2 of the source on one pattern; its 100-sample six-frame component profile had p50/p95 49.665/51.382 ms and all 121 execution-detail rows on NPU. Injecting that latent into the real policy on local CPU changed synthetic actions by 0.742% relative L2, without a task tolerance. The same encoder compiled to FP16 QNN DLC for X Elite CRD and SA8775P; same-fixture NPU-requested inferences returned finite latents at 0.0608% and 0.0613% relative L2 versus source, with X Elite profiled on NPU and SA8775P placement pending. This advances component and action-sensitivity evidence, not device-local full-policy qualification. The Place_Markpen checkpoint names dataset a2d_real/a2d_pen_holder, which returned HTTP 404 under the current authenticated Hugging Face account. The official vLLM-Omni example names a different Place_Markpen archive in InternData-A1, but access to that gated dataset returned HTTP 403; [the access record](../../../e2e_expansion_20260924/evidence/internvla_real_observation_access/README.md) leaves real-observation reference-action validation open. +A [Galaxy S25 InternVLA Cosmos attempt](../../../e2e_expansion_20260924/evidence/internvla_cosmos_qualcomm/README.md) packaged the real Place_Markpen image encoder into one ONNX file after two CPU parity fixtures. FLOAT16 QNN DLC inference returned a latent within 0.0524% relative L2 of the source on one pattern; its 100-sample six-frame component profile had p50/p95 49.665/51.382 ms and all 121 execution-detail rows on NPU. Injecting that latent into the real policy on local CPU changed synthetic actions by 0.742% relative L2, without a task tolerance. The same encoder compiled to FP16 QNN DLC for X Elite CRD and SA8775P; same-fixture NPU-requested inferences returned finite latents at 0.0608% and 0.0613% relative L2 versus source, with both X Elite and SA8775P profiled on NPU. This advances component and action-sensitivity evidence, not device-local full-policy qualification. The Place_Markpen checkpoint names dataset a2d_real/a2d_pen_holder, which returned HTTP 404 under the current authenticated Hugging Face account. The official vLLM-Omni example names a different Place_Markpen archive in InternData-A1, but access to that gated dataset returned HTTP 403; [the access record](../../../e2e_expansion_20260924/evidence/internvla_real_observation_access/README.md) leaves real-observation reference-action validation open. A [MiniCPM-o speech-head study](../../../e2e_expansion_20260924/evidence/minicpmo_tts_qualcomm/README.md) recovered the historical 20-layer source and matching S24/S25 fixtures. Both FP16 context-binary outputs preserved the ONNX CPU top-1 speech token on this one decode step with logits relative L2 of 0.346%/0.360%. S24/S25 100-sample component profiles attributed all 1,029 execution-detail rows to NPU, with p50/p95 8.537/9.006 ms and 7.197/7.615 ms. New FP16 QNN DLC compiles passed for SA8775P, X Elite and RB3. SA8775P same-fixture inference preserved CPU top-1 token 1867 with 0.3568% logits relative L2; its 100-sample component profile measured p50/p95 11.901/13.038 ms with all 944 execution-detail rows on NPU. X Elite same-fixture inference also preserved token 1867 with 0.3170% logits relative L2; its 100-sample component profile measured p50/p95 9.905/11.037 ms with all 944 execution-detail rows on NPU; RB3 FP16 QNN inference failed at graph load; its explicit TFLite CPU fallback returned 41 finite outputs, preserved CPU top-1 token 1867 and had logits relative L2 1.26e-6 versus ONNX CPU, with 100 CPU-requested component samples at p50/p95 59.518/71.347 ms and all 1,008 execution-detail rows on CPU. These are speech-head component results, not multimodal MiniCPM-o requests. @@ -46,7 +46,7 @@ NOT E2E means no complete device/model workload was verified. “Scoped E2E” i | PC Snapdragon X Elite CRD / AI Hub | NOT E2E (one Spark attention layer on NPU) | NOT E2E (CPU talker/vocoder steps; QNN vocoder numeric) | NOT E2E | NOT E2E (NPU speech-head step profiled) | NOT E2E (Cosmos NPU encoder profiled) | | Mobile Galaxy S25 / Snapdragon 8 Elite for Galaxy | NOT E2E (one Spark attention layer on NPU) | NOT E2E (FP16 talker token mismatch) | NOT E2E | NOT E2E (NPU speech head step profiled) | NOT E2E (Cosmos NPU encoder profiled) | | Mobile Galaxy S24 / Snapdragon 8 Gen 3 | NOT E2E (one Spark attention layer on NPU) | NOT E2E (FP16 talker token mismatch) | NOT E2E | NOT E2E (NPU speech head step profiled) | NOT E2E | -| Embedded SA8775P ADP / AI Hub | NOT E2E (one Spark attention layer on NPU) | NOT E2E (CPU talker/vocoder steps; NPU vocoder failed) | NOT E2E | NOT E2E (NPU speech-head step profiled) | NOT E2E (Cosmos encoder numeric; placement pending) | +| Embedded SA8775P ADP / AI Hub | NOT E2E (one Spark attention layer on NPU) | NOT E2E (CPU talker/vocoder steps; NPU vocoder failed) | NOT E2E | NOT E2E (NPU speech-head step profiled) | NOT E2E (Cosmos NPU encoder profiled) | | Embedded RB3 Gen 2 / QCS6490 / AI Hub | NOT E2E (Spark CPU layer passed; W8A8 NPU quality open) | NOT E2E (mixed vocoder numeric unqualified) | NOT E2E | NOT E2E (CPU speech-head step profiled) | NOT E2E | ## Per-cell reasons and next steps @@ -107,7 +107,7 @@ NOT E2E means no complete device/model workload was verified. “Scoped E2E” i | Embedded SA8775P ADP / AI Hub | Qwen3-TTS 0.6B CustomVoice | COMPONENT_CPU_VOCODER_PARITY_AND_PLACEMENT_PASS; TALKER_TFLITE_CPU_NUMERIC_PROFILED; QNN_NPU_VOCODER_INFERENCE_FAILED; COMPONENT_GPU_NUMERIC_FAIL; TALKER_FP16_NUMERIC_FAIL; NOT_E2E_PROFILEABLE | On the exact SA8775P, the same TFLite binary's CPU-requested Code2Wav inference matched original ONNX CPU within 4.13e-6 relative L2 / 107.67 dB SNR, without saturation. Its 100-sample CPU-requested component profile had p50/p95 5.384/5.534 s with all 738 execution-detail rows on CPU; the service estimated-time field 5.250 s is the sample minimum. The GPU-requested output saturated 96.63% of samples with 11.832 relative L2; its fresh 100-sample mixed CPU/GPU profile p50/p95 was 1.776/1.880 s, timing failed audio. Even with FP32-preserving GPU options, 84.89% of samples saturated with 11.389 relative L2. The 28-layer FP16 QNN talker step executed but changed CPU top-1 token 80 to 1720 with 9.43% logits relative L2. A separate TFLite CPU-requested talker step preserved CPU top-1 token 80 with logits relative L2 3.33e-5 and maximum K/V relative L2 3.41e-5. Its 100-sample component profile had p50/p95 52.635/76.925 ms with all 1,800 execution-detail rows on CPU. A separate FP16 QNN DLC vocoder compile passed, but same-fixture NPU-requested inference failed after compilation without a waveform or placement samples. No complete pipeline exists. | Isolate the FP16 QNN vocoder runtime failure, GPU delegate failure and FP16 talker numerical drift, then establish listening quality, co-resident streaming, memory and sustained behavior; [vocoder](../../../e2e_expansion_20260924/evidence/qwen_tts_vocoder_qualcomm/README.md) and [talker](../../../e2e_expansion_20260924/evidence/qwen_tts_talker_qualcomm/README.md) evidence. | | Embedded SA8775P ADP / AI Hub | Qwen3.8-27B | NOT_E2E_PROFILEABLE | No complete qualified target artifact and device-local stateful Omni backend. Hub jobs provide component execution, not an on-device generation/streaming controller; ADB has no attached device. | Supply and validate the missing matching artifact/backend, then run the complete specified workload. | | Embedded SA8775P ADP / AI Hub | MiniCPM-o 4.5 | COMPONENT_SPEECH_HEAD_NPU_NUMERIC_PROFILED; NOT_E2E_PROFILEABLE | The real 20-layer MiniCPM-o speech-head FP16 QNN DLC ran on the exact SA8775P ADP. Same-fixture inference returned 41 finite tensors, preserved ONNX CPU top-1 token 1867, and had 0.3568% logits relative L2 and 1.225% maximum K/V relative L2. Its 100-sample component profile had nearest-rank p50/p95 11.901/13.038 ms, with all 944 execution-detail rows on NPU. Reported inference peak memory was 134,696,960 bytes and first-load peak 1,322,835,968 bytes; no complete thinker/encoders/vocoder bundle or device-local stateful controller exists. | Validate complete multimodal generation, state, speech quality, co-resident memory, cancellation and sustained power; [SA8775P raw evidence](../../../e2e_expansion_20260924/evidence/minicpmo_tts_qualcomm/README.md). | -| Embedded SA8775P ADP / AI Hub | InternVLA-A1 | COMPONENT_COSMOS_NUMERIC_MEASURED; PLACEMENT_PENDING; NOT_E2E_PROFILEABLE | The real Place_Markpen Cosmos FP16 QNN image encoder returned a finite FP32 latent for the six-frame synthetic pattern on exact SA8775P ADP. Against the pinned source, relative L2 was 0.0613%, SNR 64.25 dB and cosine 0.999999812. The NPU-requested route awaits actual node-placement/timing profile. No real-observation action tolerance, complete policy or local stateful Omni backend exists. | Audit the [SA8775P Cosmos placement profile](../../../e2e_expansion_20260924/evidence/internvla_cosmos_qualcomm/README.md), then validate real observations, action units/timing, state, admission, cancellation and sustained behavior. | +| Embedded SA8775P ADP / AI Hub | InternVLA-A1 | COMPONENT_COSMOS_NPU_NUMERIC_PROFILED; NOT_E2E_PROFILEABLE | The real Place_Markpen Cosmos FP16 QNN image encoder returned a finite FP32 latent for the six-frame synthetic pattern on exact SA8775P ADP. Against the pinned source, relative L2 was 0.0613%, SNR 64.25 dB and cosine 0.999999812. The 100-sample six-frame encoder profile had p50/p95 116.021/116.327 ms, with all 121 execution-detail rows on NPU. Reported component inference peak memory was 116,924,416 bytes and first-load peak 659,845,120 bytes. No real-observation action tolerance, complete policy or local stateful Omni backend exists. | Validate real observations, action units/timing, state, admission, cancellation and sustained behavior; [SA8775P Cosmos raw evidence](../../../e2e_expansion_20260924/evidence/internvla_cosmos_qualcomm/README.md). | | Embedded RB3 Gen 2 / QCS6490 / AI Hub | Spark-X2.5 | COMPONENT_CPU_PARITY_PASS; COMPONENT_NPU_NUMERIC_UNQUALIFIED; NOT_E2E_PROFILEABLE | On the exact hosted Dragonwing RB3 Gen 2 Vision Kit (QCS6490, QC Linux 1.6), the reused S24 W8A16 QNN DLC and a separately RB3-compiled W8A16 DLC each failed inference/profile at QnnModel_composeGraphsFromDlc: MODEL_GRAPH_ERROR. An FP32 ONNX one-layer CPU route passed with all 56 profile rows on CPU, source-relative hidden/K/V L2 of 8.73e-7/2.98e-7/3.37e-7 and 100-sample component p50/p95 19.304/21.113 ms. A calibrated W8A8 QNN DLC ran with all 61 rows on NPU and p50/p95 5.610/5.761 ms, but hidden/K/V relative L2 was 20.914%/1.702%/4.413% versus FP32 CPU, without a next-token quality gate. These are different precisions, not a qualified speedup; no complete 28-layer prefill/decode/KV-ring/sampling, local Omni controller or complete-request timing exists. | Improve representative W8A8 calibration or find another compatible quality-passing artifact; validate token parity, then full device-resident generation, admission/cancel and sustained power; [RB3 raw evidence](../../../e2e_expansion_20260923/evidence/spark_rb3_full_attention/README.md). | | Embedded RB3 Gen 2 / QCS6490 / AI Hub | Qwen3-TTS 0.6B CustomVoice | COMPONENT_MIXED_VOCODER_PROFILED_NUMERIC_UNQUALIFIED; NOT_E2E_PROFILEABLE | Fresh same-fixture TFLite GPU-requested vocoder inference returned a finite unsaturated waveform, but relative L2 versus original ONNX CPU was 15.73% (16.06 dB SNR), without a listening-quality tolerance; it is not a qualified audio path. The fresh 100-sample component profile had nearest-rank p50/p95 3.099/3.133 s and 403 CPU plus 335 GPU execution-detail rows. Its 3.070 s estimated-time field is the sample minimum, not median. The historical NPU talker build was rejected for floating-point inputs; no full model pipeline exists. | Establish waveform quality and a compatible integer talker artifact, then validate complete local execution, memory and sustained behavior; [vocoder evidence](../../../e2e_expansion_20260924/evidence/qwen_tts_vocoder_qualcomm/README.md). | | Embedded RB3 Gen 2 / QCS6490 / AI Hub | Qwen3.8-27B | NOT_E2E_PROFILEABLE | No complete qualified target artifact and device-local stateful Omni backend. Hub jobs provide component execution, not an on-device generation/streaming controller; ADB has no attached device. | Supply and validate the missing matching artifact/backend, then run the complete specified workload. | diff --git a/docs/design/omni_edge_e2e_roadmap.md b/docs/design/omni_edge_e2e_roadmap.md index 023e622fe..7e904b55d 100644 --- a/docs/design/omni_edge_e2e_roadmap.md +++ b/docs/design/omni_edge_e2e_roadmap.md @@ -25,7 +25,7 @@ For M4, a [MiniCPM-o 4.5 speech-head study](../../benchmarks/edge_harness/result The [native Windows Qwen3-TTS sustained audit](../../benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen_tts_sustained_drift/README.md) identifies an additional M2 desktop blocker. The historical RTX 5090 Laptop complete-model stream changed regime around minute 26–27; all 17 requests in minutes 27–30 had simulated playback underruns and median RTF 1.331. The separate WSL run had no sustained underruns. Existing logs do not identify the cause; timestamped GPU clocks, power, temperature and throttling are now available in the local profiler for a controlled repeat. Native Windows sustained playback remains unqualified. -For M4, a [Galaxy S25 Cosmos encoder attempt](../../benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/internvla_cosmos_qualcomm/README.md) repackages the real Place_Markpen FP32 ONNX export after retaining two CPU parity fixtures. FLOAT16 QNN DLC inference returned a latent within 0.0524% relative L2 of the source on one synthetic pattern. Injecting that output into the real policy on local CPU changed synthetic actions by 0.742% relative L2, without a task tolerance. S25 six-frame component profile returned 100 samples at p50/p95 49.665/51.382 ms, with all 121 execution-detail rows on NPU; the action policy did not run on S25. The same encoder compiled to FP16 QNN DLC for X Elite CRD; same-fixture NPU-requested inference returned a finite latent at 0.0608% relative L2 versus source, with its 100-sample six-frame profile at p50/p95 82.939/83.837 ms and all 121 execution-detail rows on NPU. An SA8775P FP16 QNN DLC inference returned a finite same-fixture latent at 0.0613% relative L2 versus source; placement profiling is pending. This is component/action-sensitivity evidence, not device-local M4 qualification. The checkpoint train config names a2d_real/a2d_pen_holder; an [authenticated access check](../../benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/internvla_real_observation_access/README.md) returned HTTP 404. The official vLLM-Omni example names an InternData-A1 Place_Markpen archive, but that dataset returned HTTP 403 gated-access denial under the current account, so real-observation reference-action quality is still unmeasured. +For M4, a [Galaxy S25 Cosmos encoder attempt](../../benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/internvla_cosmos_qualcomm/README.md) repackages the real Place_Markpen FP32 ONNX export after retaining two CPU parity fixtures. FLOAT16 QNN DLC inference returned a latent within 0.0524% relative L2 of the source on one synthetic pattern. Injecting that output into the real policy on local CPU changed synthetic actions by 0.742% relative L2, without a task tolerance. S25 six-frame component profile returned 100 samples at p50/p95 49.665/51.382 ms, with all 121 execution-detail rows on NPU; the action policy did not run on S25. The same encoder compiled to FP16 QNN DLC for X Elite CRD; same-fixture NPU-requested inference returned a finite latent at 0.0608% relative L2 versus source, with its 100-sample six-frame profile at p50/p95 82.939/83.837 ms and all 121 execution-detail rows on NPU. An SA8775P FP16 QNN DLC inference returned a finite same-fixture latent at 0.0613% relative L2 versus source; its 100-sample six-frame profile measured p50/p95 116.021/116.327 ms with all 121 execution-detail rows on NPU. This is component/action-sensitivity evidence, not device-local M4 qualification. The checkpoint train config names a2d_real/a2d_pen_holder; an [authenticated access check](../../benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/internvla_real_observation_access/README.md) returned HTTP 404. The official vLLM-Omni example names an InternData-A1 Place_Markpen archive, but that dataset returned HTTP 403 gated-access denial under the current account, so real-observation reference-action quality is still unmeasured. The [WSL CPU expansion](../../benchmarks/edge_harness/results/e2e_expansion_20260923/README.md) adds constrained three-stage MiniCPM-o text-to-speech requests and a CPU-only