anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

Record native Windows MiniCPM-o CPU control and unstable latency

Browse Source
This commit is contained in:
tzhouam committed 2026-09-24 03:18:27 +08:00
1 parent ddb8635cfe
commit 60d07a8afd
9 files changed
+854 -6
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

benchmarks/edge_harness/results/e2e_expansion_20260923/README.md
+1 -1
View File
@@ -8,7 +8,7 @@ A further [native Windows CPU Omni policy profile](evidence/internvla_omni_windo
The matching [native Windows CPU+Radeon Omni policy profile](evidence/internvla_omni_windows_hybrid/README.md) kept the BF16 action policy on CPU and ran only the pinned FP32 six-frame Cosmos encoder through a native DirectML worker. It returned 20/20 identical finite action chunks after one warmup at nearest-rank complete-request wall p50/p95 4.575/4.725 s, versus the separate native CPU baseline 4.878/4.902 s; startup was 140.31 s versus 135.60 s. Hybrid-vs-CPU action relative L2 difference was 2.4391% without a task tolerance. Native hybrid pre-launch refusal, in-flight abort and public `AsyncOmni.generate` action output passed with a 16 GiB shared-RAM reservation. The output device was verified as Radeon 890M, not every internal graph op. This is a synthetic integration pass, not robot-task E2E or sustained benefit.
A later [native Windows Radeon MiniCPM-o Omni profile](evidence/minicpmo_omni_radeon_profile20/README.md) completed one warmup and 20 serial **combined spoken-question+image → text+speech** requests with the same pinned GGUF set and patched C++ CLI. All 20 answered the red-square question and emitted complete speech; the first measured WAV independently transcribed with WER 0 using pinned Whisper tiny.en. Nearest-rank complete-request wall p50/p95 was **87.27/91.45 s**, much slower than the earlier 18.27 s single cold Omni request. A separate cold rerun after the profile took 88.01 s. All 21 successful new worker logs report Radeon Vulkan0 37/37 language layers, CPU Token2Wav and repeated `PrefetchVirtualMemory` warnings. The 21 GiB shared-RAM budget fit the pre-load available-RAM check and cancellation released it, but no matched CPU control or full memory/power trace identifies the regression's cause. This advances repeated functional evidence while weakening any claim of predictable Radeon performance under current memory conditions.
A later [native Windows Radeon MiniCPM-o Omni profile](evidence/minicpmo_omni_radeon_profile20/README.md) completed one warmup and 20 serial **combined spoken-question+image → text+speech** requests with the same pinned GGUF set and patched C++ CLI. All 20 answered the red-square question and emitted complete speech; the first measured WAV independently transcribed with WER 0 using pinned Whisper tiny.en. Nearest-rank complete-request wall p50/p95 was **87.27/91.45 s**, much slower than the earlier 18.27 s single cold Omni request. A separate cold rerun after the profile took 88.01 s. All 21 successful new worker logs report Radeon Vulkan0 37/37 language layers, CPU Token2Wav and repeated `PrefetchVirtualMemory` warnings. The 21 GiB shared-RAM budget fit the pre-load available-RAM check and cancellation released it. A later one-request CPU-only control with the same fixture took 73.63 s and also logged prefetch warnings despite 32.28 GB OS-available before load. The CPU control was not a matched-condition comparison; no full memory/power trace identifies the regression's cause. This advances repeated functional evidence while weakening any claim of predictable Radeon performance under current memory conditions.
| Device / model | Verified depth | Observed output and time | Remaining gate |
|---|---|---|---|
benchmarks/edge_harness/results/e2e_expansion_20260923/evidence/minicpmo_omni_radeon_profile20/README.md
+2 -1
View File
@@ -10,9 +10,10 @@ The fixture is the same 16 kHz spoken question “What color is the square in th
|---|---:|---:|---|
| One warmup + 20 serial measured requests | Nearest-rank p50/p95 **87.27/91.45 s**; range 69.91–97.90 s | 51.92 s | 21 GiB shared-RAM reservation and capacity; 23.80 GB OS-available before load; max sampled worker RSS 15.30 GB |
| Separate cold rerun after profile | One **88.01 s** request | 47.63 s | Same 21 GiB budget; 23.99 GB OS-available before load; sampled worker RSS 15.30 GB ([report](cold_after_profile.json), [log](cold_after_worker_logs/)) |
| CPU-only control, run later | One **73.63 s** request with the same fixture | 51.80 s | Same 21 GiB budget but **32.28 GB** OS-available before load; sampled worker RSS 13.10 GB ([report](cpu_control_current.json), [worker log](cpu_control_worker.log)) |
| Earlier cold Omni smoke | One **18.27 s** request | 6.83 s | 24 GiB reservation under a declared 30 GiB ceiling; [earlier record](../minicpmo_omni_cpp/radeon_report.json) |
These are different runs, not a paired speed comparison. The new repeated profile and cold rerun both passed functionally but were roughly five times slower than the older single cold request. Each of the 21 successful new worker logs contains two `PrefetchVirtualMemory failed` warnings, while the earlier single-request log contains none. Representative new language-model load times were about 25–27 s versus 6.07 s in the earlier log; the first measured new request took 89.67 s wall, including speech generation. Those observations are consistent with memory pressure or changed file-cache/paging conditions, **but do not identify a root cause**. Windows free RAM was only about 22.17 GiB at admission for a 21 GiB controller ceiling. Power/thermal state, full OS commit/pagefile time series, shared-iGPU allocation and a matched-condition CPU control were not captured. Do not reuse the older 18.27 s result as expected warmed performance or claim the Radeon path is faster than CPU under this condition.
These are different runs, not a paired speed comparison. The new repeated Radeon profile and cold rerun both passed functionally but were roughly five times slower than the older single cold request. Each of the 21 successful new Radeon worker logs contains two `PrefetchVirtualMemory failed` warnings, while the earlier single-request log contains none. Representative new Radeon language-model load times were about 25–27 s versus 6.07 s in the earlier log. The later CPU-only control used the same model/input hashes and pinned CPU CLI SHA256 `29247187ddf3b13591cd14cc070e5e25c6bb7547296acc5e8a74e2945811d725`. It also reported two prefetch warnings and a 36.03 s model load, versus 13.53 s in its earlier 24.34 s cold Omni smoke. Its complete request took 73.63 s despite more OS-available RAM at admission. The CPU control answered correctly and emitted a [1.96 s WAV](cpu_control_current.wav); pinned Whisper tiny.en transcribed “The square and the image is red.” (WER 0.143 for one output, [ASR report](cpu_control_current_asr.json)). These observations show the slowdown is not unique to Radeon or explained by **pre-load** free RAM alone. Memory pressure, changed UNC file-cache/paging conditions, and power/thermal state remain possible explanations, **not established causes**. Full OS commit/pagefile time series, shared-iGPU allocation and a truly matched-condition CPU/Radeon sequence were not captured. Do not reuse the older 18.27 s result as expected warmed performance or claim the Radeon path is faster than CPU under this condition.
The stage verified artifacts and placement, admitted the explicit shared-RAM demand, and the final in-flight abort produced no stale output. Both abort and shutdown left zero reserved/quarantined bytes in the [profile report](profile20.json). The probe now refuses a declared capacity above OS-available RAM before loading. Sampled worker RSS does not establish the weight-loading peak or bound shared GPU allocation. No post-cancel restart, concurrency, real image/audio suite, playable streaming or sustained power/thermal gate was run.
benchmarks/edge_harness/results/e2e_expansion_20260923/evidence/minicpmo_omni_radeon_profile20/cpu_control_current.json
+173
View File
@@ -0,0 +1,173 @@
{
"scope": "MiniCPM-o GGUF complete audio+image\u2192text+speech through Omni StageRuntime/StagePool",
"placement": "cpu",
"model_revision": "db25077c33951fe163b42986fba0132e279872a2",
"artifact_sha256": {
"MiniCPM-o-4_5-Q4_K_M.gguf": "1237a97ee081b8abebc47aa7dad565701e8f5f904cdc92f6723ac4281bbc0932",
"audio/MiniCPM-o-4_5-audio-F16.gguf": "d5b188ac7feaf98e17175c3f9bd14bf269301bfd187439fdaa3e3a494fc32ef7",
"vision/MiniCPM-o-4_5-vision-F16.gguf": "1453678cc4e4fe18de241952962e234f265cb8dda780773526103ab8ba82f421",
"tts/MiniCPM-o-4_5-tts-F16.gguf": "c7be3748a863dd6966ae7eed42600b7f41ca67affb03729ff245247f0e5ea088",
"tts/MiniCPM-o-4_5-projector-F16.gguf": "4b1b5b377358a5e594a304ff6ea5d52df606a9ba7d886c4299d232f0c67dd1fd",
"token2wav-gguf/encoder.gguf": "7f8d265da594eaf5e1de2db8f5f1867dbcb0bb75ef5878fadf2952347116f4d0",
"token2wav-gguf/flow_matching.gguf": "eda6069f3edeb5dd3a87fbf2aedb2ddd1b46f3273926c4fcf09b24476a39cab8",
"token2wav-gguf/flow_extra.gguf": "c67611aa7d02500fe395a7798bf0bfdfb55c74d37ba93934ca74d82b4e63f78d",
"token2wav-gguf/hifigan2.gguf": "1b8b3bf5d8d3066aeee4324fdcdd41aefce170d0ee907645858de408d82835c2",
"token2wav-gguf/prompt_cache.gguf": "81fe6f541ebe0b67db06a1e395df928da47991dc9637c2cf47d6c59d5b979f2c"
},
"cli_sha256": "29247187ddf3b13591cd14cc070e5e25c6bb7547296acc5e8a74e2945811d725",
"reference_sha256": "cf4bb5cd9acbfbf67596d4a53310652b61ef4b7ede1d54eb03ef4882ca40223a",
"input_sha256": {
"audio_wav": "14ee147cbf2ce9afd9e32c8c8228d7cfe11985cd7e63456b2ad33871492aefcc",
"image_jpeg": "22c7ccea466a3f551c512045e6316c7597408d4e0a20205502f55784d977aa8d"
},
"memory_budget": {
"capacities": {
"host_ram": 22548578304
},
"demands": {
"host_ram": 22548578304
}
},
"profile_count": 1,
"warmup_count": 0,
"host_available_bytes_before": 32275951616,
"startup_s": 51.80249010003172,
"execution_plan_at_start": {
"backend": "external.minicpmo.gguf.v1",
"stage_id": 0,
"worker_generation": "0c6e504609b74d2aaeafef78473eff20",
"model_revision": "db25077c33951fe163b42986fba0132e279872a2",
"artifact_sha256": {
"MiniCPM-o-4_5-Q4_K_M.gguf": "1237a97ee081b8abebc47aa7dad565701e8f5f904cdc92f6723ac4281bbc0932",
"audio/MiniCPM-o-4_5-audio-F16.gguf": "d5b188ac7feaf98e17175c3f9bd14bf269301bfd187439fdaa3e3a494fc32ef7",
"vision/MiniCPM-o-4_5-vision-F16.gguf": "1453678cc4e4fe18de241952962e234f265cb8dda780773526103ab8ba82f421",
"tts/MiniCPM-o-4_5-tts-F16.gguf": "c7be3748a863dd6966ae7eed42600b7f41ca67affb03729ff245247f0e5ea088",
"tts/MiniCPM-o-4_5-projector-F16.gguf": "4b1b5b377358a5e594a304ff6ea5d52df606a9ba7d886c4299d232f0c67dd1fd",
"token2wav-gguf/encoder.gguf": "7f8d265da594eaf5e1de2db8f5f1867dbcb0bb75ef5878fadf2952347116f4d0",
"token2wav-gguf/flow_matching.gguf": "eda6069f3edeb5dd3a87fbf2aedb2ddd1b46f3273926c4fcf09b24476a39cab8",
"token2wav-gguf/flow_extra.gguf": "c67611aa7d02500fe395a7798bf0bfdfb55c74d37ba93934ca74d82b4e63f78d",
"token2wav-gguf/hifigan2.gguf": "1b8b3bf5d8d3066aeee4324fdcdd41aefce170d0ee907645858de408d82835c2",
"token2wav-gguf/prompt_cache.gguf": "81fe6f541ebe0b67db06a1e395df928da47991dc9637c2cf47d6c59d5b979f2c"
},
"cli_sha256": "29247187ddf3b13591cd14cc070e5e25c6bb7547296acc5e8a74e2945811d725",
"requested_placement": "cpu",
"ggml_vk_visible_devices": null,
"shared_ram_reservation": {
"host_ram": 22548578304
},
"weights_bytes": 8872297088,
"memory_overhead_bytes": 12884901888,
"context_tokens": 2048,
"max_new_tokens": 96,
"request_capacity": 1,
"stateful_session": "C++-owned within one request; discarded after completion",
"placement_evidence": "pending first request"
},
"warmups": [],
"measured": [
{
"text": "The square in the image is red.",
"wall_s": 73.63217100000475,
"audio_metadata": {
"sample_rate_hz": 24000,
"channels": 1,
"pcm_frames": 47040,
"duration_s": 1.96,
"pcm_sha256": "f3b9a5511325f873216d17f91164b6bc076786a65f4ebe5baa1e6a22d7374591"
},
"stage_event": {
"request_id": "minicpmo-measured-0",
"stage_id": 0,
"epoch": 1,
"seq": 1,
"kind": "audio",
"worker_generation": "0c6e504609b74d2aaeafef78473eff20",
"buffers": [],
"terminal": true,
"error": null
},
"metrics": {
"minicpmo_whole_request_wall_s": 73.61201989999972,
"peak_sampled_rss_bytes": 13103185920,
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_current_cpu_logs\\minicpmo-1-d232d4fad9eb4ef9b2e3b58a1978f8cb.log"
}
}
],
"complete_request": {
"text": "The square in the image is red.",
"wall_s": 73.63217100000475,
"audio_metadata": {
"sample_rate_hz": 24000,
"channels": 1,
"pcm_frames": 47040,
"duration_s": 1.96,
"pcm_sha256": "f3b9a5511325f873216d17f91164b6bc076786a65f4ebe5baa1e6a22d7374591"
},
"stage_event": {
"request_id": "minicpmo-measured-0",
"stage_id": 0,
"epoch": 1,
"seq": 1,
"kind": "audio",
"worker_generation": "0c6e504609b74d2aaeafef78473eff20",
"buffers": [],
"terminal": true,
"error": null
},
"metrics": {
"minicpmo_whole_request_wall_s": 73.61201989999972,
"peak_sampled_rss_bytes": 13103185920,
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_current_cpu_logs\\minicpmo-1-d232d4fad9eb4ef9b2e3b58a1978f8cb.log"
}
},
"nearest_rank_p50_wall_s": 73.63217100000475,
"nearest_rank_p95_wall_s": 73.63217100000475,
"output_wav": "\\\\wsl.localhost\\Ubuntu\\home\\zhout\\project\\edge_infer\\vllm-omni-edge\\benchmarks\\edge_harness\\results\\e2e_expansion_20260923\\evidence\\minicpmo_omni_radeon_profile20\\cpu_control_current.wav",
"execution_plan_after_request": {
"backend": "external.minicpmo.gguf.v1",
"stage_id": 0,
"worker_generation": "0c6e504609b74d2aaeafef78473eff20",
"model_revision": "db25077c33951fe163b42986fba0132e279872a2",
"artifact_sha256": {
"MiniCPM-o-4_5-Q4_K_M.gguf": "1237a97ee081b8abebc47aa7dad565701e8f5f904cdc92f6723ac4281bbc0932",
"audio/MiniCPM-o-4_5-audio-F16.gguf": "d5b188ac7feaf98e17175c3f9bd14bf269301bfd187439fdaa3e3a494fc32ef7",
"vision/MiniCPM-o-4_5-vision-F16.gguf": "1453678cc4e4fe18de241952962e234f265cb8dda780773526103ab8ba82f421",
"tts/MiniCPM-o-4_5-tts-F16.gguf": "c7be3748a863dd6966ae7eed42600b7f41ca67affb03729ff245247f0e5ea088",
"tts/MiniCPM-o-4_5-projector-F16.gguf": "4b1b5b377358a5e594a304ff6ea5d52df606a9ba7d886c4299d232f0c67dd1fd",
"token2wav-gguf/encoder.gguf": "7f8d265da594eaf5e1de2db8f5f1867dbcb0bb75ef5878fadf2952347116f4d0",
"token2wav-gguf/flow_matching.gguf": "eda6069f3edeb5dd3a87fbf2aedb2ddd1b46f3273926c4fcf09b24476a39cab8",
"token2wav-gguf/flow_extra.gguf": "c67611aa7d02500fe395a7798bf0bfdfb55c74d37ba93934ca74d82b4e63f78d",
"token2wav-gguf/hifigan2.gguf": "1b8b3bf5d8d3066aeee4324fdcdd41aefce170d0ee907645858de408d82835c2",
"token2wav-gguf/prompt_cache.gguf": "81fe6f541ebe0b67db06a1e395df928da47991dc9637c2cf47d6c59d5b979f2c"
},
"cli_sha256": "29247187ddf3b13591cd14cc070e5e25c6bb7547296acc5e8a74e2945811d725",
"requested_placement": "cpu",
"ggml_vk_visible_devices": null,
"shared_ram_reservation": {
"host_ram": 22548578304
},
"weights_bytes": 8872297088,
"memory_overhead_bytes": 12884901888,
"context_tokens": 2048,
"max_new_tokens": 96,
"request_capacity": 1,
"stateful_session": "C++-owned within one request; discarded after completion",
"placement_evidence": {
"observed": "cpu",
"worker_pid": 29868,
"peak_sampled_rss_bytes": 13103185920,
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_current_cpu_logs\\minicpmo-1-d232d4fad9eb4ef9b2e3b58a1978f8cb.log"
}
},
"status": "passed",
"ledger_after_shutdown": {
"capacities": {
"host_ram": 22548578304
},
"reserved": {
"host_ram": 0
},
"owners": [],
"quarantined": []
}
}
benchmarks/edge_harness/results/e2e_expansion_20260923/evidence/minicpmo_omni_radeon_profile20/cpu_control_current.wav
BIN
View File
Binary file not shown.
benchmarks/edge_harness/results/e2e_expansion_20260923/evidence/minicpmo_omni_radeon_profile20/cpu_control_current_asr.json
+16
View File
@@ -0,0 +1,16 @@
{
"scope": "local Whisper tiny.en ASR proxy for generated speech; no perceptual quality claim",
"asr_model": "openai/whisper-tiny.en",
"asr_revision": "87c7102498dcde7456f24cfd30239ca606ed9063",
"asr_weights_sha256": "db59695928ded6043adaef491a53ef4e12da9611184d77c53baa691a60b958ad",
"transformers": "5.14.1",
"torch": "2.13.0+cpu",
"wav_sha256": "f0a45a85e6fe3619b14482619d9430a5184f6530efc55d08ad7c821d7ee54f2e",
"input_sample_rate": 24000,
"input_duration_s": 1.96,
"reference_text": "The square in the image is red.",
"transcript": "The square and the image is red.",
"word_error_rate": 0.14285714285714285,
"asr_load_s": 0.7989629450021312,
"asr_inference_s": 0.2657007460366003
}
benchmarks/edge_harness/results/e2e_expansion_20260923/evidence/minicpmo_omni_radeon_profile20/cpu_control_current_asr.log
+23
View File
@@ -0,0 +1,23 @@
W0924 03:14:14.503000 32407 torch/utils/_pytree.py:630] <enum 'KernelPreference'> is an Enum subclass and is now natively supported by torch.compile as an opaque value type. Calling register_constant() on Enum subclasses is deprecated and will be an error in a future release.
W0924 03:14:14.542000 32407 torch/utils/_pytree.py:630] <enum 'ScaleCalculationMode'> is an Enum subclass and is now natively supported by torch.compile as an opaque value type. Calling register_constant() on Enum subclasses is deprecated and will be an error in a future release.
␍Loading weights: 0%| | 0/167 [00:00<?, ?it/s]␍Loading weights: 59%|█████▉ | 99/167 [00:00<00:00, 983.56it/s]␍Loading weights: 100%|██████████| 167/167 [00:00<00:00, 1191.24it/s]
[transformers] Using custom `forced_decoder_ids` from the (generation) config. This is deprecated in favor of the `task` and `language` flags/config options.
[transformers] A custom logits processor of type <class 'transformers.generation.logits_process.SuppressTokensLogitsProcessor'> has been passed to `.generate()`, but it was also created in `.generate()`, given its parameterization. The custom <class 'transformers.generation.logits_process.SuppressTokensLogitsProcessor'> will take precedence. Please check the docstring of <class 'transformers.generation.logits_process.SuppressTokensLogitsProcessor'> to see related `.generate()` flags.
[transformers] A custom logits processor of type <class 'transformers.generation.logits_process.SuppressTokensAtBeginLogitsProcessor'> has been passed to `.generate()`, but it was also created in `.generate()`, given its parameterization. The custom <class 'transformers.generation.logits_process.SuppressTokensAtBeginLogitsProcessor'> will take precedence. Please check the docstring of <class 'transformers.generation.logits_process.SuppressTokensAtBeginLogitsProcessor'> to see related `.generate()` flags.
[transformers] Ignoring clean_up_tokenization_spaces=True for BPE tokenizer WhisperTokenizer. The clean_up_tokenization post-processing step is designed for WordPiece tokenizers and is destructive for BPE (it strips spaces before punctuation). Set clean_up_tokenization_spaces=False to suppress this warning, or set clean_up_tokenization_spaces_for_bpe_even_though_it_will_corrupt_output=True to force cleanup anyway.
{
"scope": "local Whisper tiny.en ASR proxy for generated speech; no perceptual quality claim",
"asr_model": "openai/whisper-tiny.en",
"asr_revision": "87c7102498dcde7456f24cfd30239ca606ed9063",
"asr_weights_sha256": "db59695928ded6043adaef491a53ef4e12da9611184d77c53baa691a60b958ad",
"transformers": "5.14.1",
"torch": "2.13.0+cpu",
"wav_sha256": "f0a45a85e6fe3619b14482619d9430a5184f6530efc55d08ad7c821d7ee54f2e",
"input_sample_rate": 24000,
"input_duration_s": 1.96,
"reference_text": "The square in the image is red.",
"transcript": "The square and the image is red.",
"word_error_rate": 0.14285714285714285,
"asr_load_s": 0.7989629450021312,
"asr_inference_s": 0.2657007460366003
}
benchmarks/edge_harness/results/e2e_expansion_20260923/evidence/minicpmo_omni_radeon_profile20/cpu_control_current_driver.log
+165
View File
@@ -0,0 +1,165 @@
INFO 09-24 03:10:48 [patch.py:259] NVFP4 W4A4 weight_scale NaN-clamp: installed.
INFO 09-24 03:10:49 [patch.py:510] inductor factorable-divisibility patch: installed.
INFO 09-24 03:10:52 [stage_init_utils.py:768] [stage_init] Set VLLM_WORKER_MULTIPROC_METHOD=spawn
WARNING 09-24 03:10:52 [stage_init_utils.py:1887] [stage_init] Failed to load transfer config: [Errno 13] Permission denied: 'C:\\Users\\zhout'
{
"scope": "MiniCPM-o GGUF complete audio+image\u2192text+speech through Omni StageRuntime/StagePool",
"placement": "cpu",
"model_revision": "db25077c33951fe163b42986fba0132e279872a2",
"cli_sha256": "29247187ddf3b13591cd14cc070e5e25c6bb7547296acc5e8a74e2945811d725",
"reference_sha256": "cf4bb5cd9acbfbf67596d4a53310652b61ef4b7ede1d54eb03ef4882ca40223a",
"input_sha256": {
"audio_wav": "14ee147cbf2ce9afd9e32c8c8228d7cfe11985cd7e63456b2ad33871492aefcc",
"image_jpeg": "22c7ccea466a3f551c512045e6316c7597408d4e0a20205502f55784d977aa8d"
},
"memory_budget": {
"capacities": {
"host_ram": 22548578304
},
"demands": {
"host_ram": 22548578304
}
},
"profile_count": 1,
"warmup_count": 0,
"host_available_bytes_before": 32275951616,
"startup_s": 51.80249010003172,
"execution_plan_at_start": {
"backend": "external.minicpmo.gguf.v1",
"stage_id": 0,
"worker_generation": "0c6e504609b74d2aaeafef78473eff20",
"model_revision": "db25077c33951fe163b42986fba0132e279872a2",
"artifact_sha256": {
"MiniCPM-o-4_5-Q4_K_M.gguf": "1237a97ee081b8abebc47aa7dad565701e8f5f904cdc92f6723ac4281bbc0932",
"audio/MiniCPM-o-4_5-audio-F16.gguf": "d5b188ac7feaf98e17175c3f9bd14bf269301bfd187439fdaa3e3a494fc32ef7",
"vision/MiniCPM-o-4_5-vision-F16.gguf": "1453678cc4e4fe18de241952962e234f265cb8dda780773526103ab8ba82f421",
"tts/MiniCPM-o-4_5-tts-F16.gguf": "c7be3748a863dd6966ae7eed42600b7f41ca67affb03729ff245247f0e5ea088",
"tts/MiniCPM-o-4_5-projector-F16.gguf": "4b1b5b377358a5e594a304ff6ea5d52df606a9ba7d886c4299d232f0c67dd1fd",
"token2wav-gguf/encoder.gguf": "7f8d265da594eaf5e1de2db8f5f1867dbcb0bb75ef5878fadf2952347116f4d0",
"token2wav-gguf/flow_matching.gguf": "eda6069f3edeb5dd3a87fbf2aedb2ddd1b46f3273926c4fcf09b24476a39cab8",
"token2wav-gguf/flow_extra.gguf": "c67611aa7d02500fe395a7798bf0bfdfb55c74d37ba93934ca74d82b4e63f78d",
"token2wav-gguf/hifigan2.gguf": "1b8b3bf5d8d3066aeee4324fdcdd41aefce170d0ee907645858de408d82835c2",
"token2wav-gguf/prompt_cache.gguf": "81fe6f541ebe0b67db06a1e395df928da47991dc9637c2cf47d6c59d5b979f2c"
},
"cli_sha256": "29247187ddf3b13591cd14cc070e5e25c6bb7547296acc5e8a74e2945811d725",
"requested_placement": "cpu",
"ggml_vk_visible_devices": null,
"shared_ram_reservation": {
"host_ram": 22548578304
},
"weights_bytes": 8872297088,
"memory_overhead_bytes": 12884901888,
"context_tokens": 2048,
"max_new_tokens": 96,
"request_capacity": 1,
"stateful_session": "C++-owned within one request; discarded after completion",
"placement_evidence": "pending first request"
},
"warmups": [],
"measured": [
{
"text": "The square in the image is red.",
"wall_s": 73.63217100000475,
"audio_metadata": {
"sample_rate_hz": 24000,
"channels": 1,
"pcm_frames": 47040,
"duration_s": 1.96,
"pcm_sha256": "f3b9a5511325f873216d17f91164b6bc076786a65f4ebe5baa1e6a22d7374591"
},
"stage_event": {
"request_id": "minicpmo-measured-0",
"stage_id": 0,
"epoch": 1,
"seq": 1,
"kind": "audio",
"worker_generation": "0c6e504609b74d2aaeafef78473eff20",
"buffers": [],
"terminal": true,
"error": null
},
"metrics": {
"minicpmo_whole_request_wall_s": 73.61201989999972,
"peak_sampled_rss_bytes": 13103185920,
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_current_cpu_logs\\minicpmo-1-d232d4fad9eb4ef9b2e3b58a1978f8cb.log"
}
}
],
"complete_request": {
"text": "The square in the image is red.",
"wall_s": 73.63217100000475,
"audio_metadata": {
"sample_rate_hz": 24000,
"channels": 1,
"pcm_frames": 47040,
"duration_s": 1.96,
"pcm_sha256": "f3b9a5511325f873216d17f91164b6bc076786a65f4ebe5baa1e6a22d7374591"
},
"stage_event": {
"request_id": "minicpmo-measured-0",
"stage_id": 0,
"epoch": 1,
"seq": 1,
"kind": "audio",
"worker_generation": "0c6e504609b74d2aaeafef78473eff20",
"buffers": [],
"terminal": true,
"error": null
},
"metrics": {
"minicpmo_whole_request_wall_s": 73.61201989999972,
"peak_sampled_rss_bytes": 13103185920,
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_current_cpu_logs\\minicpmo-1-d232d4fad9eb4ef9b2e3b58a1978f8cb.log"
}
},
"nearest_rank_p50_wall_s": 73.63217100000475,
"nearest_rank_p95_wall_s": 73.63217100000475,
"output_wav": "\\\\wsl.localhost\\Ubuntu\\home\\zhout\\project\\edge_infer\\vllm-omni-edge\\benchmarks\\edge_harness\\results\\e2e_expansion_20260923\\evidence\\minicpmo_omni_radeon_profile20\\cpu_control_current.wav",
"execution_plan_after_request": {
"backend": "external.minicpmo.gguf.v1",
"stage_id": 0,
"worker_generation": "0c6e504609b74d2aaeafef78473eff20",
"model_revision": "db25077c33951fe163b42986fba0132e279872a2",
"artifact_sha256": {
"MiniCPM-o-4_5-Q4_K_M.gguf": "1237a97ee081b8abebc47aa7dad565701e8f5f904cdc92f6723ac4281bbc0932",
"audio/MiniCPM-o-4_5-audio-F16.gguf": "d5b188ac7feaf98e17175c3f9bd14bf269301bfd187439fdaa3e3a494fc32ef7",
"vision/MiniCPM-o-4_5-vision-F16.gguf": "1453678cc4e4fe18de241952962e234f265cb8dda780773526103ab8ba82f421",
"tts/MiniCPM-o-4_5-tts-F16.gguf": "c7be3748a863dd6966ae7eed42600b7f41ca67affb03729ff245247f0e5ea088",
"tts/MiniCPM-o-4_5-projector-F16.gguf": "4b1b5b377358a5e594a304ff6ea5d52df606a9ba7d886c4299d232f0c67dd1fd",
"token2wav-gguf/encoder.gguf": "7f8d265da594eaf5e1de2db8f5f1867dbcb0bb75ef5878fadf2952347116f4d0",
"token2wav-gguf/flow_matching.gguf": "eda6069f3edeb5dd3a87fbf2aedb2ddd1b46f3273926c4fcf09b24476a39cab8",
"token2wav-gguf/flow_extra.gguf": "c67611aa7d02500fe395a7798bf0bfdfb55c74d37ba93934ca74d82b4e63f78d",
"token2wav-gguf/hifigan2.gguf": "1b8b3bf5d8d3066aeee4324fdcdd41aefce170d0ee907645858de408d82835c2",
"token2wav-gguf/prompt_cache.gguf": "81fe6f541ebe0b67db06a1e395df928da47991dc9637c2cf47d6c59d5b979f2c"
},
"cli_sha256": "29247187ddf3b13591cd14cc070e5e25c6bb7547296acc5e8a74e2945811d725",
"requested_placement": "cpu",
"ggml_vk_visible_devices": null,
"shared_ram_reservation": {
"host_ram": 22548578304
},
"weights_bytes": 8872297088,
"memory_overhead_bytes": 12884901888,
"context_tokens": 2048,
"max_new_tokens": 96,
"request_capacity": 1,
"stateful_session": "C++-owned within one request; discarded after completion",
"placement_evidence": {
"observed": "cpu",
"worker_pid": 29868,
"peak_sampled_rss_bytes": 13103185920,
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_current_cpu_logs\\minicpmo-1-d232d4fad9eb4ef9b2e3b58a1978f8cb.log"
}
},
"status": "passed",
"ledger_after_shutdown": {
"capacities": {
"host_ram": 22548578304
},
"reserved": {
"host_ram": 0
},
"owners": [],
"quarantined": []
}
}
benchmarks/edge_harness/results/e2e_expansion_20260923/evidence/minicpmo_omni_radeon_profile20/cpu_control_worker.log
+470
View File
@@ -0,0 +1,470 @@
build: 64 (5202b7b) with x86_64-w64-mingw32-gcc-posix (GCC) 13-posix for x86_64-w64-mingw32
llama_model_loader: loaded meta data with 27 key-value pairs and 399 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv 0: general.architecture str = qwen3
llama_model_loader: - kv 1: general.type str = model
llama_model_loader: - kv 2: general.name str = Llm
llama_model_loader: - kv 3: general.size_label str = 8.2B
llama_model_loader: - kv 4: qwen3.block_count u32 = 36
llama_model_loader: - kv 5: qwen3.context_length u32 = 40960
llama_model_loader: - kv 6: qwen3.embedding_length u32 = 4096
llama_model_loader: - kv 7: qwen3.feed_forward_length u32 = 12288
llama_model_loader: - kv 8: qwen3.attention.head_count u32 = 32
llama_model_loader: - kv 9: qwen3.attention.head_count_kv u32 = 8
llama_model_loader: - kv 10: qwen3.rope.freq_base f32 = 1000000.000000
llama_model_loader: - kv 11: qwen3.attention.layer_norm_rms_epsilon f32 = 0.000001
llama_model_loader: - kv 12: qwen3.attention.key_length u32 = 128
llama_model_loader: - kv 13: qwen3.attention.value_length u32 = 128
llama_model_loader: - kv 14: tokenizer.ggml.model str = gpt2
llama_model_loader: - kv 15: tokenizer.ggml.pre str = qwen2
llama_model_loader: - kv 16: tokenizer.ggml.tokens arr[str,151748] = ["!", "\"", "#", "$", "%", "&", "'", ...
llama_model_loader: - kv 17: tokenizer.ggml.token_type arr[i32,151748] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv 18: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
llama_model_loader: - kv 19: tokenizer.ggml.bos_token_id u32 = 151644
llama_model_loader: - kv 20: tokenizer.ggml.eos_token_id u32 = 151645
llama_model_loader: - kv 21: tokenizer.ggml.unknown_token_id u32 = 128244
llama_model_loader: - kv 22: tokenizer.ggml.padding_token_id u32 = 151643
llama_model_loader: - kv 23: tokenizer.ggml.add_bos_token bool = false
llama_model_loader: - kv 24: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
llama_model_loader: - kv 25: general.quantization_version u32 = 2
llama_model_loader: - kv 26: general.file_type u32 = 15
llama_model_loader: - type f32: 145 tensors
llama_model_loader: - type q4_K: 217 tensors
llama_model_loader: - type q6_K: 37 tensors
print_info: file format = GGUF V3 (latest)
print_info: file type = Q4_K - Medium
print_info: file size = 4.68 GiB (4.90 BPW)
load: printing all EOG tokens:
load: - 151643 ('<|endoftext|>')
load: - 151645 ('<|im_end|>')
load: - 151662 ('<|fim_pad|>')
load: - 151663 ('<|repo_name|>')
load: - 151664 ('<|file_sep|>')
load: special tokens cache size = 106
load: token to piece cache size = 0.9319 MB
print_info: arch = qwen3
print_info: vocab_only = 0
print_info: n_ctx_train = 40960
print_info: n_embd = 4096
print_info: n_layer = 36
print_info: n_head = 32
print_info: n_head_kv = 8
print_info: n_rot = 128
print_info: n_swa = 0
print_info: is_swa_any = 0
print_info: n_embd_head_k = 128
print_info: n_embd_head_v = 128
print_info: n_gqa = 4
print_info: n_embd_k_gqa = 1024
print_info: n_embd_v_gqa = 1024
print_info: f_norm_eps = 0.0e+00
print_info: f_norm_rms_eps = 1.0e-06
print_info: f_clamp_kqv = 0.0e+00
print_info: f_max_alibi_bias = 0.0e+00
print_info: f_logit_scale = 0.0e+00
print_info: f_attn_scale = 0.0e+00
print_info: n_ff = 12288
print_info: n_expert = 0
print_info: n_expert_used = 0
print_info: causal attn = 1
print_info: pooling type = -1
print_info: rope type = 2
print_info: rope scaling = linear
print_info: freq_base_train = 1000000.0
print_info: freq_scale_train = 1
print_info: n_ctx_orig_yarn = 40960
print_info: rope_finetuned = unknown
print_info: model type = 8B
print_info: model params = 8.19 B
print_info: general.name = Llm
print_info: vocab type = BPE
print_info: n_vocab = 151748
print_info: n_merges = 151387
print_info: BOS token = 151644 '<|im_start|>'
print_info: EOS token = 151645 '<|im_end|>'
print_info: EOT token = 151645 '<|im_end|>'
print_info: UNK token = 128244 '<unk>'
print_info: PAD token = 151643 '<|endoftext|>'
print_info: LF token = 198 '膴'
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
print_info: FIM MID token = 151660 '<|fim_middle|>'
print_info: FIM PAD token = 151662 '<|fim_pad|>'
print_info: FIM REP token = 151663 '<|repo_name|>'
print_info: FIM SEP token = 151664 '<|file_sep|>'
print_info: EOG token = 151643 '<|endoftext|>'
print_info: EOG token = 151645 '<|im_end|>'
print_info: EOG token = 151662 '<|fim_pad|>'
print_info: EOG token = 151663 '<|repo_name|>'
print_info: EOG token = 151664 '<|file_sep|>'
print_info: max token length = 256
load_tensors: loading model tensors, this can take a while... (mmap = true)
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
load_tensors: CPU_REPACK model buffer size = 3199.50 MiB
load_tensors: CPU_Mapped model buffer size = 4761.17 MiB
.....................................................................................
llama_context: constructing llama_context
llama_context: n_seq_max = 1
llama_context: n_ctx = 2048
llama_context: n_ctx_per_seq = 2048
llama_context: n_batch = 2048
llama_context: n_ubatch = 512
llama_context: causal_attn = 1
llama_context: flash_attn = auto
llama_context: kv_unified = false
llama_context: freq_base = 1000000.0
llama_context: freq_scale = 1
llama_context: n_ctx_per_seq (2048) < n_ctx_train (40960) -- the full capacity of the model will not be utilized
llama_context: CPU output buffer size = 0.58 MiB
llama_kv_cache: CPU KV buffer size = 288.00 MiB
llama_kv_cache: size = 288.00 MiB ( 2048 cells, 36 layers, 1/1 seqs), K (f16): 144.00 MiB, V (f16): 144.00 MiB
llama_context: Flash Attention was auto, set to enabled
llama_context: CPU compute buffer size = 304.38 MiB
llama_context: graph nodes = 1267
llama_context: graph splits = 1
llama_model_loader: loaded meta data with 30 key-value pairs and 193 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv 0: general.architecture str = llama
llama_model_loader: - kv 1: general.type str = model
llama_model_loader: - kv 2: general.name str = Tts
llama_model_loader: - kv 3: general.size_label str = 445M
llama_model_loader: - kv 4: llama.block_count u32 = 20
llama_model_loader: - kv 5: llama.context_length u32 = 4096
llama_model_loader: - kv 6: llama.embedding_length u32 = 768
llama_model_loader: - kv 7: llama.feed_forward_length u32 = 3072
llama_model_loader: - kv 8: llama.attention.head_count u32 = 12
llama_model_loader: - kv 9: llama.attention.head_count_kv u32 = 12
llama_model_loader: - kv 10: llama.rope.freq_base f32 = 10000.000000
llama_model_loader: - kv 11: llama.attention.layer_norm_rms_epsilon f32 = 0.000001
llama_model_loader: - kv 12: llama.attention.key_length u32 = 64
llama_model_loader: - kv 13: llama.attention.value_length u32 = 64
llama_model_loader: - kv 14: general.file_type u32 = 1
llama_model_loader: - kv 15: llama.vocab_size u32 = 158626
llama_model_loader: - kv 16: llama.rope.dimension_count u32 = 64
llama_model_loader: - kv 17: general.quantization_version u32 = 2
llama_model_loader: - kv 18: tokenizer.ggml.model str = gpt2
llama_model_loader: - kv 19: tokenizer.ggml.pre str = qwen2
llama_model_loader: - kv 20: tokenizer.ggml.tokens arr[str,158626] = ["!", "\"", "#", "$", "%", "&", "'", ...
llama_model_loader: - kv 21: tokenizer.ggml.token_type arr[i32,158626] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv 22: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
llama_model_loader: - kv 23: tokenizer.ggml.bos_token_id u32 = 151644
llama_model_loader: - kv 24: tokenizer.ggml.eos_token_id u32 = 151645
llama_model_loader: - kv 25: tokenizer.ggml.unknown_token_id u32 = 128244
llama_model_loader: - kv 26: tokenizer.ggml.padding_token_id u32 = 151643
llama_model_loader: - kv 27: tokenizer.ggml.add_bos_token bool = false
llama_model_loader: - kv 28: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
llama_model_loader: - kv 29: tokenizer.ggml.add_space_prefix bool = false
llama_model_loader: - type f32: 50 tensors
llama_model_loader: - type f16: 143 tensors
print_info: file format = GGUF V3 (latest)
print_info: file type = F16
print_info: file size = 1.07 GiB (20.70 BPW)
load: printing all EOG tokens:
load: - 151643 ('<|endoftext|>')
load: - 151645 ('<|im_end|>')
load: - 151662 ('<|fim_pad|>')
load: - 151663 ('<|repo_name|>')
load: - 151664 ('<|file_sep|>')
load: special tokens cache size = 106
load: token to piece cache size = 0.9319 MB
print_info: arch = llama
print_info: vocab_only = 0
print_info: n_ctx_train = 4096
print_info: n_embd = 768
print_info: n_layer = 20
print_info: n_head = 12
print_info: n_head_kv = 12
print_info: n_rot = 64
print_info: n_swa = 0
print_info: is_swa_any = 0
print_info: n_embd_head_k = 64
print_info: n_embd_head_v = 64
print_info: n_gqa = 1
print_info: n_embd_k_gqa = 768
print_info: n_embd_v_gqa = 768
print_info: f_norm_eps = 0.0e+00
print_info: f_norm_rms_eps = 1.0e-06
print_info: f_clamp_kqv = 0.0e+00
print_info: f_max_alibi_bias = 0.0e+00
print_info: f_logit_scale = 0.0e+00
print_info: f_attn_scale = 0.0e+00
print_info: n_ff = 3072
print_info: n_expert = 0
print_info: n_expert_used = 0
print_info: causal attn = 1
print_info: pooling type = 0
print_info: rope type = 0
print_info: rope scaling = linear
print_info: freq_base_train = 10000.0
print_info: freq_scale_train = 1
print_info: n_ctx_orig_yarn = 4096
print_info: rope_finetuned = unknown
print_info: model type = ?B
print_info: model params = 444.94 M
print_info: general.name = Tts
print_info: vocab type = BPE
print_info: n_vocab = 158626
print_info: n_merges = 151387
print_info: BOS token = 151644 '<|im_start|>'
print_info: EOS token = 151645 '<|im_end|>'
print_info: EOT token = 151645 '<|im_end|>'
print_info: UNK token = 128244 '<unk>'
print_info: PAD token = 151643 '<|endoftext|>'
print_info: LF token = 198 '膴'
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
print_info: FIM MID token = 151660 '<|fim_middle|>'
print_info: FIM PAD token = 151662 '<|fim_pad|>'
print_info: FIM REP token = 151663 '<|repo_name|>'
print_info: FIM SEP token = 151664 '<|file_sep|>'
print_info: EOG token = 151643 '<|endoftext|>'
print_info: EOG token = 151645 '<|im_end|>'
print_info: EOG token = 151662 '<|fim_pad|>'
print_info: EOG token = 151663 '<|repo_name|>'
print_info: EOG token = 151664 '<|file_sep|>'
print_info: max token length = 256
load_tensors: loading model tensors, this can take a while... (mmap = true)
load_tensors: skipping TTS tensor emb_code.0.weight (size = 20158464 bytes) -- will be loaded separately
load_tensors: skipping TTS tensor emb_text.weight (size = 467140608 bytes) -- will be loaded separately
load_tensors: skipping TTS tensor projector_semantic.linear1.weight (size = 12582912 bytes) -- will be loaded separately
load_tensors: skipping TTS tensor projector_semantic.linear1.bias (size = 3072 bytes) -- will be loaded separately
load_tensors: skipping TTS tensor projector_semantic.linear2.weight (size = 2359296 bytes) -- will be loaded separately
load_tensors: skipping TTS tensor projector_semantic.linear2.bias (size = 3072 bytes) -- will be loaded separately
load_tensors: skipping TTS tensor projector_spk.linear1.weight (size = 6291456 bytes) -- will be loaded separately
load_tensors: skipping TTS tensor projector_spk.linear1.bias (size = 3072 bytes) -- will be loaded separately
load_tensors: skipping TTS tensor projector_spk.linear2.weight (size = 1179648 bytes) -- will be loaded separately
load_tensors: skipping TTS tensor projector_spk.linear2.bias (size = 3072 bytes) -- will be loaded separately
load_tensors: skipping TTS tensor head_code.0.weight (size = 20158464 bytes) -- will be loaded separately
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
load_tensors: CPU_Mapped model buffer size = 592.48 MiB
..............................................................
llama_init_from_model: model default pooling_type is [0], but [-1] was specified
llama_context: constructing llama_context
llama_context: n_seq_max = 1
llama_context: n_ctx = 2048
llama_context: n_ctx_per_seq = 2048
llama_context: n_batch = 2048
llama_context: n_ubatch = 512
llama_context: causal_attn = 1
llama_context: flash_attn = auto
llama_context: kv_unified = false
llama_context: freq_base = 10000.0
llama_context: freq_scale = 1
llama_context: n_ctx_per_seq (2048) < n_ctx_train (4096) -- the full capacity of the model will not be utilized
llama_context: CPU output buffer size = 0.61 MiB
llama_kv_cache: CPU KV buffer size = 120.00 MiB
llama_kv_cache: size = 120.00 MiB ( 2048 cells, 20 layers, 1/1 seqs), K (f16): 60.00 MiB, V (f16): 60.00 MiB
llama_context: Flash Attention was auto, set to enabled
llama_context: CPU compute buffer size = 25.51 MiB
llama_context: graph nodes = 613
llama_context: graph splits = 1
vision_model_loader: model name:
vision_model_loader: description: image encoder for MiniCPM-V
vision_model_loader: GGUF version: 3
vision_model_loader: alignment: 32
vision_model_loader: n_tensors: 455
vision_model_loader: n_kv: 20
vision_ctx: vision using CPU backend
load_hparams: model_type: 0
load_hparams: n_embd: 1152
load_hparams: n_head: 16
load_hparams: n_ff: 4304
load_hparams: n_layer: 27
load_hparams: ffn_op: gelu
load_hparams: projection_dim: 0
--- vision hparams ---
load_hparams: image_size: 448
load_hparams: patch_size: 14
load_hparams: minicpmv_version: 100045
load_hparams: proj_scale_factor: 0
load_hparams: n_wa_pattern: 0
load_hparams: model size: 1044.36 MiB
load_hparams: metadata size: 0.16 MiB
load_tensors: ffn up/down are swapped
alloc_compute_meta: CPU compute buffer size = 100.30 MiB
init tts....init t2w....voc_hg2_model: CPU backend using 8 threads
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=32000
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
audition_audio_preprocess: Mel spectrogram ready - n_len=200, n_mel=80, total_size=16000
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
audition_audio_batch_encode: Audio input data - n_step=200, n_mel=80, total_elements=16000
audition_audio_batch_encode: Before incrementing iter: input_frames=200, n_tokens=100, current_iter=0, current_total=0, new_total=100, max_tokens=1500
audition_audio_batch_encode: KV cache iter incremented to 1 (total_tokens=100, max_tokens=1500)
audition_audio_batch_encode: Final output shape: [4096, 20]
audition_audio_batch_encode: Final output: 20 tokens x 4096 dims
=== Model Paths ===
Base dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf
LLM: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf [OK]
Vision: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/vision/MiniCPM-o-4_5-vision-F16.gguf [OK]
Audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf [OK]
TTS: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf [OK]
Projector: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf [OK]
===================
=== Initializing Omni Context ===
Media type: 2 (omni: audio+vision)
TTS enabled: yes
Context size: 2048
GPU layers: 0
Token2Wav device: cpu
Vision backend: metal
TTS bin dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts
Ref audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\vllm-omni-edge\benchmarks\edge_harness\results\e2e_expansion_20260923\evidence\minicpmo_cpp_radeon890m_prompted\fixture_0000.wav
03:11:44.461 === omni_init start
03:11:44.462 media_type = 2, duplex_mode = 0, base_output_dir = ./tools/omni/output
03:11:44.462 === omni_init: loading new LLM model
03:12:14.668 === omni_init: loading TTS model
03:12:14.668 TTS model: loading with n_gpu_layers=0
03:12:18.151 TTS sampler: temp=0.80, top_p=0.85, top_k=25, rep_penalty=1.05
03:12:18.151 TTS: loading weights from GGUF (emb_code, emb_text, projector_semantic, head_code)...
03:12:23.703 TTS: head_code shape: dim0=6562, dim1=768, will transpose to [6562, 768]
03:12:23.709 TTS: head_code loaded, verifying first few values:
03:12:23.709 head_code_weight[0] = 0.01361084 (expect ~0.01385498)
03:12:23.709 head_code_weight[1] = -0.01660156 (expect ~-0.01647949)
03:12:23.709 head_code_weight[768] = 0.04101562 (this is weight[1, 0], expect ~0.04150391)
03:12:23.762 TTS: weights loaded successfully
03:12:23.762 Projector: loading from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf
03:12:23.868 Projector: loaded successfully
03:12:23.868 === omni_init: loading APM model
03:12:26.939 APM: init from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf
03:12:31.896 Token2Wav: found models in \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf
03:12:31.950 Token2Wav: all model files found, initializing session...
03:12:31.950 Token2Wav: non-CUDA backend, vocoder using CPU for better performance
03:12:31.950 Token2Wav: using prompt_cache from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf/prompt_cache.gguf
03:12:36.461 Token2Wav: initialized successfully
03:12:36.461 Python T2W: script_dir=./tools/omni/pyt2w, model_dir=./tools/omni/pyt2w/token2wav
03:12:36.461 Token2Wav: 浣跨敤 C++ 瀹炵幇
03:12:36.461 === omni_init success: ctx_llama = 00000196917e6700
=== Running test case ===
Audio prefix: C:\Users\zhout\w2\minicpmo_omni_current_cpu_work\omni-minicpmo-1-x38wkip1\fixture_
Count: 2
03:12:36.461 stream_prefill: n_past = 0
voice_clone_prompt = <|im_start|>system
妯′豢闊抽�鏍锋湰鐨勯煶鑹插苟鐢熸垚鏂扮殑鍐呭�銆�
<|audio_start|>
assistant_prompt = <|audio_end|>浣犵殑浠诲姟鏄�敤杩欑�澹伴煶妯″紡鏉ュ綋涓€涓�姪鎵嬨€傝�璁ょ湡銆侀珮璐ㄩ噺鍦板洖澶嶇敤鎴风殑闂��銆傝�鐢ㄩ珮鑷�劧搴︾殑鏂瑰紡鍜岀敤鎴疯亰澶┿€�<|im_end|>
<|im_start|>user
03:12:36.461 system prompt ref_audio: C:\Users\zhout\w2\minicpmo_omni_current_cpu_work\omni-minicpmo-1-x38wkip1\fixture_0000.wav
03:12:40.425 system prompt ref_audio embedding: n_pos=20
03:12:42.645 馃敀 n_keep 璁剧疆涓� 72 (system prompt tokens)锛岃繖閮ㄥ垎姘歌繙涓嶄細琚�粦鍔ㄧ獥鍙ｅ垹闄�
03:12:42.645 stream_prefill(index=0): system prompt 鍒濆�鍖栧畬鎴愶紝ref_audio 宸插湪鍏朵腑 prefill
03:12:42.645 [SW] system_prompt registered: preserve_length=72 (will be protected from sliding)
03:12:42.645 n_past = 72
03:12:42.645
c++ finish stream_prefill(index=0). n_past=72, n_keep=72, n_ctx=2048
prefill 0 (audio) : 6.1837 s
encode_image_with_vision_chunks: step 1 of 1 encoded in 2927.74 ms
encode_image_with_vision_chunks: all 1 chunks encoded in 2945.85 ms (grid: 1x1)
encode_image_with_vision_chunks: image encoded in 2945.86 ms by vision ( 2945.86 ms per chunk, 64 total tokens)
omni_image_embed_make_chunks_with_filename: created 1 vision chunks from C:\Users\zhout\w2\minicpmo_omni_current_cpu_work\omni-minicpmo-1-x38wkip1\fixture_0001.jpg
stream_prefill: prefilled 1 vision chunks (64 tokens each)
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=49339
audition_audio_preprocess: Audio not aligned to 100ms (49339 samples = 3083.7ms), padded with silence to 49600 samples (3100ms)
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
audition_audio_preprocess: Mel spectrogram ready - n_len=310, n_mel=80, total_size=24800
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
audition_audio_batch_encode: Audio input data - n_step=310, n_mel=80, total_elements=24800
audition_audio_batch_encode: Before incrementing iter: input_frames=310, n_tokens=155, current_iter=1, current_total=155, new_total=310, max_tokens=1500
audition_audio_batch_encode: KV cache iter incremented to 2 (total_tokens=310, max_tokens=1500)
audition_audio_batch_encode: Final output shape: [4096, 31]
audition_audio_batch_encode: Final output: 31 tokens x 4096 dims
03:12:48.423 stream_prefill(index=1): processing user audio: C:\Users\zhout\w2\minicpmo_omni_current_cpu_work\omni-minicpmo-1-x38wkip1\fixture_0001.wav
03:12:48.712 stream_prefill(index=1): user audio embedding: n_pos=31
03:12:50.226
c++ finish stream_prefill(index=1). n_past=172, n_keep=72, n_ctx=2048
prefill 1 (audio+vision) : 7.57891 s
03:12:50.226 馃搷 stream_decode 寮€濮�: n_past=172, n_keep=72, n_ctx=2048, duplex_mode=0
03:12:50.226 stream_decode: create tts thread (simplex mode)
03:12:50.226 stream_decode: create t2w thread
03:12:50.226 wait prefill done
03:12:50.226 馃搷 [鍗曞伐TTS] 娣诲姞 assistant prompt: "<|im_end|>
<|im_start|>assistant
<think>
</think>
<|tts_bos|>", n_past=172
03:12:50.226 T2W thread (C++) started
<user>
<assistant>03:12:50.229 TTS: 鍒涘缓鍗曞伐妯″紡杈撳嚭鐩�綍: ./tools/omni/output/round_000
03:12:50.230 TTS thread started
03:12:50.694 馃搷 [鍗曞伐TTS] assistant prompt 瀹屾垚, n_past=182
03:12:50.694 LLM decode: max_tgt_len = 96, n_predict = 96, n_ctx = 2048
03:12:51.498 LLM: detected end token, set llm_generation_done=true
03:12:51.498 LLM->TTS: text='The square in the image is red.', n_tokens=8, hidden_size=32768, n_embd=4096, token_ids=[785 9334 304 279 2168 374 2518 13]
03:12:51.498 馃搷 Decode 缁撴潫: n_past=191, 鍓╀綑绌洪棿=1857, 鏃犻渶婊戠獥
03:12:51.498 馃搷 杞�� 1 缁撴潫锛岃�褰曡竟鐣屼簬 n_past=191
03:12:51.498 TTS<-LLM: chunk_idx=0, text='The square in the image is red.', n_tokens=8, hidden_size=32768, token_ids=[785 9334 304 279 2168 374 2518 13]
03:12:51.498 TTS: after queue pop - speek_done=0, llm_finish=1, llm_text.empty=0, token_ids.size=8
03:12:51.498 TTS: before empty check - response.empty=0, response='The square in the image is red.', llm_finish=1
Waiting for audio generation to complete...
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_0.wav does not exist or is empty
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_1.wav does not exist or is empty
TTS: no valid WAV files to merge
03:12:51.498 TTS: DEBUG before has_llm_data - token_ids.size=8, hidden_states.size=32768, n_embd=4096
03:12:51.498 TTS: has_llm_data=1, token_ids=8, hidden_states=32768, n_embd=4096, llm_finish=1
03:12:51.499 TTS: n_tokens_orig=8, n_tokens_filtered=8 (filtered 0 special tokens)
03:12:51.540 TTS: saved merged embeddings (text) to ./tools/omni/output/round_000/llm_debug/chunk_0/merged_embeddings.txt
03:12:51.540 TTS Simplex: generating audio tokens for chunk 0 (n_tokens=8, tts_n_embd=768)
03:12:51.540 TTS Simplex: 鍦� prefill 鍓嶆坊鍔� audio_bos (chunk_idx=0, new_size=9)
03:12:51.540 TTS Simplex: first chunk - cleared KV cache
03:12:51.540 TTS Simplex: first chunk - cleared tts_all_generated_tokens and tts_token_buffer
03:12:51.797 馃搷 涓轰笅涓€杞�噯澶�: eval <|im_end|>\n<|im_start|>user\n, n_past=196
03:12:51.895 TTS Simplex: prefill completed, n_past_tts=9
03:12:51.895 TTS Simplex: sampler created
03:12:51.895 TTS simplex: is_audio_bos=true (first audio token)
03:12:51.920 TTS Simplex Phase1: token 1/500: rel_id=758
03:12:51.931 TTS Simplex Phase1: token 2/500: rel_id=1950
03:12:51.941 TTS Simplex Phase1: token 3/500: rel_id=4137
03:12:51.952 TTS Simplex Phase1: token 4/500: rel_id=4218
03:12:51.962 TTS Simplex Phase1: token 5/500: rel_id=4218
03:12:52.175 TTS Simplex Phase1: yield 25 tokens 鍒� T2W (step 25, buffer=25)
03:12:52.175 TTS Simplex Phase1: token 25/500: rel_id=4913
03:12:52.175 T2W绾跨▼(C++): 杞��鍒囨崲 (-1 -> 0)锛屾洿鏂拌緭鍑虹洰褰�
03:12:52.175 T2W绾跨▼(C++): 鏂拌緭鍑虹洰褰� ./tools/omni/output/round_000/tts_wav
03:12:52.307 TTS Simplex Phase1: EOS token at step 38
03:12:52.307 TTS Simplex: is_final_text_chunk=true, will enter Phase 2 for text_eos_embed
03:12:52.307 TTS Simplex Phase2: injecting text_eos_embed + audio_bos at n_past=46
03:12:52.314 TTS Simplex Phase2: text_eos_embed injected, n_past=47
03:12:52.321 TTS Simplex Phase2: audio_bos injected, n_past=48, starting final generation
03:12:52.427 TTS Simplex Phase2: EOS at step 10 (final end)
03:12:52.427 TTS Simplex: chunk 缁撴潫锛宼ts_token_buffer 鍓╀綑 21 tokens (淇濈暀绛変笅涓€涓� chunk)
03:12:52.427 TTS Simplex: is_final_text_chunk=true, flush 鍓╀綑 21 tokens 浠� tts_token_buffer
03:12:52.427 TTS Simplex: generated 46 audio tokens for chunk 0
03:12:52.427 TTS Simplex: chunk 0 first tokens: [758 1950 4137 4218 4218 ], last tokens: [139 57 878 729 1944 ]
03:12:52.427 TTS Simplex: updated tts_n_past_accumulated=58, total_generated_tokens=46
03:12:52.437 TTS: finished processing all chunks
03:12:52.437 TTS: 鍗曞伐妯″紡杞��缁撴潫锛屼笅涓€杞� round_idx=1
03:12:52.437 TTS: sent is_final=true to T2W queue (turn end)
03:12:54.480 馃帀 棣栧搷鏃堕棿 (First Audio Response): 4253ms
03:12:54.480 T2W绾跨▼: wav_0.wav | 0.84s audio | 2304.0ms inference | RTF=2.74 | t=4253ms
03:12:56.692 T2W绾跨▼: wav_1.wav | 1.12s audio | 2211.3ms inference | RTF=1.97 | t=6465ms
03:12:56.692 T2W绾跨▼: 鍐欏叆缁撴潫鏍囪� ./tools/omni/output/round_000/tts_wav/generation_done.flag (last_wav=1)
03:12:56.692 T2W绾跨▼: 杞��缁撴潫鍚庢洿鏂拌緭鍑虹洰褰曚负 ./tools/omni/output/round_001/tts_wav
03:12:56.741 omni_stop_threads: stop signals sent
03:12:56.741 T2W(C++) 绾跨▼: 鍋滄�
[timing] call=0 tokens=28 final=0 token2mel=1966.463ms vocoder=337.463ms total=2303.954ms audio=20160
[timing] call=1 tokens=24 final=1 token2mel=1839.495ms vocoder=371.756ms total=2211.262ms audio=26880
Audio generation completed.
llama_perf_context_print: load time = 36025.14 ms
llama_perf_context_print: prompt eval time = 14231.79 ms / 182 tokens ( 78.20 ms per token, 12.79 tokens per second)
llama_perf_context_print: eval time = 801.78 ms / 9 runs ( 89.09 ms per token, 11.23 tokens per second)
llama_perf_context_print: total time = 42072.15 ms / 191 tokens
llama_perf_context_print: graphs reused = 10
Token2Wav (C++): session released
tts thread end
t2w thread end
benchmarks/edge_harness/results/e2e_profiling_20260922/evidence/summary/README.md
+4 -4
View File
@@ -17,10 +17,10 @@ NOT E2E means no complete device/model workload was verified. “Scoped E2E” i
| Device | Spark-X2.5 | Qwen3-TTS 0.6B CustomVoice | Qwen3.8-27B | MiniCPM-o 4.5 | InternVLA-A1 |
|---|---|---|---|---|---|
| PC HX370: CPU / WSL | profile completed | scoped E2E: Omni CPU text → complete WAV | NOT E2E | scoped E2E: Omni GGUF combined image+audio → text+speech | synthetic policy pass: Omni CPU |
| PC HX370: CPU / native Windows | scoped E2E: Omni Spark text | scoped E2E: Omni CPU text → complete WAV | NOT E2E | scoped E2E: Omni GGUF image+audio → text+speech | synthetic policy pass: Omni CPU |
| PC HX370: CPU / native Windows | scoped E2E: Omni Spark text | scoped E2E: Omni CPU text → complete WAV | NOT E2E | scoped E2E: Omni GGUF image+audio → text+speech (unstable latency) | synthetic policy pass: Omni CPU |
| PC HX370 + RTX 5090 Laptop / WSL | profile completed | profile completed | scoped E2E: text + image | scoped E2E: text/image/audio → speech | synthetic policy pass |
| PC HX370 + RTX 5090 Laptop / Windows | profile completed | profile completed | NOT E2E | scoped E2E: text/image/audio → speech | synthetic policy pass |
| PC HX370 + Radeon 890M / Windows worker | scoped E2E: Omni Spark text | scoped E2E: Omni Radeon talker/codec + CPU predictor | NOT E2E | scoped E2E: Omni Radeon+CPU image+audio → text+speech | synthetic policy pass: Omni CPU+Radeon Cosmos (WSL/native Windows) |
| PC HX370 + Radeon 890M / Windows worker | scoped E2E: Omni Spark text | scoped E2E: Omni Radeon talker/codec + CPU predictor | NOT E2E | scoped E2E: Omni Radeon+CPU image+audio → text+speech (unstable latency) | synthetic policy pass: Omni CPU+Radeon Cosmos (WSL/native Windows) |
| PC HX370 + AMD NPU / Windows worker | NOT E2E (output-head component passed) | NOT E2E | NOT E2E | NOT E2E | NOT E2E |
| PC CPU+iGPU+NPU +/- discrete GPU, joint model execution | NOT E2E | NOT E2E | NOT E2E | NOT E2E | NOT E2E |
| PC Snapdragon X Elite CRD / AI Hub | NOT E2E | NOT E2E | NOT E2E | NOT E2E | NOT E2E |
@@ -41,7 +41,7 @@ NOT E2E means no complete device/model workload was verified. “Scoped E2E” i
| PC HX370: CPU / native Windows | Spark-X2.5 | SCOPED_E2E_OMNI_COMPLETE_REQUEST | The pinned Spark-X2.5-1.7B Q4_K_M GGUF ran in native Windows llama.cpp CPU (`-dev none -ngl 0`) as a whole-session Omni StageRuntime/StagePool backend. `Paris` and the correct `1511` inventory answer passed; one warmup plus 20 serial complete requests were 20/20 correct, nearest-rank wall p50/p95 6.93/7.05 s. The public AsyncOmni short request passed. The stage verifies artifact hashes and CPU placement (0/29 layers offloaded), reserves 4 GiB host RAM, emits a terminal text event and releases reservation after in-flight cancellation. The separate prior standalone profile was 6.65/6.86 s. No vLLM CPU attention repair, incremental stream, restart-after-cancel, loading peak or sustained power/thermal gate is claimed. | Validate broader text/token quality, streaming and cancellation recovery, memory peak and sustained load; [Omni raw evidence](../../../e2e_expansion_20260923/evidence/spark_omni_llamacpp/README.md), [standalone evidence](../../../e2e_expansion_20260923/evidence/spark_radeon890m_llamacpp/README.md). |
| PC HX370: CPU / native Windows | Qwen3-TTS 0.6B CustomVoice | SCOPED_E2E_OMNI_COMPLETE_REQUEST | The pinned 0.6B CustomVoice BF16/SDPA checkpoint ran wholly on CPU through an isolated Qwen wrapper worker owned by one bounded Omni StageRuntime stage. Two named 24 kHz WAVs matched the earlier standalone PCM SHA256 values exactly. One warmup + 20 serial complete Omni requests for 4.56 s of audio had 20/20 identical PCM and nearest-rank wall p50/p95 15.32/15.60 s (RTF 3.36/3.42); independent Whisper tiny.en ASR gave WER 0.20 on the first Omni WAV, an intelligibility proxy only. A public AsyncOmni audio request passed. The stage verified pinned weights/interpreter and CPU placement, reserved 10 GiB host RAM, rejected an insufficient 4 GiB demand before worker launch, and after in-flight cancellation emitted no stale output and returned the ledger to zero. Sampled process-tree private memory peaked at 5.39 GB, not a loading peak. Native vLLM CPU attention operators remain absent; playable streaming, post-cancel restart, broader speech quality, loading peak and sustained power/thermal behavior remain unverified. | Validate playable streaming and restart/recovery, broader voice quality, peak memory, concurrency and sustained behavior; [Omni raw evidence](../../../e2e_expansion_20260923/evidence/qwen_tts_omni_cpu/README.md), [standalone baseline](../../../e2e_expansion_20260923/evidence/qwen_tts_native_cpu/README.md). |
| PC HX370: CPU / native Windows | Qwen3.8-27B | NOT_E2E_PROFILEABLE | The installed native Windows vLLM extension lacks cpu_attn_reshape_and_cache and cpu_attn_get_scheduler_metadata. The separate ORT CPU graph worker is not a whole-model text/TTS adapter; a qualified CPU build/binding is missing. | Supply and validate the missing matching artifact/backend, then run the complete specified workload. |
| PC HX370: CPU / native Windows | MiniCPM-o 4.5 | SCOPED_E2E_OMNI_COMPLETE_REQUEST | The pinned Q4_K_M/F16 GGUF set ran one combined synthetic JPEG+spoken-question → text+speech request through a bounded Omni whole-session graph stage on native Windows CPU. It answered “The square in the image is red.” and produced 2.24 s of 24 kHz speech in 24.34 s complete-request wall time (one cold sample, zero warmups); sampled worker RSS peaked at 13.11 GB. The stage verified ten weight hashes and the CLI hash, observed CPU placement, reserved 24 GiB shared RAM, and emitted a terminal audio event. An 8 GiB reservation was refused before worker launch; in-flight cancellation left no stale output and released the ledger. The prior standalone prompted run took 24.66 s and independently transcribed output with WER 0. This is a whole-request C++ worker under Omni control, not native vLLM execution or playable streaming; one synthetic quality case is not general qualification. | Validate real image/audio suites, playable streaming, post-cancel restart, loading peak, repeated latency and sustained thermal behavior. [Omni raw evidence](../../../e2e_expansion_20260923/evidence/minicpmo_omni_cpp/README.md), [standalone prompted evidence](../../../e2e_expansion_20260923/evidence/minicpmo_cpp_windows_cpu_prompted/README.md), [failed first run](../../../e2e_expansion_20260923/evidence/minicpmo_cpp_windows_cpu_attempt1/README.md). |
| PC HX370: CPU / native Windows | MiniCPM-o 4.5 | SCOPED_E2E_OMNI_COMPLETE_REQUEST; PERFORMANCE_UNSTABLE | The pinned Q4_K_M/F16 GGUF set ran combined synthetic JPEG+spoken-question → text+speech requests through a bounded Omni whole-session graph stage on native Windows CPU. The earlier cold request answered “The square in the image is red.” and produced 2.24 s of 24 kHz speech in 24.34 s; the later one-request control with the same fixture and a 21 GiB budget also answered correctly but took 73.63 s and produced 1.96 s of speech. Its pinned Whisper tiny.en transcript had WER 0.143 (one-output intelligibility proxy). OS-available RAM before the later load was 32.28 GB, sampled worker RSS peaked at 13.10 GB, and the worker logged two `PrefetchVirtualMemory failed` warnings. The stage verified hashes and CPU placement, rejected insufficient RAM before launch, emitted a terminal audio event, and released the ledger after in-flight cancellation. These separate one-request runs do not form a latency distribution or establish the slowdown's cause. This is a whole-request C++ worker under Omni control, not native vLLM execution or playable streaming; one synthetic quality case is not general qualification. | Measure repeated CPU latency and OS memory/power with matched Radeon conditions; validate real image/audio suites, playable streaming, post-cancel restart, loading peak and sustained thermal behavior. [Later CPU control](../../../e2e_expansion_20260923/evidence/minicpmo_omni_radeon_profile20/README.md), [earlier Omni raw evidence](../../../e2e_expansion_20260923/evidence/minicpmo_omni_cpp/README.md), [standalone prompted evidence](../../../e2e_expansion_20260923/evidence/minicpmo_cpp_windows_cpu_prompted/README.md), [failed first run](../../../e2e_expansion_20260923/evidence/minicpmo_cpp_windows_cpu_attempt1/README.md). |
| PC HX370: CPU / native Windows | InternVLA-A1 | SYNTHETIC_POLICY_PASS_OMNI; NOT_TASK_E2E | The real Place_Markpen checkpoint now runs through an admitted native Windows Omni StageRuntime whole-policy stage with the policy and output on CPU. One warmup plus 20 serial patterned synthetic camera/state/noise requests returned identical finite `[1,50,32]` action chunks; nearest-rank complete-request wall p50/p95 was 4.878/4.902 s, excluding 135.60 s startup. A separate public `AsyncOmni.generate` request returned a terminal action event with the same hash. The stage verified checkpoint/interpreter hashes, used a 16 GiB host-RAM capacity and demand within observed Windows availability, refused 8 GiB before worker launch, and after in-flight abort emitted no stale output and released the ledger. Compared with the WSL CPU Omni output for this fixture, action relative L2 difference was 2.1795% and cosine 0.999764, without a task tolerance. The earlier one-forward zero-input policy result is a different fixture. | Validate real observations/reference actions, physical units/order/step time and deadlines, loading peak, post-cancel restart, concurrency and sustained power/thermal behavior; [native Omni evidence](../../../e2e_expansion_20260923/evidence/internvla_omni_windows_cpu/README.md), [earlier direct policy evidence](../../../e2e_expansion_20260923/README.md). |
| PC HX370 + RTX 5090 Laptop / WSL | Spark-X2.5 | PROFILE_COMPLETED | not fully qualified: reference-quality and remaining reliability/performance gates must be reviewed | Broaden only with a named workload/device/precision qualification. |
| PC HX370 + RTX 5090 Laptop / WSL | Qwen3-TTS 0.6B CustomVoice | PROFILE_COMPLETED | not fully qualified: reference-quality and remaining reliability/performance gates must be reviewed | Resolve and measure playback stalls; run quality, interruption and thermal tests. |
@@ -56,7 +56,7 @@ NOT E2E means no complete device/model workload was verified. “Scoped E2E” i
| PC HX370 + Radeon 890M / Windows worker | Spark-X2.5 | SCOPED_E2E_OMNI_COMPLETE_REQUEST | The same pinned Spark-X2.5-1.7B Q4_K_M GGUF completed text prefill/decode through a bounded Omni whole-session stage with llama.cpp Vulkan1. The log identifies Radeon 890M and 29/29 layers offloaded there, rather than RTX 5090; an intentionally wrong device name was refused. `Paris` and `1511` passed; one warmup plus 20 serial complete requests were 20/20 correct, nearest-rank wall p50/p95 2.10/2.14 s. A public AsyncOmni request passed; an oversized context was rejected with a later correct request; in-flight cancellation produced no stale output and released the ledger. The prior standalone run was 2.18/2.20 s. No streaming, restart-after-cancel, GPU-memory peak or sustained power/thermal check is claimed. | Qualify public configuration, broader text/token quality, streaming and cancellation recovery, shared-RAM/GPU peak and sustained use; [Omni raw evidence](../../../e2e_expansion_20260923/evidence/spark_omni_llamacpp/README.md), [standalone evidence](../../../e2e_expansion_20260923/evidence/spark_radeon890m_llamacpp/README.md). |
| PC HX370 + Radeon 890M / Windows worker | Qwen3-TTS 0.6B CustomVoice | SCOPED_E2E_OMNI_COMPLETE_REQUEST_HYBRID | The pinned Q8_0 CustomVoice talker, F16 codec and punctuation GGUFs completed two named text-to-WAV requests through an Omni whole-session CrispASR stage on native Windows. The runtime log identifies Radeon 890M Vulkan0 for talker/codec and the 432 MB code predictor pinned to CPU FP32; this is not all-Radeon or NPU execution. Both PCM hashes exactly match the earlier standalone outputs. Whisper tiny.en transcribed the first Omni WAV and both standalone WAVs exactly (WER 0 as an intelligibility proxy). One warmup plus 20 serial complete Omni requests for 2.56 s of audio had identical PCM and nearest-rank wall p50/p95 3.298/3.336 s (RTF 1.29/1.30). A public AsyncOmni audio request passed; cancellation drained the worker without stale output or remaining reservation. The earlier standalone profile was 3.310/3.345 s. The default Vulkan configuration previously fell back to CPU, all-Vulkan ran away to a 360-frame cap, and DirectML failed waveform quality. No playable stream, post-cancel restart, loading peak, broad speech quality or sustained power/thermal run is claimed. | Validate playable streaming, restart/recovery, broader voice quality, shared-RAM/GPU peak, concurrency and sustained real-time behavior before wider qualification; [Omni raw evidence](../../../e2e_expansion_20260923/evidence/qwen_tts_omni_hybrid/README.md), [standalone evidence](../../../e2e_expansion_20260923/evidence/qwen_tts_radeon890m_joint/README.md), [failed DirectML route](../../../e2e_expansion_20260923/evidence/qwen_tts_radeon890m_directml/README.md). |
| PC HX370 + Radeon 890M / Windows worker | Qwen3.8-27B | NOT_E2E_PROFILEABLE | FP32 448x448 vision tower: DirectML 85/113 nodes; normalized max error 9.095e-06 versus torch CPU. Historical component path, not full 27B or current graph-stage image-to-text integration. | Recover the hashed tower bundle and qualify its current Omni graph handoff with the language stage. |
| PC HX370 + Radeon 890M / Windows worker | MiniCPM-o 4.5 | SCOPED_E2E_OMNI_COMPLETE_REQUEST_HYBRID; PERFORMANCE_UNSTABLE | The pinned GGUF set ran a combined synthetic spoken-question+JPEG → text+speech workload through a bounded Omni whole-session graph stage. All 21 successful worker logs in a new serial profile verified Radeon 890M Vulkan0 language (37/37 layers) and vision, with CPU TTS/Token2Wav/vocoder; each request launched a fresh C++ model session. After one warmup, 20/20 requests answered “The square in the image is red.” and produced nonzero 2.08–3.00 s 24 kHz speech with terminal audio events. Nearest-rank complete-request wall p50/p95 was **87.27/91.45 s**, range 69.91–97.90 s, excluding 51.92 s controller startup. A separate cold rerun was 88.01 s. The first measured WAV transcribed exactly with pinned Whisper tiny.en (WER 0, one-output intelligibility proxy). A 21 GiB shared-RAM demand/capacity fit the 23.80 GB OS-available pre-load sample; maximum sampled worker RSS was 15.30 GB. Every successful new worker logged `PrefetchVirtualMemory failed` twice. The older one-cold-request Omni result was 18.27 s under a 24 GiB reservation/30 GiB declared ceiling and had no prefetch warning. The cause of the roughly fivefold regression is unqualified; no matched CPU control, OS memory/power time series or shared-iGPU allocation trace exists. In-flight cancellation left no stale output or reservation. This is CPU+iGPU joint execution, not all-Radeon, NPU, general quality or sustained performance qualification. | Measure Windows available/commit/pagefile and GPU shared-memory over matched CPU/Radeon runs; isolate UNC weight I/O, then validate real multimodal quality, playable streaming, post-cancel restart, loading peak, concurrency and sustained thermal behavior. [20-request Omni evidence](../../../e2e_expansion_20260923/evidence/minicpmo_omni_radeon_profile20/README.md), [earlier single-request Omni evidence](../../../e2e_expansion_20260923/evidence/minicpmo_omni_cpp/README.md). |
| PC HX370 + Radeon 890M / Windows worker | MiniCPM-o 4.5 | SCOPED_E2E_OMNI_COMPLETE_REQUEST_HYBRID; PERFORMANCE_UNSTABLE | The pinned GGUF set ran a combined synthetic spoken-question+JPEG → text+speech workload through a bounded Omni whole-session graph stage. All 21 successful worker logs in a new serial profile verified Radeon 890M Vulkan0 language (37/37 layers) and vision, with CPU TTS/Token2Wav/vocoder; each request launched a fresh C++ model session. After one warmup, 20/20 requests answered “The square in the image is red.” and produced nonzero 2.08–3.00 s 24 kHz speech with terminal audio events. Nearest-rank complete-request wall p50/p95 was **87.27/91.45 s**, range 69.91–97.90 s, excluding 51.92 s controller startup. A separate cold rerun was 88.01 s. The first measured WAV transcribed exactly with pinned Whisper tiny.en (WER 0, one-output intelligibility proxy). A 21 GiB shared-RAM demand/capacity fit the 23.80 GB OS-available pre-load sample; maximum sampled worker RSS was 15.30 GB. Every successful new worker logged `PrefetchVirtualMemory failed` twice. The older one-cold-request Omni result was 18.27 s under a 24 GiB reservation/30 GiB declared ceiling and had no prefetch warning. The cause of the roughly fivefold regression is unqualified. A later CPU-only control with the same fixture also slowed to 73.63 s and logged prefetch warnings despite 32.28 GB OS-available before load; it was not a matched-condition comparison. OS memory/power time series and shared-iGPU allocation traces are still missing. In-flight cancellation left no stale output or reservation. This is CPU+iGPU joint execution, not all-Radeon, NPU, general quality or sustained performance qualification. | Measure Windows available/commit/pagefile and GPU shared-memory over matched CPU/Radeon runs; isolate UNC weight I/O, then validate real multimodal quality, playable streaming, post-cancel restart, loading peak, concurrency and sustained thermal behavior. [20-request Omni evidence](../../../e2e_expansion_20260923/evidence/minicpmo_omni_radeon_profile20/README.md), [earlier single-request Omni evidence](../../../e2e_expansion_20260923/evidence/minicpmo_omni_cpp/README.md). |
| PC HX370 + Radeon 890M / Windows worker | InternVLA-A1 | SYNTHETIC_POLICY_PASS_HYBRID_OMNI; NOT_TASK_E2E | The real Place_Markpen checkpoint ran in admitted Omni whole-policy stages on both WSL CPU+Radeon and native Windows CPU+Radeon. In each, the BF16 Qwen3-VL/flow action policy stayed on CPU and a pinned six-frame FP32 Cosmos encoder used Omni's external DirectML worker; the selected Radeon 890M and graph output device were verified, without internal per-op attribution. Each placement returned 20/20 identical finite `[1,50,32]` actions after one warmup. WSL hybrid complete-request p50/p95 was 3.120/3.197 s versus its separate CPU baseline 3.175/3.219 s, with action relative L2 difference 2.305%. Native Windows hybrid was 4.575/4.725 s versus its separate CPU baseline 4.878/4.902 s, with action relative L2 difference 2.4391%. Native hybrid startup was 140.31 s versus CPU 135.60 s. These are separate one-fixture serial runs, not paired speedup or task-quality proof. FP32 encoder is an explicit precision change after DirectML BF16 aborted. Both stages verified hashes, reserved 16 GiB shared host RAM, rejected insufficient demand before launch, emitted terminal actions through public `AsyncOmni.generate`, and released reservations after in-flight cancellation/shutdown. An initial WSL hybrid shutdown quarantined 16 GiB; the lifecycle fix and reruns cleared it. The older direct hybrid zero-input profile is a different experiment. | Validate real observations/reference actions, physical units/order/step time, post-cancel restart, loading peak, concurrency and sustained tail/power before deployment; [WSL Omni evidence](../../../e2e_expansion_20260923/evidence/internvla_omni_policy/README.md), [native Windows Omni evidence](../../../e2e_expansion_20260923/evidence/internvla_omni_windows_hybrid/README.md), [earlier direct component evidence](../../../e2e_expansion_20260923/evidence/internvla_cosmos_radeon890m/README.md). |
| PC HX370 + AMD NPU / Windows worker | Spark-X2.5 | COMPONENT_PASS; NOT_E2E_PROFILEABLE | The real 1.7B output head (RMS norm and full 131,072-logit projection) ran through Omni `external.graph.v1` with VitisAI: one fused NPU partition and seven CPU nodes per request. The A16W8 graph matched top-1 on four captured activations (full-logit SNR 55.43–56.81 dB). One warmup plus 20 serial repeated-input component requests gave p50/p95 10.21/12.44 ms, versus 17.97/20.29 ms for the original FP32 output head on the same Omni CPU stage. NPU startup was 18.21 s versus 4.48 s for FP32 CPU, and NPU worker peak RSS was 3.52 GiB versus 2.06 GiB. No full prefill/decode/KV/sampling, real handoff, power, or sustained run was verified. | Measure an actual vLLM-to-output-head handoff and whole-generation cost, state/cancellation, memory admission and quality before considering a split plan; [component evidence](../../../e2e_expansion_20260923/evidence/spark_amd_npu_output_head/README.md). |
| PC HX370 + AMD NPU / Windows worker | Qwen3-TTS 0.6B CustomVoice | NOT_E2E_PROFILEABLE | No placement- and quality-verified whole-model graph bundle/stateful adapter for this model on this AMD route. Small backend graphs do not establish model support. | Supply and validate the missing matching artifact/backend, then run the complete specified workload. |