Profile repeated Radeon MiniCPM-o requests and record memory regression
This commit is contained in:
tzhouam committed
1 parent
5dd5d70671
commit
c5f47b22fd
36 files changed
+12518
-3
No files matched your search
@@ -34,6 +34,14 @@ async def main() -> None:
|
||||
args = parser.parse_args()
|
||||
if args.warmups < 0 or args.repeats < 1:
|
||||
parser.error("warmups must be nonnegative and repeats must be positive")
|
||||
if args.reserve_gib < 1 or args.capacity_gib < args.reserve_gib:
|
||||
parser.error("invalid explicit host-RAM budget")
|
||||
|
||||
import psutil
|
||||
|
||||
host_available_bytes = psutil.virtual_memory().available
|
||||
if args.capacity_gib << 30 > host_available_bytes:
|
||||
raise RuntimeError("declared host-RAM capacity exceeds OS available RAM before load")
|
||||
|
||||
from vllm_omni.config.stage_config import DeployConfig, StageDeployConfig, merge_pipeline_deploy
|
||||
from vllm_omni.engine.stage_runtime import StageRuntime
|
||||
@@ -76,7 +84,8 @@ async def main() -> None:
|
||||
"cli_sha256": args.cli_sha256, "reference_sha256": reference_hash,
|
||||
"input_sha256": {key: hashlib.sha256(value).hexdigest() for key, value in inputs.items()},
|
||||
"memory_budget": budget, "profile_count": args.repeats,
|
||||
"warmup_count": args.warmups}
|
||||
"warmup_count": args.warmups,
|
||||
"host_available_bytes_before": host_available_bytes}
|
||||
try:
|
||||
started = time.perf_counter()
|
||||
if args.expect_admission_refusal:
|
||||
|
||||
@@ -8,6 +8,8 @@ A further [native Windows CPU Omni policy profile](evidence/internvla_omni_windo
|
||||
|
||||
The matching [native Windows CPU+Radeon Omni policy profile](evidence/internvla_omni_windows_hybrid/README.md) kept the BF16 action policy on CPU and ran only the pinned FP32 six-frame Cosmos encoder through a native DirectML worker. It returned 20/20 identical finite action chunks after one warmup at nearest-rank complete-request wall p50/p95 4.575/4.725 s, versus the separate native CPU baseline 4.878/4.902 s; startup was 140.31 s versus 135.60 s. Hybrid-vs-CPU action relative L2 difference was 2.4391% without a task tolerance. Native hybrid pre-launch refusal, in-flight abort and public `AsyncOmni.generate` action output passed with a 16 GiB shared-RAM reservation. The output device was verified as Radeon 890M, not every internal graph op. This is a synthetic integration pass, not robot-task E2E or sustained benefit.
|
||||
|
||||
A later [native Windows Radeon MiniCPM-o Omni profile](evidence/minicpmo_omni_radeon_profile20/README.md) completed one warmup and 20 serial **combined spoken-question+image → text+speech** requests with the same pinned GGUF set and patched C++ CLI. All 20 answered the red-square question and emitted complete speech; the first measured WAV independently transcribed with WER 0 using pinned Whisper tiny.en. Nearest-rank complete-request wall p50/p95 was **87.27/91.45 s**, much slower than the earlier 18.27 s single cold Omni request. A separate cold rerun after the profile took 88.01 s. All 21 successful new worker logs report Radeon Vulkan0 37/37 language layers, CPU Token2Wav and repeated `PrefetchVirtualMemory` warnings. The 21 GiB shared-RAM budget fit the pre-load available-RAM check and cancellation released it, but no matched CPU control or full memory/power trace identifies the regression's cause. This advances repeated functional evidence while weakening any claim of predictable Radeon performance under current memory conditions.
|
||||
|
||||
| Device / model | Verified depth | Observed output and time | Remaining gate |
|
||||
|---|---|---|---|
|
||||
| Native Windows HX370 Radeon 890M / InternVLA-A1 Cosmos CI8x8 encoder | Component C; unchanged Omni encoder source in separate DirectML/PyTorch 2.4 environment, not an Omni full-policy stage | One warmup + 20 serial patterned 256×256 image encodes, including upload/readback: Radeon p50/p95 26.56/27.49 ms versus CPU 156.49/158.71 ms; relative L2 error 4.73e-07. Encoder weights and output device checked. | Complete policy/action and real-observation parity; policy/controller runtime compatibility, bounded handoff, shared-RAM admission and end-to-end benefit. [Raw evidence](evidence/internvla_cosmos_radeon890m/README.md). |
|
||||
|
||||
+46
@@ -0,0 +1,46 @@
|
||||
# Native Windows Radeon MiniCPM-o: repeated Omni complete requests
|
||||
|
||||
**Twenty scoped audio+image → text+speech requests passed; performance regressed under this memory condition.** The same pinned MiniCPM-o 4.5 GGUF bundle and bounded `llama-omni-cli.exe` from the [earlier one-request Omni pass](../minicpmo_omni_cpp/README.md) ran through `StageRuntime`/`StagePool` on the HX370 laptop, native Windows 11 build 26200 with Radeon 890M driver 32.0.22018.6001, Python 3.12.10, installed vLLM 0.29 and this Omni checkout. Every request launches a fresh C++ model session; the timings include its weight load, combined input processing, text generation, Token2Wav, and complete 24 kHz WAV assembly. They are not resident-model inference latency or playable streaming.
|
||||
|
||||
The exact model revision `db25077c33951fe163b42986fba0132e279872a2`, ten GGUF artifact hashes, patched CLI SHA256 `66a512386aafeb708f49a7394a9054d94ec777c362395c8917a3d198aef1b523`, input hashes and 2,048-token/96-output-token bounds are in [profile20.json](profile20.json). The requested placement was `radeon-hybrid`: `GGML_VK_VISIBLE_DEVICES=1` selected `AMD Radeon(TM) 890M Graphics` Vulkan0 for language and vision, with CPU TTS/Token2Wav/vocoder. All **21 successful** worker logs (one warmup and 20 measured) identify the Radeon adapter, 37/37 language layers offloaded, and CPU Token2Wav. The [full logs](worker_logs/) are preserved with line endings and trailing whitespace normalized for text review; the original Windows files remain under the paths recorded in the JSON report. The 22nd log is the deliberately aborted request. This is CPU+iGPU joint execution, not NPU or all-GPU execution.
|
||||
|
||||
The fixture is the same 16 kHz spoken question “What color is the square in the image?” plus a 448×448 synthetic red-square JPEG used by the earlier run. After one excluded warmup, all 20 measured requests answered “The square in the image is red.” and emitted a finite, nonzero complete WAV with a terminal audio event. Speech duration ranged **2.08–3.00 s** and the 20 PCM hashes differed; this is expected nondeterminism, not text-answer disagreement. The [first measured WAV](profile20.wav) transcribed exactly as “The square in the image is red.” with pinned local Whisper tiny.en revision `87c7102498dcde7456f24cfd30239ca606ed9063`, WER 0 for **one output only** ([ASR report](profile20_first_asr.json)). This is an intelligibility proxy, not broad speech or image-grounding quality.
|
||||
|
||||
| Condition | Complete-request wall | Startup outside requests | Memory evidence |
|
||||
|---|---:|---:|---|
|
||||
| One warmup + 20 serial measured requests | Nearest-rank p50/p95 **87.27/91.45 s**; range 69.91–97.90 s | 51.92 s | 21 GiB shared-RAM reservation and capacity; 23.80 GB OS-available before load; max sampled worker RSS 15.30 GB |
|
||||
| Separate cold rerun after profile | One **88.01 s** request | 47.63 s | Same 21 GiB budget; 23.99 GB OS-available before load; sampled worker RSS 15.30 GB ([report](cold_after_profile.json), [log](cold_after_worker_logs/)) |
|
||||
| Earlier cold Omni smoke | One **18.27 s** request | 6.83 s | 24 GiB reservation under a declared 30 GiB ceiling; [earlier record](../minicpmo_omni_cpp/radeon_report.json) |
|
||||
|
||||
These are different runs, not a paired speed comparison. The new repeated profile and cold rerun both passed functionally but were roughly five times slower than the older single cold request. Each of the 21 successful new worker logs contains two `PrefetchVirtualMemory failed` warnings, while the earlier single-request log contains none. Representative new language-model load times were about 25–27 s versus 6.07 s in the earlier log; the first measured new request took 89.67 s wall, including speech generation. Those observations are consistent with memory pressure or changed file-cache/paging conditions, **but do not identify a root cause**. Windows free RAM was only about 22.17 GiB at admission for a 21 GiB controller ceiling. Power/thermal state, full OS commit/pagefile time series, shared-iGPU allocation and a matched-condition CPU control were not captured. Do not reuse the older 18.27 s result as expected warmed performance or claim the Radeon path is faster than CPU under this condition.
|
||||
|
||||
The stage verified artifacts and placement, admitted the explicit shared-RAM demand, and the final in-flight abort produced no stale output. Both abort and shutdown left zero reserved/quarantined bytes in the [profile report](profile20.json). The probe now refuses a declared capacity above OS-available RAM before loading. Sampled worker RSS does not establish the weight-loading peak or bound shared GPU allocation. No post-cancel restart, concurrency, real image/audio suite, playable streaming or sustained power/thermal gate was run.
|
||||
|
||||
Reproduce from the repository root in PowerShell with the pinned local artifacts:
|
||||
|
||||
```powershell
|
||||
$root = '\\wsl.localhost\Ubuntu\home\zhout\project\edge_infer'
|
||||
$repo = Join-Path $root 'vllm-omni-edge'
|
||||
$fixture = Join-Path $repo 'benchmarks\edge_harness\results\e2e_expansion_20260923\evidence\minicpmo_cpp_radeon890m_prompted'
|
||||
$evidence = Join-Path $repo 'benchmarks\edge_harness\results\e2e_expansion_20260923\evidence\minicpmo_omni_radeon_profile20'
|
||||
$py = 'C:\Users\zhout\w2\omni029venv\Scripts\python.exe'
|
||||
$env:PYTHONUTF8 = '1'
|
||||
$env:PYTHONPATH = $repo
|
||||
$env:VLLM_ENABLE_V1_MULTIPROCESSING = '0'
|
||||
& $py (Join-Path $repo 'benchmarks\edge_harness\experiments\probe_omni_minicpmo_cpp.py') `
|
||||
--model-dir (Join-Path $root 'models\MiniCPM-o-4_5-gguf') `
|
||||
--cli-bin 'C:\Users\zhout\w2\minicpmo_omni_bin_radeon\llama-omni-cli.exe' `
|
||||
--cli-sha256 '66a512386aafeb708f49a7394a9054d94ec777c362395c8917a3d198aef1b523' `
|
||||
--reference-wav (Join-Path $fixture 'fixture_0000.wav') `
|
||||
--audio-wav (Join-Path $fixture 'fixture_0001.wav') `
|
||||
--image-jpeg (Join-Path $fixture 'fixture_0001.jpg') `
|
||||
--artifact-report (Join-Path $fixture 'full_report.json') `
|
||||
--work-root 'C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon' `
|
||||
--log-root 'C:\Users\zhout\w2\minicpmo_omni_profile20_logs_radeon' `
|
||||
--placement radeon-hybrid --reserve-gib 21 --capacity-gib 21 `
|
||||
--warmups 1 --repeats 20 --abort-check `
|
||||
--output-report (Join-Path $evidence 'profile20.json') `
|
||||
--output-wav (Join-Path $evidence 'profile20.wav')
|
||||
```
|
||||
|
||||
Next experiment: capture Windows available/commit/pagefile and GPU shared-memory time series across a matched CPU/Radeon pair, then isolate UNC weight I/O from model execution. Keep the full input/output quality and task checks separate from performance diagnosis.
|
||||
+173
@@ -0,0 +1,173 @@
|
||||
{
|
||||
"scope": "MiniCPM-o GGUF complete audio+image\u2192text+speech through Omni StageRuntime/StagePool",
|
||||
"placement": "radeon-hybrid",
|
||||
"model_revision": "db25077c33951fe163b42986fba0132e279872a2",
|
||||
"artifact_sha256": {
|
||||
"MiniCPM-o-4_5-Q4_K_M.gguf": "1237a97ee081b8abebc47aa7dad565701e8f5f904cdc92f6723ac4281bbc0932",
|
||||
"audio/MiniCPM-o-4_5-audio-F16.gguf": "d5b188ac7feaf98e17175c3f9bd14bf269301bfd187439fdaa3e3a494fc32ef7",
|
||||
"vision/MiniCPM-o-4_5-vision-F16.gguf": "1453678cc4e4fe18de241952962e234f265cb8dda780773526103ab8ba82f421",
|
||||
"tts/MiniCPM-o-4_5-tts-F16.gguf": "c7be3748a863dd6966ae7eed42600b7f41ca67affb03729ff245247f0e5ea088",
|
||||
"tts/MiniCPM-o-4_5-projector-F16.gguf": "4b1b5b377358a5e594a304ff6ea5d52df606a9ba7d886c4299d232f0c67dd1fd",
|
||||
"token2wav-gguf/encoder.gguf": "7f8d265da594eaf5e1de2db8f5f1867dbcb0bb75ef5878fadf2952347116f4d0",
|
||||
"token2wav-gguf/flow_matching.gguf": "eda6069f3edeb5dd3a87fbf2aedb2ddd1b46f3273926c4fcf09b24476a39cab8",
|
||||
"token2wav-gguf/flow_extra.gguf": "c67611aa7d02500fe395a7798bf0bfdfb55c74d37ba93934ca74d82b4e63f78d",
|
||||
"token2wav-gguf/hifigan2.gguf": "1b8b3bf5d8d3066aeee4324fdcdd41aefce170d0ee907645858de408d82835c2",
|
||||
"token2wav-gguf/prompt_cache.gguf": "81fe6f541ebe0b67db06a1e395df928da47991dc9637c2cf47d6c59d5b979f2c"
|
||||
},
|
||||
"cli_sha256": "66a512386aafeb708f49a7394a9054d94ec777c362395c8917a3d198aef1b523",
|
||||
"reference_sha256": "cf4bb5cd9acbfbf67596d4a53310652b61ef4b7ede1d54eb03ef4882ca40223a",
|
||||
"input_sha256": {
|
||||
"audio_wav": "14ee147cbf2ce9afd9e32c8c8228d7cfe11985cd7e63456b2ad33871492aefcc",
|
||||
"image_jpeg": "22c7ccea466a3f551c512045e6316c7597408d4e0a20205502f55784d977aa8d"
|
||||
},
|
||||
"memory_budget": {
|
||||
"capacities": {
|
||||
"host_ram": 22548578304
|
||||
},
|
||||
"demands": {
|
||||
"host_ram": 22548578304
|
||||
}
|
||||
},
|
||||
"profile_count": 1,
|
||||
"warmup_count": 0,
|
||||
"host_available_bytes_before": 23985426432,
|
||||
"startup_s": 47.634812799980864,
|
||||
"execution_plan_at_start": {
|
||||
"backend": "external.minicpmo.gguf.v1",
|
||||
"stage_id": 0,
|
||||
"worker_generation": "21544f963e164e55872e99c89e92e955",
|
||||
"model_revision": "db25077c33951fe163b42986fba0132e279872a2",
|
||||
"artifact_sha256": {
|
||||
"MiniCPM-o-4_5-Q4_K_M.gguf": "1237a97ee081b8abebc47aa7dad565701e8f5f904cdc92f6723ac4281bbc0932",
|
||||
"audio/MiniCPM-o-4_5-audio-F16.gguf": "d5b188ac7feaf98e17175c3f9bd14bf269301bfd187439fdaa3e3a494fc32ef7",
|
||||
"vision/MiniCPM-o-4_5-vision-F16.gguf": "1453678cc4e4fe18de241952962e234f265cb8dda780773526103ab8ba82f421",
|
||||
"tts/MiniCPM-o-4_5-tts-F16.gguf": "c7be3748a863dd6966ae7eed42600b7f41ca67affb03729ff245247f0e5ea088",
|
||||
"tts/MiniCPM-o-4_5-projector-F16.gguf": "4b1b5b377358a5e594a304ff6ea5d52df606a9ba7d886c4299d232f0c67dd1fd",
|
||||
"token2wav-gguf/encoder.gguf": "7f8d265da594eaf5e1de2db8f5f1867dbcb0bb75ef5878fadf2952347116f4d0",
|
||||
"token2wav-gguf/flow_matching.gguf": "eda6069f3edeb5dd3a87fbf2aedb2ddd1b46f3273926c4fcf09b24476a39cab8",
|
||||
"token2wav-gguf/flow_extra.gguf": "c67611aa7d02500fe395a7798bf0bfdfb55c74d37ba93934ca74d82b4e63f78d",
|
||||
"token2wav-gguf/hifigan2.gguf": "1b8b3bf5d8d3066aeee4324fdcdd41aefce170d0ee907645858de408d82835c2",
|
||||
"token2wav-gguf/prompt_cache.gguf": "81fe6f541ebe0b67db06a1e395df928da47991dc9637c2cf47d6c59d5b979f2c"
|
||||
},
|
||||
"cli_sha256": "66a512386aafeb708f49a7394a9054d94ec777c362395c8917a3d198aef1b523",
|
||||
"requested_placement": "radeon-hybrid",
|
||||
"ggml_vk_visible_devices": "1",
|
||||
"shared_ram_reservation": {
|
||||
"host_ram": 22548578304
|
||||
},
|
||||
"weights_bytes": 8872297088,
|
||||
"memory_overhead_bytes": 12884901888,
|
||||
"context_tokens": 2048,
|
||||
"max_new_tokens": 96,
|
||||
"request_capacity": 1,
|
||||
"stateful_session": "C++-owned within one request; discarded after completion",
|
||||
"placement_evidence": "pending first request"
|
||||
},
|
||||
"warmups": [],
|
||||
"measured": [
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 88.00621939997654,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 71040,
|
||||
"duration_s": 2.96,
|
||||
"pcm_sha256": "ec297eabb746949c9a3e86683feea75a2226c2dda207d89e04dbb5281dda71a2"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-0",
|
||||
"stage_id": 0,
|
||||
"epoch": 1,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "21544f963e164e55872e99c89e92e955",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 87.98023680003826,
|
||||
"peak_sampled_rss_bytes": 15296360448,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_cold_after_logs_radeon\\minicpmo-1-1e3b8d41a49e4144aec4b3c3f346ec1c.log"
|
||||
}
|
||||
}
|
||||
],
|
||||
"complete_request": {
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 88.00621939997654,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 71040,
|
||||
"duration_s": 2.96,
|
||||
"pcm_sha256": "ec297eabb746949c9a3e86683feea75a2226c2dda207d89e04dbb5281dda71a2"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-0",
|
||||
"stage_id": 0,
|
||||
"epoch": 1,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "21544f963e164e55872e99c89e92e955",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 87.98023680003826,
|
||||
"peak_sampled_rss_bytes": 15296360448,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_cold_after_logs_radeon\\minicpmo-1-1e3b8d41a49e4144aec4b3c3f346ec1c.log"
|
||||
}
|
||||
},
|
||||
"nearest_rank_p50_wall_s": 88.00621939997654,
|
||||
"nearest_rank_p95_wall_s": 88.00621939997654,
|
||||
"output_wav": "\\\\wsl.localhost\\Ubuntu\\home\\zhout\\project\\edge_infer\\vllm-omni-edge\\benchmarks\\edge_harness\\results\\e2e_expansion_20260923\\evidence\\minicpmo_omni_radeon_profile20\\cold_after_profile.wav",
|
||||
"execution_plan_after_request": {
|
||||
"backend": "external.minicpmo.gguf.v1",
|
||||
"stage_id": 0,
|
||||
"worker_generation": "21544f963e164e55872e99c89e92e955",
|
||||
"model_revision": "db25077c33951fe163b42986fba0132e279872a2",
|
||||
"artifact_sha256": {
|
||||
"MiniCPM-o-4_5-Q4_K_M.gguf": "1237a97ee081b8abebc47aa7dad565701e8f5f904cdc92f6723ac4281bbc0932",
|
||||
"audio/MiniCPM-o-4_5-audio-F16.gguf": "d5b188ac7feaf98e17175c3f9bd14bf269301bfd187439fdaa3e3a494fc32ef7",
|
||||
"vision/MiniCPM-o-4_5-vision-F16.gguf": "1453678cc4e4fe18de241952962e234f265cb8dda780773526103ab8ba82f421",
|
||||
"tts/MiniCPM-o-4_5-tts-F16.gguf": "c7be3748a863dd6966ae7eed42600b7f41ca67affb03729ff245247f0e5ea088",
|
||||
"tts/MiniCPM-o-4_5-projector-F16.gguf": "4b1b5b377358a5e594a304ff6ea5d52df606a9ba7d886c4299d232f0c67dd1fd",
|
||||
"token2wav-gguf/encoder.gguf": "7f8d265da594eaf5e1de2db8f5f1867dbcb0bb75ef5878fadf2952347116f4d0",
|
||||
"token2wav-gguf/flow_matching.gguf": "eda6069f3edeb5dd3a87fbf2aedb2ddd1b46f3273926c4fcf09b24476a39cab8",
|
||||
"token2wav-gguf/flow_extra.gguf": "c67611aa7d02500fe395a7798bf0bfdfb55c74d37ba93934ca74d82b4e63f78d",
|
||||
"token2wav-gguf/hifigan2.gguf": "1b8b3bf5d8d3066aeee4324fdcdd41aefce170d0ee907645858de408d82835c2",
|
||||
"token2wav-gguf/prompt_cache.gguf": "81fe6f541ebe0b67db06a1e395df928da47991dc9637c2cf47d6c59d5b979f2c"
|
||||
},
|
||||
"cli_sha256": "66a512386aafeb708f49a7394a9054d94ec777c362395c8917a3d198aef1b523",
|
||||
"requested_placement": "radeon-hybrid",
|
||||
"ggml_vk_visible_devices": "1",
|
||||
"shared_ram_reservation": {
|
||||
"host_ram": 22548578304
|
||||
},
|
||||
"weights_bytes": 8872297088,
|
||||
"memory_overhead_bytes": 12884901888,
|
||||
"context_tokens": 2048,
|
||||
"max_new_tokens": 96,
|
||||
"request_capacity": 1,
|
||||
"stateful_session": "C++-owned within one request; discarded after completion",
|
||||
"placement_evidence": {
|
||||
"observed": "radeon-hybrid",
|
||||
"worker_pid": 38976,
|
||||
"peak_sampled_rss_bytes": 15296360448,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_cold_after_logs_radeon\\minicpmo-1-1e3b8d41a49e4144aec4b3c3f346ec1c.log"
|
||||
}
|
||||
},
|
||||
"status": "passed",
|
||||
"ledger_after_shutdown": {
|
||||
"capacities": {
|
||||
"host_ram": 22548578304
|
||||
},
|
||||
"reserved": {
|
||||
"host_ram": 0
|
||||
},
|
||||
"owners": [],
|
||||
"quarantined": []
|
||||
}
|
||||
}
|
||||
BIN
Binary file not shown.
+165
@@ -0,0 +1,165 @@
|
||||
INFO 09-24 03:01:03 [patch.py:259] NVFP4 W4A4 weight_scale NaN-clamp: installed.
|
||||
INFO 09-24 03:01:03 [patch.py:510] inductor factorable-divisibility patch: installed.
|
||||
INFO 09-24 03:01:07 [stage_init_utils.py:768] [stage_init] Set VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
WARNING 09-24 03:01:07 [stage_init_utils.py:1887] [stage_init] Failed to load transfer config: [Errno 13] Permission denied: 'C:\\Users\\zhout'
|
||||
{
|
||||
"scope": "MiniCPM-o GGUF complete audio+image\u2192text+speech through Omni StageRuntime/StagePool",
|
||||
"placement": "radeon-hybrid",
|
||||
"model_revision": "db25077c33951fe163b42986fba0132e279872a2",
|
||||
"cli_sha256": "66a512386aafeb708f49a7394a9054d94ec777c362395c8917a3d198aef1b523",
|
||||
"reference_sha256": "cf4bb5cd9acbfbf67596d4a53310652b61ef4b7ede1d54eb03ef4882ca40223a",
|
||||
"input_sha256": {
|
||||
"audio_wav": "14ee147cbf2ce9afd9e32c8c8228d7cfe11985cd7e63456b2ad33871492aefcc",
|
||||
"image_jpeg": "22c7ccea466a3f551c512045e6316c7597408d4e0a20205502f55784d977aa8d"
|
||||
},
|
||||
"memory_budget": {
|
||||
"capacities": {
|
||||
"host_ram": 22548578304
|
||||
},
|
||||
"demands": {
|
||||
"host_ram": 22548578304
|
||||
}
|
||||
},
|
||||
"profile_count": 1,
|
||||
"warmup_count": 0,
|
||||
"host_available_bytes_before": 23985426432,
|
||||
"startup_s": 47.634812799980864,
|
||||
"execution_plan_at_start": {
|
||||
"backend": "external.minicpmo.gguf.v1",
|
||||
"stage_id": 0,
|
||||
"worker_generation": "21544f963e164e55872e99c89e92e955",
|
||||
"model_revision": "db25077c33951fe163b42986fba0132e279872a2",
|
||||
"artifact_sha256": {
|
||||
"MiniCPM-o-4_5-Q4_K_M.gguf": "1237a97ee081b8abebc47aa7dad565701e8f5f904cdc92f6723ac4281bbc0932",
|
||||
"audio/MiniCPM-o-4_5-audio-F16.gguf": "d5b188ac7feaf98e17175c3f9bd14bf269301bfd187439fdaa3e3a494fc32ef7",
|
||||
"vision/MiniCPM-o-4_5-vision-F16.gguf": "1453678cc4e4fe18de241952962e234f265cb8dda780773526103ab8ba82f421",
|
||||
"tts/MiniCPM-o-4_5-tts-F16.gguf": "c7be3748a863dd6966ae7eed42600b7f41ca67affb03729ff245247f0e5ea088",
|
||||
"tts/MiniCPM-o-4_5-projector-F16.gguf": "4b1b5b377358a5e594a304ff6ea5d52df606a9ba7d886c4299d232f0c67dd1fd",
|
||||
"token2wav-gguf/encoder.gguf": "7f8d265da594eaf5e1de2db8f5f1867dbcb0bb75ef5878fadf2952347116f4d0",
|
||||
"token2wav-gguf/flow_matching.gguf": "eda6069f3edeb5dd3a87fbf2aedb2ddd1b46f3273926c4fcf09b24476a39cab8",
|
||||
"token2wav-gguf/flow_extra.gguf": "c67611aa7d02500fe395a7798bf0bfdfb55c74d37ba93934ca74d82b4e63f78d",
|
||||
"token2wav-gguf/hifigan2.gguf": "1b8b3bf5d8d3066aeee4324fdcdd41aefce170d0ee907645858de408d82835c2",
|
||||
"token2wav-gguf/prompt_cache.gguf": "81fe6f541ebe0b67db06a1e395df928da47991dc9637c2cf47d6c59d5b979f2c"
|
||||
},
|
||||
"cli_sha256": "66a512386aafeb708f49a7394a9054d94ec777c362395c8917a3d198aef1b523",
|
||||
"requested_placement": "radeon-hybrid",
|
||||
"ggml_vk_visible_devices": "1",
|
||||
"shared_ram_reservation": {
|
||||
"host_ram": 22548578304
|
||||
},
|
||||
"weights_bytes": 8872297088,
|
||||
"memory_overhead_bytes": 12884901888,
|
||||
"context_tokens": 2048,
|
||||
"max_new_tokens": 96,
|
||||
"request_capacity": 1,
|
||||
"stateful_session": "C++-owned within one request; discarded after completion",
|
||||
"placement_evidence": "pending first request"
|
||||
},
|
||||
"warmups": [],
|
||||
"measured": [
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 88.00621939997654,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 71040,
|
||||
"duration_s": 2.96,
|
||||
"pcm_sha256": "ec297eabb746949c9a3e86683feea75a2226c2dda207d89e04dbb5281dda71a2"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-0",
|
||||
"stage_id": 0,
|
||||
"epoch": 1,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "21544f963e164e55872e99c89e92e955",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 87.98023680003826,
|
||||
"peak_sampled_rss_bytes": 15296360448,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_cold_after_logs_radeon\\minicpmo-1-1e3b8d41a49e4144aec4b3c3f346ec1c.log"
|
||||
}
|
||||
}
|
||||
],
|
||||
"complete_request": {
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 88.00621939997654,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 71040,
|
||||
"duration_s": 2.96,
|
||||
"pcm_sha256": "ec297eabb746949c9a3e86683feea75a2226c2dda207d89e04dbb5281dda71a2"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-0",
|
||||
"stage_id": 0,
|
||||
"epoch": 1,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "21544f963e164e55872e99c89e92e955",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 87.98023680003826,
|
||||
"peak_sampled_rss_bytes": 15296360448,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_cold_after_logs_radeon\\minicpmo-1-1e3b8d41a49e4144aec4b3c3f346ec1c.log"
|
||||
}
|
||||
},
|
||||
"nearest_rank_p50_wall_s": 88.00621939997654,
|
||||
"nearest_rank_p95_wall_s": 88.00621939997654,
|
||||
"output_wav": "\\\\wsl.localhost\\Ubuntu\\home\\zhout\\project\\edge_infer\\vllm-omni-edge\\benchmarks\\edge_harness\\results\\e2e_expansion_20260923\\evidence\\minicpmo_omni_radeon_profile20\\cold_after_profile.wav",
|
||||
"execution_plan_after_request": {
|
||||
"backend": "external.minicpmo.gguf.v1",
|
||||
"stage_id": 0,
|
||||
"worker_generation": "21544f963e164e55872e99c89e92e955",
|
||||
"model_revision": "db25077c33951fe163b42986fba0132e279872a2",
|
||||
"artifact_sha256": {
|
||||
"MiniCPM-o-4_5-Q4_K_M.gguf": "1237a97ee081b8abebc47aa7dad565701e8f5f904cdc92f6723ac4281bbc0932",
|
||||
"audio/MiniCPM-o-4_5-audio-F16.gguf": "d5b188ac7feaf98e17175c3f9bd14bf269301bfd187439fdaa3e3a494fc32ef7",
|
||||
"vision/MiniCPM-o-4_5-vision-F16.gguf": "1453678cc4e4fe18de241952962e234f265cb8dda780773526103ab8ba82f421",
|
||||
"tts/MiniCPM-o-4_5-tts-F16.gguf": "c7be3748a863dd6966ae7eed42600b7f41ca67affb03729ff245247f0e5ea088",
|
||||
"tts/MiniCPM-o-4_5-projector-F16.gguf": "4b1b5b377358a5e594a304ff6ea5d52df606a9ba7d886c4299d232f0c67dd1fd",
|
||||
"token2wav-gguf/encoder.gguf": "7f8d265da594eaf5e1de2db8f5f1867dbcb0bb75ef5878fadf2952347116f4d0",
|
||||
"token2wav-gguf/flow_matching.gguf": "eda6069f3edeb5dd3a87fbf2aedb2ddd1b46f3273926c4fcf09b24476a39cab8",
|
||||
"token2wav-gguf/flow_extra.gguf": "c67611aa7d02500fe395a7798bf0bfdfb55c74d37ba93934ca74d82b4e63f78d",
|
||||
"token2wav-gguf/hifigan2.gguf": "1b8b3bf5d8d3066aeee4324fdcdd41aefce170d0ee907645858de408d82835c2",
|
||||
"token2wav-gguf/prompt_cache.gguf": "81fe6f541ebe0b67db06a1e395df928da47991dc9637c2cf47d6c59d5b979f2c"
|
||||
},
|
||||
"cli_sha256": "66a512386aafeb708f49a7394a9054d94ec777c362395c8917a3d198aef1b523",
|
||||
"requested_placement": "radeon-hybrid",
|
||||
"ggml_vk_visible_devices": "1",
|
||||
"shared_ram_reservation": {
|
||||
"host_ram": 22548578304
|
||||
},
|
||||
"weights_bytes": 8872297088,
|
||||
"memory_overhead_bytes": 12884901888,
|
||||
"context_tokens": 2048,
|
||||
"max_new_tokens": 96,
|
||||
"request_capacity": 1,
|
||||
"stateful_session": "C++-owned within one request; discarded after completion",
|
||||
"placement_evidence": {
|
||||
"observed": "radeon-hybrid",
|
||||
"worker_pid": 38976,
|
||||
"peak_sampled_rss_bytes": 15296360448,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_cold_after_logs_radeon\\minicpmo-1-1e3b8d41a49e4144aec4b3c3f346ec1c.log"
|
||||
}
|
||||
},
|
||||
"status": "passed",
|
||||
"ledger_after_shutdown": {
|
||||
"capacities": {
|
||||
"host_ram": 22548578304
|
||||
},
|
||||
"reserved": {
|
||||
"host_ram": 0
|
||||
},
|
||||
"owners": [],
|
||||
"quarantined": []
|
||||
}
|
||||
}
|
||||
+484
@@ -0,0 +1,484 @@
|
||||
build: 64 (5202b7b) with x86_64-w64-mingw32-gcc-posix (GCC) 13-posix for x86_64-w64-mingw32
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 30720 MiB free
|
||||
llama_model_loader: loaded meta data with 27 key-value pairs and 399 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = qwen3
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Llm
|
||||
llama_model_loader: - kv 3: general.size_label str = 8.2B
|
||||
llama_model_loader: - kv 4: qwen3.block_count u32 = 36
|
||||
llama_model_loader: - kv 5: qwen3.context_length u32 = 40960
|
||||
llama_model_loader: - kv 6: qwen3.embedding_length u32 = 4096
|
||||
llama_model_loader: - kv 7: qwen3.feed_forward_length u32 = 12288
|
||||
llama_model_loader: - kv 8: qwen3.attention.head_count u32 = 32
|
||||
llama_model_loader: - kv 9: qwen3.attention.head_count_kv u32 = 8
|
||||
llama_model_loader: - kv 10: qwen3.rope.freq_base f32 = 1000000.000000
|
||||
llama_model_loader: - kv 11: qwen3.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: qwen3.attention.key_length u32 = 128
|
||||
llama_model_loader: - kv 13: qwen3.attention.value_length u32 = 128
|
||||
llama_model_loader: - kv 14: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 15: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 16: tokenizer.ggml.tokens arr[str,151748] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 17: tokenizer.ggml.token_type arr[i32,151748] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 24: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 25: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 26: general.file_type u32 = 15
|
||||
llama_model_loader: - type f32: 145 tensors
|
||||
llama_model_loader: - type q4_K: 217 tensors
|
||||
llama_model_loader: - type q6_K: 37 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = Q4_K - Medium
|
||||
print_info: file size = 4.68 GiB (4.90 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = qwen3
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 40960
|
||||
print_info: n_embd = 4096
|
||||
print_info: n_layer = 36
|
||||
print_info: n_head = 32
|
||||
print_info: n_head_kv = 8
|
||||
print_info: n_rot = 128
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 128
|
||||
print_info: n_embd_head_v = 128
|
||||
print_info: n_gqa = 4
|
||||
print_info: n_embd_k_gqa = 1024
|
||||
print_info: n_embd_v_gqa = 1024
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 12288
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = -1
|
||||
print_info: rope type = 2
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 1000000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 40960
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = 8B
|
||||
print_info: model params = 8.19 B
|
||||
print_info: general.name = Llm
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 151748
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 36 repeating layers to GPU
|
||||
load_tensors: offloading output layer to GPU
|
||||
load_tensors: offloaded 37/37 layers to GPU
|
||||
load_tensors: Vulkan0 model buffer size = 4454.74 MiB
|
||||
load_tensors: CPU_Mapped model buffer size = 333.43 MiB
|
||||
.....................................................................................
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 1000000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (40960) -- the full capacity of the model will not be utilized
|
||||
llama_context: Vulkan_Host output buffer size = 0.58 MiB
|
||||
llama_kv_cache: Vulkan0 KV buffer size = 288.00 MiB
|
||||
llama_kv_cache: size = 288.00 MiB ( 2048 cells, 36 layers, 1/1 seqs), K (f16): 144.00 MiB, V (f16): 144.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 304.38 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 12.01 MiB
|
||||
llama_context: graph nodes = 1267
|
||||
llama_context: graph splits = 2
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 25673 MiB free
|
||||
llama_model_loader: loaded meta data with 30 key-value pairs and 193 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = llama
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Tts
|
||||
llama_model_loader: - kv 3: general.size_label str = 445M
|
||||
llama_model_loader: - kv 4: llama.block_count u32 = 20
|
||||
llama_model_loader: - kv 5: llama.context_length u32 = 4096
|
||||
llama_model_loader: - kv 6: llama.embedding_length u32 = 768
|
||||
llama_model_loader: - kv 7: llama.feed_forward_length u32 = 3072
|
||||
llama_model_loader: - kv 8: llama.attention.head_count u32 = 12
|
||||
llama_model_loader: - kv 9: llama.attention.head_count_kv u32 = 12
|
||||
llama_model_loader: - kv 10: llama.rope.freq_base f32 = 10000.000000
|
||||
llama_model_loader: - kv 11: llama.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: llama.attention.key_length u32 = 64
|
||||
llama_model_loader: - kv 13: llama.attention.value_length u32 = 64
|
||||
llama_model_loader: - kv 14: general.file_type u32 = 1
|
||||
llama_model_loader: - kv 15: llama.vocab_size u32 = 158626
|
||||
llama_model_loader: - kv 16: llama.rope.dimension_count u32 = 64
|
||||
llama_model_loader: - kv 17: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.tokens arr[str,158626] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.token_type arr[i32,158626] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 24: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 25: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 26: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 27: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 28: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 29: tokenizer.ggml.add_space_prefix bool = false
|
||||
llama_model_loader: - type f32: 50 tensors
|
||||
llama_model_loader: - type f16: 143 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = F16
|
||||
print_info: file size = 1.07 GiB (20.70 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = llama
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 4096
|
||||
print_info: n_embd = 768
|
||||
print_info: n_layer = 20
|
||||
print_info: n_head = 12
|
||||
print_info: n_head_kv = 12
|
||||
print_info: n_rot = 64
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 64
|
||||
print_info: n_embd_head_v = 64
|
||||
print_info: n_gqa = 1
|
||||
print_info: n_embd_k_gqa = 768
|
||||
print_info: n_embd_v_gqa = 768
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 3072
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = 0
|
||||
print_info: rope type = 0
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 10000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 4096
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = ?B
|
||||
print_info: model params = 444.94 M
|
||||
print_info: general.name = Tts
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 158626
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
load_tensors: skipping TTS tensor emb_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor emb_text.weight (size = 467140608 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.weight (size = 12582912 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.weight (size = 2359296 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.weight (size = 6291456 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.weight (size = 1179648 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor head_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 0 repeating layers to GPU
|
||||
load_tensors: offloaded 0/21 layers to GPU
|
||||
load_tensors: CPU_Mapped model buffer size = 592.48 MiB
|
||||
..............................................................
|
||||
llama_init_from_model: model default pooling_type is [0], but [-1] was specified
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 10000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (4096) -- the full capacity of the model will not be utilized
|
||||
llama_context: CPU output buffer size = 0.61 MiB
|
||||
llama_kv_cache: CPU KV buffer size = 120.00 MiB
|
||||
llama_kv_cache: size = 120.00 MiB ( 2048 cells, 20 layers, 1/1 seqs), K (f16): 60.00 MiB, V (f16): 60.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 30.75 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 7.01 MiB
|
||||
llama_context: graph nodes = 613
|
||||
llama_context: graph splits = 216 (with bs=512), 1 (with bs=1)
|
||||
vision_model_loader: model name:
|
||||
vision_model_loader: description: image encoder for MiniCPM-V
|
||||
vision_model_loader: GGUF version: 3
|
||||
vision_model_loader: alignment: 32
|
||||
vision_model_loader: n_tensors: 455
|
||||
vision_model_loader: n_kv: 20
|
||||
|
||||
vision_ctx: vision using Vulkan0 backend
|
||||
load_hparams: model_type: 0
|
||||
load_hparams: n_embd: 1152
|
||||
load_hparams: n_head: 16
|
||||
load_hparams: n_ff: 4304
|
||||
load_hparams: n_layer: 27
|
||||
load_hparams: ffn_op: gelu
|
||||
load_hparams: projection_dim: 0
|
||||
|
||||
--- vision hparams ---
|
||||
load_hparams: image_size: 448
|
||||
load_hparams: patch_size: 14
|
||||
load_hparams: minicpmv_version: 100045
|
||||
load_hparams: proj_scale_factor: 0
|
||||
load_hparams: n_wa_pattern: 0
|
||||
|
||||
load_hparams: model size: 1044.36 MiB
|
||||
load_hparams: metadata size: 0.16 MiB
|
||||
load_tensors: ffn up/down are swapped
|
||||
alloc_compute_meta: Vulkan0 compute buffer size = 100.30 MiB
|
||||
alloc_compute_meta: CPU compute buffer size = 18.30 MiB
|
||||
init tts....init t2w....voc_hg2_model: CPU backend using 8 threads
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=32000
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=200, n_mel=80, total_size=16000
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=200, n_mel=80, total_elements=16000
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=200, n_tokens=100, current_iter=0, current_total=0, new_total=100, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 1 (total_tokens=100, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 20]
|
||||
audition_audio_batch_encode: Final output: 20 tokens x 4096 dims
|
||||
=== Model Paths ===
|
||||
Base dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf
|
||||
LLM: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf [OK]
|
||||
Vision: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/vision/MiniCPM-o-4_5-vision-F16.gguf [OK]
|
||||
Audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf [OK]
|
||||
TTS: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf [OK]
|
||||
Projector: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf [OK]
|
||||
===================
|
||||
=== Initializing Omni Context ===
|
||||
Media type: 2 (omni: audio+vision)
|
||||
TTS enabled: yes
|
||||
Context size: 2048
|
||||
GPU layers: 99
|
||||
Token2Wav device: cpu
|
||||
Vision backend: metal
|
||||
TTS bin dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts
|
||||
Ref audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\vllm-omni-edge\benchmarks\edge_harness\results\e2e_expansion_20260923\evidence\minicpmo_cpp_radeon890m_prompted\fixture_0000.wav
|
||||
03:01:55.319 === omni_init start
|
||||
03:01:55.319 media_type = 2, duplex_mode = 0, base_output_dir = ./tools/omni/output
|
||||
03:01:55.319 === omni_init: loading new LLM model
|
||||
03:02:48.504 === omni_init: loading TTS model
|
||||
03:02:48.504 TTS model: loading with n_gpu_layers=0
|
||||
03:02:52.165 TTS sampler: temp=0.80, top_p=0.85, top_k=25, rep_penalty=1.05
|
||||
03:02:52.165 TTS: loading weights from GGUF (emb_code, emb_text, projector_semantic, head_code)...
|
||||
03:02:57.684 TTS: head_code shape: dim0=6562, dim1=768, will transpose to [6562, 768]
|
||||
03:02:57.691 TTS: head_code loaded, verifying first few values:
|
||||
03:02:57.691 head_code_weight[0] = 0.01361084 (expect ~0.01385498)
|
||||
03:02:57.691 head_code_weight[1] = -0.01660156 (expect ~-0.01647949)
|
||||
03:02:57.691 head_code_weight[768] = 0.04101562 (this is weight[1, 0], expect ~0.04150391)
|
||||
03:02:57.744 TTS: weights loaded successfully
|
||||
03:02:57.744 Projector: loading from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf
|
||||
03:02:57.839 Projector: loaded successfully
|
||||
03:02:57.839 === omni_init: loading APM model
|
||||
03:03:00.873 APM: init from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf
|
||||
03:03:06.156 Token2Wav: found models in \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf
|
||||
03:03:06.210 Token2Wav: all model files found, initializing session...
|
||||
03:03:06.210 Token2Wav: non-CUDA backend, vocoder using CPU for better performance
|
||||
03:03:06.210 Token2Wav: using prompt_cache from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf/prompt_cache.gguf
|
||||
03:03:10.653 Token2Wav: initialized successfully
|
||||
03:03:10.653 Python T2W: script_dir=./tools/omni/pyt2w, model_dir=./tools/omni/pyt2w/token2wav
|
||||
03:03:10.653 Token2Wav: 浣跨敤 C++ 瀹炵幇
|
||||
03:03:10.653 === omni_init success: ctx_llama = 0000024c7a50f830
|
||||
=== Running test case ===
|
||||
Audio prefix: C:\Users\zhout\w2\minicpmo_omni_cold_after_work_radeon\omni-minicpmo-1-kx55ncuz\fixture_
|
||||
Count: 2
|
||||
03:03:10.654 stream_prefill: n_past = 0
|
||||
voice_clone_prompt = <|im_start|>system
|
||||
妯′豢闊抽�鏍锋湰鐨勯煶鑹插苟鐢熸垚鏂扮殑鍐呭�銆�
|
||||
<|audio_start|>
|
||||
assistant_prompt = <|audio_end|>浣犵殑浠诲姟鏄�敤杩欑�澹伴煶妯″紡鏉ュ綋涓€涓�姪鎵嬨€傝�璁ょ湡銆侀珮璐ㄩ噺鍦板洖澶嶇敤鎴风殑闂��銆傝�鐢ㄩ珮鑷�劧搴︾殑鏂瑰紡鍜岀敤鎴疯亰澶┿€�<|im_end|>
|
||||
<|im_start|>user
|
||||
|
||||
03:03:10.654 system prompt ref_audio: C:\Users\zhout\w2\minicpmo_omni_cold_after_work_radeon\omni-minicpmo-1-kx55ncuz\fixture_0000.wav
|
||||
03:03:11.016 system prompt ref_audio embedding: n_pos=20
|
||||
03:03:11.515 馃敀 n_keep 璁剧疆涓� 72 (system prompt tokens)锛岃繖閮ㄥ垎姘歌繙涓嶄細琚�粦鍔ㄧ獥鍙ｅ垹闄�
|
||||
03:03:11.515 stream_prefill(index=0): system prompt 鍒濆�鍖栧畬鎴愶紝ref_audio 宸插湪鍏朵腑 prefill
|
||||
03:03:11.515 [SW] system_prompt registered: preserve_length=72 (will be protected from sliding)
|
||||
03:03:11.515 n_past = 72
|
||||
03:03:11.515
|
||||
|
||||
c++ finish stream_prefill(index=0). n_past=72, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 0 (audio) : 0.86122 s
|
||||
encode_image_with_vision_chunks: step 1 of 1 encoded in 630.40 ms
|
||||
encode_image_with_vision_chunks: all 1 chunks encoded in 648.45 ms (grid: 1x1)
|
||||
|
||||
encode_image_with_vision_chunks: image encoded in 648.45 ms by vision ( 648.45 ms per chunk, 64 total tokens)
|
||||
omni_image_embed_make_chunks_with_filename: created 1 vision chunks from C:\Users\zhout\w2\minicpmo_omni_cold_after_work_radeon\omni-minicpmo-1-kx55ncuz\fixture_0001.jpg
|
||||
stream_prefill: prefilled 1 vision chunks (64 tokens each)
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=49339
|
||||
audition_audio_preprocess: Audio not aligned to 100ms (49339 samples = 3083.7ms), padded with silence to 49600 samples (3100ms)
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=310, n_mel=80, total_size=24800
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=310, n_mel=80, total_elements=24800
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=310, n_tokens=155, current_iter=1, current_total=155, new_total=310, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 2 (total_tokens=310, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 31]
|
||||
audition_audio_batch_encode: Final output: 31 tokens x 4096 dims
|
||||
03:03:12.695 stream_prefill(index=1): processing user audio: C:\Users\zhout\w2\minicpmo_omni_cold_after_work_radeon\omni-minicpmo-1-kx55ncuz\fixture_0001.wav
|
||||
03:03:12.974 stream_prefill(index=1): user audio embedding: n_pos=31
|
||||
03:03:13.285
|
||||
|
||||
c++ finish stream_prefill(index=1). n_past=172, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 1 (audio+vision) : 1.7681 s
|
||||
03:03:13.285 馃搷 stream_decode 寮€濮�: n_past=172, n_keep=72, n_ctx=2048, duplex_mode=0
|
||||
03:03:13.285 stream_decode: create tts thread (simplex mode)
|
||||
03:03:13.285 stream_decode: create t2w thread
|
||||
03:03:13.285 wait prefill done
|
||||
03:03:13.285 馃搷 [鍗曞伐TTS] 娣诲姞 assistant prompt: "<|im_end|>
|
||||
<|im_start|>assistant
|
||||
<think>
|
||||
|
||||
</think>
|
||||
|
||||
<|tts_bos|>", n_past=172
|
||||
03:03:13.286 T2W thread (C++) started
|
||||
<user>
|
||||
<assistant>03:03:13.289 TTS: 鍒涘缓鍗曞伐妯″紡杈撳嚭鐩�綍: ./tools/omni/output/round_000
|
||||
03:03:13.290 TTS thread started
|
||||
03:03:13.449 馃搷 [鍗曞伐TTS] assistant prompt 瀹屾垚, n_past=182
|
||||
03:03:13.450 LLM decode: max_tgt_len = 96, n_predict = 96, n_ctx = 2048
|
||||
03:03:13.970 LLM: detected end token, set llm_generation_done=true
|
||||
03:03:13.970 LLM->TTS: text='The square in the image is red.', n_tokens=8, hidden_size=32768, n_embd=4096, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
03:03:13.970 馃搷 Decode 缁撴潫: n_past=191, 鍓╀綑绌洪棿=1857, 鏃犻渶婊戠獥
|
||||
03:03:13.970 馃搷 杞�� 1 缁撴潫锛岃�褰曡竟鐣屼簬 n_past=191
|
||||
03:03:13.970 TTS<-LLM: chunk_idx=0, text='The square in the image is red.', n_tokens=8, hidden_size=32768, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
03:03:13.971 TTS: after queue pop - speek_done=0, llm_finish=1, llm_text.empty=0, token_ids.size=8
|
||||
03:03:13.971 TTS: before empty check - response.empty=0, response='The square in the image is red.', llm_finish=1
|
||||
Waiting for audio generation to complete...
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_0.wav does not exist or is empty
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_1.wav does not exist or is empty
|
||||
TTS: no valid WAV files to merge
|
||||
03:03:13.971 TTS: DEBUG before has_llm_data - token_ids.size=8, hidden_states.size=32768, n_embd=4096
|
||||
03:03:13.971 TTS: has_llm_data=1, token_ids=8, hidden_states=32768, n_embd=4096, llm_finish=1
|
||||
03:03:13.972 TTS: n_tokens_orig=8, n_tokens_filtered=8 (filtered 0 special tokens)
|
||||
03:03:14.001 TTS: saved merged embeddings (text) to ./tools/omni/output/round_000/llm_debug/chunk_0/merged_embeddings.txt
|
||||
03:03:14.001 TTS Simplex: generating audio tokens for chunk 0 (n_tokens=8, tts_n_embd=768)
|
||||
03:03:14.001 TTS Simplex: 鍦� prefill 鍓嶆坊鍔� audio_bos (chunk_idx=0, new_size=9)
|
||||
03:03:14.001 TTS Simplex: first chunk - cleared KV cache
|
||||
03:03:14.001 TTS Simplex: first chunk - cleared tts_all_generated_tokens and tts_token_buffer
|
||||
03:03:14.064 馃搷 涓轰笅涓€杞�噯澶�: eval <|im_end|>\n<|im_start|>user\n, n_past=196
|
||||
03:03:14.264 TTS Simplex: prefill completed, n_past_tts=9
|
||||
03:03:14.264 TTS Simplex: sampler created
|
||||
03:03:14.264 TTS simplex: is_audio_bos=true (first audio token)
|
||||
03:03:14.289 TTS Simplex Phase1: token 1/500: rel_id=1490
|
||||
03:03:14.300 TTS Simplex Phase1: token 2/500: rel_id=4218
|
||||
03:03:14.312 TTS Simplex Phase1: token 3/500: rel_id=4299
|
||||
03:03:14.322 TTS Simplex Phase1: token 4/500: rel_id=6486
|
||||
03:03:14.333 TTS Simplex Phase1: token 5/500: rel_id=4299
|
||||
03:03:14.550 TTS Simplex Phase1: yield 25 tokens 鍒� T2W (step 25, buffer=25)
|
||||
03:03:14.550 TTS Simplex Phase1: token 25/500: rel_id=1509
|
||||
03:03:14.550 T2W绾跨▼(C++): 杞��鍒囨崲 (-1 -> 0)锛屾洿鏂拌緭鍑虹洰褰�
|
||||
03:03:14.550 T2W绾跨▼(C++): 鏂拌緭鍑虹洰褰� ./tools/omni/output/round_000/tts_wav
|
||||
03:03:14.815 TTS Simplex Phase1: EOS token at step 50
|
||||
03:03:14.815 TTS Simplex: is_final_text_chunk=true, will enter Phase 2 for text_eos_embed
|
||||
03:03:14.815 TTS Simplex Phase1: token 50/500: rel_id=6561
|
||||
03:03:14.815 TTS Simplex Phase2: injecting text_eos_embed + audio_bos at n_past=58
|
||||
03:03:14.824 TTS Simplex Phase2: text_eos_embed injected, n_past=59
|
||||
03:03:14.831 TTS Simplex Phase2: audio_bos injected, n_past=60, starting final generation
|
||||
03:03:14.842 TTS Simplex Phase2: yield 25 tokens 鍒� T2W
|
||||
03:03:15.081 TTS Simplex Phase2: EOS at step 23 (final end)
|
||||
03:03:15.081 TTS Simplex: chunk 缁撴潫锛宼ts_token_buffer 鍓╀綑 21 tokens (淇濈暀绛変笅涓€涓� chunk)
|
||||
03:03:15.081 TTS Simplex: is_final_text_chunk=true, flush 鍓╀綑 21 tokens 浠� tts_token_buffer
|
||||
03:03:15.081 TTS Simplex: generated 71 audio tokens for chunk 0
|
||||
03:03:15.081 TTS Simplex: chunk 0 first tokens: [1490 4218 4299 6486 4299 ], last tokens: [6486 6486 4299 4299 6162 ]
|
||||
03:03:15.081 TTS Simplex: updated tts_n_past_accumulated=83, total_generated_tokens=71
|
||||
03:03:15.090 TTS: finished processing all chunks
|
||||
03:03:15.090 TTS: 鍗曞伐妯″紡杞��缁撴潫锛屼笅涓€杞� round_idx=1
|
||||
03:03:15.090 TTS: sent is_final=true to T2W queue (turn end)
|
||||
03:03:16.936 馃帀 棣栧搷鏃堕棿 (First Audio Response): 3650ms
|
||||
03:03:16.936 T2W绾跨▼: wav_0.wav | 0.84s audio | 2384.9ms inference | RTF=2.84 | t=3650ms
|
||||
03:03:19.173 T2W绾跨▼: wav_1.wav | 1.00s audio | 2236.5ms inference | RTF=2.24 | t=5887ms
|
||||
03:03:21.410 T2W绾跨▼: wav_2.wav | 1.12s audio | 2236.3ms inference | RTF=2.00 | t=8124ms
|
||||
03:03:21.410 T2W绾跨▼: 鍐欏叆缁撴潫鏍囪� ./tools/omni/output/round_000/tts_wav/generation_done.flag (last_wav=2)
|
||||
03:03:21.410 T2W绾跨▼: 杞��缁撴潫鍚庢洿鏂拌緭鍑虹洰褰曚负 ./tools/omni/output/round_001/tts_wav
|
||||
03:03:03:21.472 03:omni_stop_threads: stop signals sent
|
||||
21.472 T2W(C++) 绾跨▼: 鍋滄�
|
||||
[timing] call=0 tokens=28 final=0 token2mel=2051.438ms vocoder=333.439ms total=2384.898ms audio=20160
|
||||
[timing] call=1 tokens=28 final=0 token2mel=1855.506ms vocoder=380.973ms total=2236.497ms audio=24000
|
||||
[timing] call=2 tokens=24 final=1 token2mel=1865.654ms vocoder=370.579ms total=2236.243ms audio=26880
|
||||
Audio generation completed.
|
||||
llama_perf_context_print: load time = 24790.65 ms
|
||||
llama_perf_context_print: prompt eval time = 2795.33 ms / 182 tokens ( 15.36 ms per token, 65.11 tokens per second)
|
||||
llama_perf_context_print: eval time = 518.11 ms / 9 runs ( 57.57 ms per token, 17.37 tokens per second)
|
||||
llama_perf_context_print: total time = 32812.85 ms / 191 tokens
|
||||
llama_perf_context_print: graphs reused = 10
|
||||
Token2Wav (C++): session released
|
||||
tts thread end
|
||||
t2w thread end
|
||||
+727
@@ -0,0 +1,727 @@
|
||||
{
|
||||
"scope": "MiniCPM-o GGUF complete audio+image\u2192text+speech through Omni StageRuntime/StagePool",
|
||||
"placement": "radeon-hybrid",
|
||||
"model_revision": "db25077c33951fe163b42986fba0132e279872a2",
|
||||
"artifact_sha256": {
|
||||
"MiniCPM-o-4_5-Q4_K_M.gguf": "1237a97ee081b8abebc47aa7dad565701e8f5f904cdc92f6723ac4281bbc0932",
|
||||
"audio/MiniCPM-o-4_5-audio-F16.gguf": "d5b188ac7feaf98e17175c3f9bd14bf269301bfd187439fdaa3e3a494fc32ef7",
|
||||
"vision/MiniCPM-o-4_5-vision-F16.gguf": "1453678cc4e4fe18de241952962e234f265cb8dda780773526103ab8ba82f421",
|
||||
"tts/MiniCPM-o-4_5-tts-F16.gguf": "c7be3748a863dd6966ae7eed42600b7f41ca67affb03729ff245247f0e5ea088",
|
||||
"tts/MiniCPM-o-4_5-projector-F16.gguf": "4b1b5b377358a5e594a304ff6ea5d52df606a9ba7d886c4299d232f0c67dd1fd",
|
||||
"token2wav-gguf/encoder.gguf": "7f8d265da594eaf5e1de2db8f5f1867dbcb0bb75ef5878fadf2952347116f4d0",
|
||||
"token2wav-gguf/flow_matching.gguf": "eda6069f3edeb5dd3a87fbf2aedb2ddd1b46f3273926c4fcf09b24476a39cab8",
|
||||
"token2wav-gguf/flow_extra.gguf": "c67611aa7d02500fe395a7798bf0bfdfb55c74d37ba93934ca74d82b4e63f78d",
|
||||
"token2wav-gguf/hifigan2.gguf": "1b8b3bf5d8d3066aeee4324fdcdd41aefce170d0ee907645858de408d82835c2",
|
||||
"token2wav-gguf/prompt_cache.gguf": "81fe6f541ebe0b67db06a1e395df928da47991dc9637c2cf47d6c59d5b979f2c"
|
||||
},
|
||||
"cli_sha256": "66a512386aafeb708f49a7394a9054d94ec777c362395c8917a3d198aef1b523",
|
||||
"reference_sha256": "cf4bb5cd9acbfbf67596d4a53310652b61ef4b7ede1d54eb03ef4882ca40223a",
|
||||
"input_sha256": {
|
||||
"audio_wav": "14ee147cbf2ce9afd9e32c8c8228d7cfe11985cd7e63456b2ad33871492aefcc",
|
||||
"image_jpeg": "22c7ccea466a3f551c512045e6316c7597408d4e0a20205502f55784d977aa8d"
|
||||
},
|
||||
"memory_budget": {
|
||||
"capacities": {
|
||||
"host_ram": 22548578304
|
||||
},
|
||||
"demands": {
|
||||
"host_ram": 22548578304
|
||||
}
|
||||
},
|
||||
"profile_count": 20,
|
||||
"warmup_count": 1,
|
||||
"host_available_bytes_before": 23804850176,
|
||||
"startup_s": 51.91764660004992,
|
||||
"execution_plan_at_start": {
|
||||
"backend": "external.minicpmo.gguf.v1",
|
||||
"stage_id": 0,
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"model_revision": "db25077c33951fe163b42986fba0132e279872a2",
|
||||
"artifact_sha256": {
|
||||
"MiniCPM-o-4_5-Q4_K_M.gguf": "1237a97ee081b8abebc47aa7dad565701e8f5f904cdc92f6723ac4281bbc0932",
|
||||
"audio/MiniCPM-o-4_5-audio-F16.gguf": "d5b188ac7feaf98e17175c3f9bd14bf269301bfd187439fdaa3e3a494fc32ef7",
|
||||
"vision/MiniCPM-o-4_5-vision-F16.gguf": "1453678cc4e4fe18de241952962e234f265cb8dda780773526103ab8ba82f421",
|
||||
"tts/MiniCPM-o-4_5-tts-F16.gguf": "c7be3748a863dd6966ae7eed42600b7f41ca67affb03729ff245247f0e5ea088",
|
||||
"tts/MiniCPM-o-4_5-projector-F16.gguf": "4b1b5b377358a5e594a304ff6ea5d52df606a9ba7d886c4299d232f0c67dd1fd",
|
||||
"token2wav-gguf/encoder.gguf": "7f8d265da594eaf5e1de2db8f5f1867dbcb0bb75ef5878fadf2952347116f4d0",
|
||||
"token2wav-gguf/flow_matching.gguf": "eda6069f3edeb5dd3a87fbf2aedb2ddd1b46f3273926c4fcf09b24476a39cab8",
|
||||
"token2wav-gguf/flow_extra.gguf": "c67611aa7d02500fe395a7798bf0bfdfb55c74d37ba93934ca74d82b4e63f78d",
|
||||
"token2wav-gguf/hifigan2.gguf": "1b8b3bf5d8d3066aeee4324fdcdd41aefce170d0ee907645858de408d82835c2",
|
||||
"token2wav-gguf/prompt_cache.gguf": "81fe6f541ebe0b67db06a1e395df928da47991dc9637c2cf47d6c59d5b979f2c"
|
||||
},
|
||||
"cli_sha256": "66a512386aafeb708f49a7394a9054d94ec777c362395c8917a3d198aef1b523",
|
||||
"requested_placement": "radeon-hybrid",
|
||||
"ggml_vk_visible_devices": "1",
|
||||
"shared_ram_reservation": {
|
||||
"host_ram": 22548578304
|
||||
},
|
||||
"weights_bytes": 8872297088,
|
||||
"memory_overhead_bytes": 12884901888,
|
||||
"context_tokens": 2048,
|
||||
"max_new_tokens": 96,
|
||||
"request_capacity": 1,
|
||||
"stateful_session": "C++-owned within one request; discarded after completion",
|
||||
"placement_evidence": "pending first request"
|
||||
},
|
||||
"warmups": [
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 88.43239450000692,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 56640,
|
||||
"duration_s": 2.36,
|
||||
"pcm_sha256": "ceabf4ec4fd61ebac2b4e316c06fbf961aafa371a278ae21a76e6b5b2042b5a3"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-warmup-0",
|
||||
"stage_id": 0,
|
||||
"epoch": 1,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 88.40392130007967,
|
||||
"peak_sampled_rss_bytes": 13763682304,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-1-27686b55445e443f955c0a221e4e72ed.log"
|
||||
}
|
||||
}
|
||||
],
|
||||
"measured": [
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 89.67069400008768,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 59520,
|
||||
"duration_s": 2.48,
|
||||
"pcm_sha256": "28d1c312e3da6e034d1063ccc4f3c927308f42d6c02de197e620afabcc4a53a3"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-0",
|
||||
"stage_id": 0,
|
||||
"epoch": 2,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 89.66583519999404,
|
||||
"peak_sampled_rss_bytes": 15297159168,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-2-765d0081ff11481199ba99c70808d3f7.log"
|
||||
}
|
||||
},
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 89.2410724000074,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 66240,
|
||||
"duration_s": 2.76,
|
||||
"pcm_sha256": "b3a801e56e5c280aec423c5f29abb96a9096d917f9d457021861dfc2dab2091a"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-1",
|
||||
"stage_id": 0,
|
||||
"epoch": 3,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 89.22992990002967,
|
||||
"peak_sampled_rss_bytes": 15297896448,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-3-5a5e09d95930426493ceb0d3c5d0f667.log"
|
||||
}
|
||||
},
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 84.44369159999769,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 49920,
|
||||
"duration_s": 2.08,
|
||||
"pcm_sha256": "1b17ec4945b7db9d69084f545bea331d217c104d380030c4d7dd588195c033e2"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-2",
|
||||
"stage_id": 0,
|
||||
"epoch": 4,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 84.43399200006388,
|
||||
"peak_sampled_rss_bytes": 15301582848,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-4-9ea9b162dbbc4e99b2764d814900c4f5.log"
|
||||
}
|
||||
},
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 91.44574589992408,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 59520,
|
||||
"duration_s": 2.48,
|
||||
"pcm_sha256": "c153a4ee757314dbfa93692807d760737b0a8fdf75bce4dfe1400d090723bced"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-3",
|
||||
"stage_id": 0,
|
||||
"epoch": 5,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 91.43500569998287,
|
||||
"peak_sampled_rss_bytes": 15296962560,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-5-3f3a853496f84850a3b8d3e59ee80328.log"
|
||||
}
|
||||
},
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 97.89790800004266,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 67200,
|
||||
"duration_s": 2.8,
|
||||
"pcm_sha256": "e606c42e79f1682ebb38ffd5999be2d9d71136ed3bbbb206b54b13959b626b49"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-4",
|
||||
"stage_id": 0,
|
||||
"epoch": 6,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 97.88728310004808,
|
||||
"peak_sampled_rss_bytes": 15296348160,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-6-765c426a238146cca67a2cf5a31dcec6.log"
|
||||
}
|
||||
},
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 86.5171454999363,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 57600,
|
||||
"duration_s": 2.4,
|
||||
"pcm_sha256": "aa40a934ebe0fa335e64a17760cdb69db78f020237999ad3aa2a8e57738ef8b7"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-5",
|
||||
"stage_id": 0,
|
||||
"epoch": 7,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 86.50678499997593,
|
||||
"peak_sampled_rss_bytes": 15297114112,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-7-509ef5e86bce44f3962216500758e762.log"
|
||||
}
|
||||
},
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 87.32478409993928,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 49920,
|
||||
"duration_s": 2.08,
|
||||
"pcm_sha256": "4275292fc770992bcddfcc773e5ada7ffc14f959900adae01c48d8368add53ab"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-6",
|
||||
"stage_id": 0,
|
||||
"epoch": 8,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 87.31482480000705,
|
||||
"peak_sampled_rss_bytes": 15300722688,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-8-7919065c767c42ddb58856319722e6d0.log"
|
||||
}
|
||||
},
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 69.91491160006262,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 72000,
|
||||
"duration_s": 3.0,
|
||||
"pcm_sha256": "f9a88dc9cc294ac2ded1d81e076ef571045af818ba540554ff9ba422df22943e"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-7",
|
||||
"stage_id": 0,
|
||||
"epoch": 9,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 69.90729420003481,
|
||||
"peak_sampled_rss_bytes": 15296557056,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-9-b84f40b2f7e54848ad7d8137a4b602c9.log"
|
||||
}
|
||||
},
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 87.5758816000307,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 61440,
|
||||
"duration_s": 2.56,
|
||||
"pcm_sha256": "527ce564235f3781e74a41710c0e5a4fae27989d19ff233ee64efad5d533e9b1"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-8",
|
||||
"stage_id": 0,
|
||||
"epoch": 10,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 87.56872690003365,
|
||||
"peak_sampled_rss_bytes": 15296552960,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-10-45806b84a1394f5f8fc45d8638ef6504.log"
|
||||
}
|
||||
},
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 87.2738595000701,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 57600,
|
||||
"duration_s": 2.4,
|
||||
"pcm_sha256": "a1110fdc2f1945112bccf1566f51bdb65a2ddc36e772584f747a3b8be95b2b7e"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-9",
|
||||
"stage_id": 0,
|
||||
"epoch": 11,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 87.26336149999406,
|
||||
"peak_sampled_rss_bytes": 15296577536,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-11-5bf017cff9ff4d59b3f59c2fc3bf2266.log"
|
||||
}
|
||||
},
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 80.4882060000673,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 64320,
|
||||
"duration_s": 2.68,
|
||||
"pcm_sha256": "0d070361d5a2d08a0d5234144f9e69256e169ead3a543b4671ee9753a34e244a"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-10",
|
||||
"stage_id": 0,
|
||||
"epoch": 12,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 80.47764310007915,
|
||||
"peak_sampled_rss_bytes": 15296012288,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-12-61bda20e0fa141738dde2bd58c1311e3.log"
|
||||
}
|
||||
},
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 85.45749559998512,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 59520,
|
||||
"duration_s": 2.48,
|
||||
"pcm_sha256": "1ff503a03b0411eb781e9425edaf256b303ca129abef1d6714cad9fc57589954"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-11",
|
||||
"stage_id": 0,
|
||||
"epoch": 13,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 85.44423809996806,
|
||||
"peak_sampled_rss_bytes": 15295561728,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-13-b9ce8af2384f4702bc457ff245859deb.log"
|
||||
}
|
||||
},
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 89.06362529995386,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 55680,
|
||||
"duration_s": 2.32,
|
||||
"pcm_sha256": "75f1825c97b1229a787277a1695eaa7749bed87fb459394ff259261052320179"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-12",
|
||||
"stage_id": 0,
|
||||
"epoch": 14,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 89.05177350004669,
|
||||
"peak_sampled_rss_bytes": 15295856640,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-14-6a2560adbf8c404ba4ecb9b5bafe0f33.log"
|
||||
}
|
||||
},
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 86.04584340006113,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 61440,
|
||||
"duration_s": 2.56,
|
||||
"pcm_sha256": "e20ed2cc01012a362be35a1cae41d4fd41eed2f268ea9be12d6990dcec780502"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-13",
|
||||
"stage_id": 0,
|
||||
"epoch": 15,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 86.03722810000181,
|
||||
"peak_sampled_rss_bytes": 15295610880,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-15-84c5ffad02684029bafbca2aba2106af.log"
|
||||
}
|
||||
},
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 86.03778659994714,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 61440,
|
||||
"duration_s": 2.56,
|
||||
"pcm_sha256": "c8b5d1af4328d0ee202d592b4c8b48603f7f5505fffba8d8e513407d866ec90a"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-14",
|
||||
"stage_id": 0,
|
||||
"epoch": 16,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 86.03154140000697,
|
||||
"peak_sampled_rss_bytes": 15296053248,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-16-f66088565b534d1ab2749109959f8d63.log"
|
||||
}
|
||||
},
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 89.40287510002963,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 53760,
|
||||
"duration_s": 2.24,
|
||||
"pcm_sha256": "053e3a14170269a10218e678ade4223e28b2653a28ea75c15621c02d03a3b041"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-15",
|
||||
"stage_id": 0,
|
||||
"epoch": 17,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 89.39397819992155,
|
||||
"peak_sampled_rss_bytes": 15296118784,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-17-1dbbf8d44c554b0ab0e9d63d3d3c6cfb.log"
|
||||
}
|
||||
},
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 87.37631849991158,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 60480,
|
||||
"duration_s": 2.52,
|
||||
"pcm_sha256": "7b14deb1f5215fc003ec006cf68623644862aa33457d2ad3dcb8663b05472f12"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-16",
|
||||
"stage_id": 0,
|
||||
"epoch": 18,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 87.36507729999721,
|
||||
"peak_sampled_rss_bytes": 15296081920,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-18-0d33268ae2ec4cca9d14bbaa000aed11.log"
|
||||
}
|
||||
},
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 84.44159840000793,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 57600,
|
||||
"duration_s": 2.4,
|
||||
"pcm_sha256": "bd002f117e6873122151b2c4dc8267f5c2de2e98a464c890a076f11c27dcf972"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-17",
|
||||
"stage_id": 0,
|
||||
"epoch": 19,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 84.43294430000242,
|
||||
"peak_sampled_rss_bytes": 15296143360,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-19-a1fe46c104ba46409bbcd1553640fab1.log"
|
||||
}
|
||||
},
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 86.14088329998776,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 51840,
|
||||
"duration_s": 2.16,
|
||||
"pcm_sha256": "7c1a2617b1b315a67bd35854acaed5e1f0fad877858682f2f257e8c7bb8b9cfd"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-18",
|
||||
"stage_id": 0,
|
||||
"epoch": 20,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 86.13427959999535,
|
||||
"peak_sampled_rss_bytes": 15296356352,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-20-60049a03aa98440796d4f3d095fab1da.log"
|
||||
}
|
||||
},
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 90.84336910001002,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 65280,
|
||||
"duration_s": 2.72,
|
||||
"pcm_sha256": "05552b37ab9df6d0e6c15fa854219b9d3bf53f7cc90fa62e1f66af145f4f3fa3"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-19",
|
||||
"stage_id": 0,
|
||||
"epoch": 21,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 90.83368909999263,
|
||||
"peak_sampled_rss_bytes": 15295959040,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-21-5a4d63dd3dcb4f3c92057150efda2aa9.log"
|
||||
}
|
||||
}
|
||||
],
|
||||
"complete_request": {
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 89.67069400008768,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 59520,
|
||||
"duration_s": 2.48,
|
||||
"pcm_sha256": "28d1c312e3da6e034d1063ccc4f3c927308f42d6c02de197e620afabcc4a53a3"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-0",
|
||||
"stage_id": 0,
|
||||
"epoch": 2,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 89.66583519999404,
|
||||
"peak_sampled_rss_bytes": 15297159168,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-2-765d0081ff11481199ba99c70808d3f7.log"
|
||||
}
|
||||
},
|
||||
"nearest_rank_p50_wall_s": 87.2738595000701,
|
||||
"nearest_rank_p95_wall_s": 91.44574589992408,
|
||||
"output_wav": "\\\\wsl.localhost\\Ubuntu\\home\\zhout\\project\\edge_infer\\vllm-omni-edge\\benchmarks\\edge_harness\\results\\e2e_expansion_20260923\\evidence\\minicpmo_omni_radeon_profile20\\profile20.wav",
|
||||
"execution_plan_after_request": {
|
||||
"backend": "external.minicpmo.gguf.v1",
|
||||
"stage_id": 0,
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"model_revision": "db25077c33951fe163b42986fba0132e279872a2",
|
||||
"artifact_sha256": {
|
||||
"MiniCPM-o-4_5-Q4_K_M.gguf": "1237a97ee081b8abebc47aa7dad565701e8f5f904cdc92f6723ac4281bbc0932",
|
||||
"audio/MiniCPM-o-4_5-audio-F16.gguf": "d5b188ac7feaf98e17175c3f9bd14bf269301bfd187439fdaa3e3a494fc32ef7",
|
||||
"vision/MiniCPM-o-4_5-vision-F16.gguf": "1453678cc4e4fe18de241952962e234f265cb8dda780773526103ab8ba82f421",
|
||||
"tts/MiniCPM-o-4_5-tts-F16.gguf": "c7be3748a863dd6966ae7eed42600b7f41ca67affb03729ff245247f0e5ea088",
|
||||
"tts/MiniCPM-o-4_5-projector-F16.gguf": "4b1b5b377358a5e594a304ff6ea5d52df606a9ba7d886c4299d232f0c67dd1fd",
|
||||
"token2wav-gguf/encoder.gguf": "7f8d265da594eaf5e1de2db8f5f1867dbcb0bb75ef5878fadf2952347116f4d0",
|
||||
"token2wav-gguf/flow_matching.gguf": "eda6069f3edeb5dd3a87fbf2aedb2ddd1b46f3273926c4fcf09b24476a39cab8",
|
||||
"token2wav-gguf/flow_extra.gguf": "c67611aa7d02500fe395a7798bf0bfdfb55c74d37ba93934ca74d82b4e63f78d",
|
||||
"token2wav-gguf/hifigan2.gguf": "1b8b3bf5d8d3066aeee4324fdcdd41aefce170d0ee907645858de408d82835c2",
|
||||
"token2wav-gguf/prompt_cache.gguf": "81fe6f541ebe0b67db06a1e395df928da47991dc9637c2cf47d6c59d5b979f2c"
|
||||
},
|
||||
"cli_sha256": "66a512386aafeb708f49a7394a9054d94ec777c362395c8917a3d198aef1b523",
|
||||
"requested_placement": "radeon-hybrid",
|
||||
"ggml_vk_visible_devices": "1",
|
||||
"shared_ram_reservation": {
|
||||
"host_ram": 22548578304
|
||||
},
|
||||
"weights_bytes": 8872297088,
|
||||
"memory_overhead_bytes": 12884901888,
|
||||
"context_tokens": 2048,
|
||||
"max_new_tokens": 96,
|
||||
"request_capacity": 1,
|
||||
"stateful_session": "C++-owned within one request; discarded after completion",
|
||||
"placement_evidence": {
|
||||
"observed": "radeon-hybrid",
|
||||
"worker_pid": 63224,
|
||||
"peak_sampled_rss_bytes": 15295959040,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-21-5a4d63dd3dcb4f3c92057150efda2aa9.log"
|
||||
}
|
||||
},
|
||||
"abort_check": {
|
||||
"stale_output": false,
|
||||
"ledger": {
|
||||
"capacities": {
|
||||
"host_ram": 22548578304
|
||||
},
|
||||
"reserved": {
|
||||
"host_ram": 0
|
||||
},
|
||||
"owners": [],
|
||||
"quarantined": []
|
||||
}
|
||||
},
|
||||
"status": "passed",
|
||||
"ledger_after_shutdown": {
|
||||
"capacities": {
|
||||
"host_ram": 22548578304
|
||||
},
|
||||
"reserved": {
|
||||
"host_ram": 0
|
||||
},
|
||||
"owners": [],
|
||||
"quarantined": []
|
||||
}
|
||||
}
|
||||
BIN
Binary file not shown.
+719
@@ -0,0 +1,719 @@
|
||||
INFO 09-24 02:27:30 [patch.py:259] NVFP4 W4A4 weight_scale NaN-clamp: installed.
|
||||
INFO 09-24 02:27:30 [patch.py:510] inductor factorable-divisibility patch: installed.
|
||||
INFO 09-24 02:27:33 [stage_init_utils.py:768] [stage_init] Set VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
WARNING 09-24 02:27:33 [stage_init_utils.py:1887] [stage_init] Failed to load transfer config: [Errno 13] Permission denied: 'C:\\Users\\zhout'
|
||||
{
|
||||
"scope": "MiniCPM-o GGUF complete audio+image\u2192text+speech through Omni StageRuntime/StagePool",
|
||||
"placement": "radeon-hybrid",
|
||||
"model_revision": "db25077c33951fe163b42986fba0132e279872a2",
|
||||
"cli_sha256": "66a512386aafeb708f49a7394a9054d94ec777c362395c8917a3d198aef1b523",
|
||||
"reference_sha256": "cf4bb5cd9acbfbf67596d4a53310652b61ef4b7ede1d54eb03ef4882ca40223a",
|
||||
"input_sha256": {
|
||||
"audio_wav": "14ee147cbf2ce9afd9e32c8c8228d7cfe11985cd7e63456b2ad33871492aefcc",
|
||||
"image_jpeg": "22c7ccea466a3f551c512045e6316c7597408d4e0a20205502f55784d977aa8d"
|
||||
},
|
||||
"memory_budget": {
|
||||
"capacities": {
|
||||
"host_ram": 22548578304
|
||||
},
|
||||
"demands": {
|
||||
"host_ram": 22548578304
|
||||
}
|
||||
},
|
||||
"profile_count": 20,
|
||||
"warmup_count": 1,
|
||||
"host_available_bytes_before": 23804850176,
|
||||
"startup_s": 51.91764660004992,
|
||||
"execution_plan_at_start": {
|
||||
"backend": "external.minicpmo.gguf.v1",
|
||||
"stage_id": 0,
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"model_revision": "db25077c33951fe163b42986fba0132e279872a2",
|
||||
"artifact_sha256": {
|
||||
"MiniCPM-o-4_5-Q4_K_M.gguf": "1237a97ee081b8abebc47aa7dad565701e8f5f904cdc92f6723ac4281bbc0932",
|
||||
"audio/MiniCPM-o-4_5-audio-F16.gguf": "d5b188ac7feaf98e17175c3f9bd14bf269301bfd187439fdaa3e3a494fc32ef7",
|
||||
"vision/MiniCPM-o-4_5-vision-F16.gguf": "1453678cc4e4fe18de241952962e234f265cb8dda780773526103ab8ba82f421",
|
||||
"tts/MiniCPM-o-4_5-tts-F16.gguf": "c7be3748a863dd6966ae7eed42600b7f41ca67affb03729ff245247f0e5ea088",
|
||||
"tts/MiniCPM-o-4_5-projector-F16.gguf": "4b1b5b377358a5e594a304ff6ea5d52df606a9ba7d886c4299d232f0c67dd1fd",
|
||||
"token2wav-gguf/encoder.gguf": "7f8d265da594eaf5e1de2db8f5f1867dbcb0bb75ef5878fadf2952347116f4d0",
|
||||
"token2wav-gguf/flow_matching.gguf": "eda6069f3edeb5dd3a87fbf2aedb2ddd1b46f3273926c4fcf09b24476a39cab8",
|
||||
"token2wav-gguf/flow_extra.gguf": "c67611aa7d02500fe395a7798bf0bfdfb55c74d37ba93934ca74d82b4e63f78d",
|
||||
"token2wav-gguf/hifigan2.gguf": "1b8b3bf5d8d3066aeee4324fdcdd41aefce170d0ee907645858de408d82835c2",
|
||||
"token2wav-gguf/prompt_cache.gguf": "81fe6f541ebe0b67db06a1e395df928da47991dc9637c2cf47d6c59d5b979f2c"
|
||||
},
|
||||
"cli_sha256": "66a512386aafeb708f49a7394a9054d94ec777c362395c8917a3d198aef1b523",
|
||||
"requested_placement": "radeon-hybrid",
|
||||
"ggml_vk_visible_devices": "1",
|
||||
"shared_ram_reservation": {
|
||||
"host_ram": 22548578304
|
||||
},
|
||||
"weights_bytes": 8872297088,
|
||||
"memory_overhead_bytes": 12884901888,
|
||||
"context_tokens": 2048,
|
||||
"max_new_tokens": 96,
|
||||
"request_capacity": 1,
|
||||
"stateful_session": "C++-owned within one request; discarded after completion",
|
||||
"placement_evidence": "pending first request"
|
||||
},
|
||||
"warmups": [
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 88.43239450000692,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 56640,
|
||||
"duration_s": 2.36,
|
||||
"pcm_sha256": "ceabf4ec4fd61ebac2b4e316c06fbf961aafa371a278ae21a76e6b5b2042b5a3"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-warmup-0",
|
||||
"stage_id": 0,
|
||||
"epoch": 1,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 88.40392130007967,
|
||||
"peak_sampled_rss_bytes": 13763682304,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-1-27686b55445e443f955c0a221e4e72ed.log"
|
||||
}
|
||||
}
|
||||
],
|
||||
"measured": [
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 89.67069400008768,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 59520,
|
||||
"duration_s": 2.48,
|
||||
"pcm_sha256": "28d1c312e3da6e034d1063ccc4f3c927308f42d6c02de197e620afabcc4a53a3"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-0",
|
||||
"stage_id": 0,
|
||||
"epoch": 2,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 89.66583519999404,
|
||||
"peak_sampled_rss_bytes": 15297159168,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-2-765d0081ff11481199ba99c70808d3f7.log"
|
||||
}
|
||||
},
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 89.2410724000074,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 66240,
|
||||
"duration_s": 2.76,
|
||||
"pcm_sha256": "b3a801e56e5c280aec423c5f29abb96a9096d917f9d457021861dfc2dab2091a"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-1",
|
||||
"stage_id": 0,
|
||||
"epoch": 3,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 89.22992990002967,
|
||||
"peak_sampled_rss_bytes": 15297896448,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-3-5a5e09d95930426493ceb0d3c5d0f667.log"
|
||||
}
|
||||
},
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 84.44369159999769,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 49920,
|
||||
"duration_s": 2.08,
|
||||
"pcm_sha256": "1b17ec4945b7db9d69084f545bea331d217c104d380030c4d7dd588195c033e2"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-2",
|
||||
"stage_id": 0,
|
||||
"epoch": 4,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 84.43399200006388,
|
||||
"peak_sampled_rss_bytes": 15301582848,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-4-9ea9b162dbbc4e99b2764d814900c4f5.log"
|
||||
}
|
||||
},
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 91.44574589992408,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 59520,
|
||||
"duration_s": 2.48,
|
||||
"pcm_sha256": "c153a4ee757314dbfa93692807d760737b0a8fdf75bce4dfe1400d090723bced"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-3",
|
||||
"stage_id": 0,
|
||||
"epoch": 5,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 91.43500569998287,
|
||||
"peak_sampled_rss_bytes": 15296962560,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-5-3f3a853496f84850a3b8d3e59ee80328.log"
|
||||
}
|
||||
},
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 97.89790800004266,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 67200,
|
||||
"duration_s": 2.8,
|
||||
"pcm_sha256": "e606c42e79f1682ebb38ffd5999be2d9d71136ed3bbbb206b54b13959b626b49"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-4",
|
||||
"stage_id": 0,
|
||||
"epoch": 6,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 97.88728310004808,
|
||||
"peak_sampled_rss_bytes": 15296348160,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-6-765c426a238146cca67a2cf5a31dcec6.log"
|
||||
}
|
||||
},
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 86.5171454999363,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 57600,
|
||||
"duration_s": 2.4,
|
||||
"pcm_sha256": "aa40a934ebe0fa335e64a17760cdb69db78f020237999ad3aa2a8e57738ef8b7"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-5",
|
||||
"stage_id": 0,
|
||||
"epoch": 7,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 86.50678499997593,
|
||||
"peak_sampled_rss_bytes": 15297114112,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-7-509ef5e86bce44f3962216500758e762.log"
|
||||
}
|
||||
},
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 87.32478409993928,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 49920,
|
||||
"duration_s": 2.08,
|
||||
"pcm_sha256": "4275292fc770992bcddfcc773e5ada7ffc14f959900adae01c48d8368add53ab"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-6",
|
||||
"stage_id": 0,
|
||||
"epoch": 8,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 87.31482480000705,
|
||||
"peak_sampled_rss_bytes": 15300722688,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-8-7919065c767c42ddb58856319722e6d0.log"
|
||||
}
|
||||
},
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 69.91491160006262,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 72000,
|
||||
"duration_s": 3.0,
|
||||
"pcm_sha256": "f9a88dc9cc294ac2ded1d81e076ef571045af818ba540554ff9ba422df22943e"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-7",
|
||||
"stage_id": 0,
|
||||
"epoch": 9,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 69.90729420003481,
|
||||
"peak_sampled_rss_bytes": 15296557056,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-9-b84f40b2f7e54848ad7d8137a4b602c9.log"
|
||||
}
|
||||
},
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 87.5758816000307,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 61440,
|
||||
"duration_s": 2.56,
|
||||
"pcm_sha256": "527ce564235f3781e74a41710c0e5a4fae27989d19ff233ee64efad5d533e9b1"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-8",
|
||||
"stage_id": 0,
|
||||
"epoch": 10,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 87.56872690003365,
|
||||
"peak_sampled_rss_bytes": 15296552960,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-10-45806b84a1394f5f8fc45d8638ef6504.log"
|
||||
}
|
||||
},
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 87.2738595000701,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 57600,
|
||||
"duration_s": 2.4,
|
||||
"pcm_sha256": "a1110fdc2f1945112bccf1566f51bdb65a2ddc36e772584f747a3b8be95b2b7e"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-9",
|
||||
"stage_id": 0,
|
||||
"epoch": 11,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 87.26336149999406,
|
||||
"peak_sampled_rss_bytes": 15296577536,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-11-5bf017cff9ff4d59b3f59c2fc3bf2266.log"
|
||||
}
|
||||
},
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 80.4882060000673,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 64320,
|
||||
"duration_s": 2.68,
|
||||
"pcm_sha256": "0d070361d5a2d08a0d5234144f9e69256e169ead3a543b4671ee9753a34e244a"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-10",
|
||||
"stage_id": 0,
|
||||
"epoch": 12,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 80.47764310007915,
|
||||
"peak_sampled_rss_bytes": 15296012288,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-12-61bda20e0fa141738dde2bd58c1311e3.log"
|
||||
}
|
||||
},
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 85.45749559998512,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 59520,
|
||||
"duration_s": 2.48,
|
||||
"pcm_sha256": "1ff503a03b0411eb781e9425edaf256b303ca129abef1d6714cad9fc57589954"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-11",
|
||||
"stage_id": 0,
|
||||
"epoch": 13,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 85.44423809996806,
|
||||
"peak_sampled_rss_bytes": 15295561728,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-13-b9ce8af2384f4702bc457ff245859deb.log"
|
||||
}
|
||||
},
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 89.06362529995386,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 55680,
|
||||
"duration_s": 2.32,
|
||||
"pcm_sha256": "75f1825c97b1229a787277a1695eaa7749bed87fb459394ff259261052320179"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-12",
|
||||
"stage_id": 0,
|
||||
"epoch": 14,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 89.05177350004669,
|
||||
"peak_sampled_rss_bytes": 15295856640,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-14-6a2560adbf8c404ba4ecb9b5bafe0f33.log"
|
||||
}
|
||||
},
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 86.04584340006113,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 61440,
|
||||
"duration_s": 2.56,
|
||||
"pcm_sha256": "e20ed2cc01012a362be35a1cae41d4fd41eed2f268ea9be12d6990dcec780502"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-13",
|
||||
"stage_id": 0,
|
||||
"epoch": 15,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 86.03722810000181,
|
||||
"peak_sampled_rss_bytes": 15295610880,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-15-84c5ffad02684029bafbca2aba2106af.log"
|
||||
}
|
||||
},
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 86.03778659994714,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 61440,
|
||||
"duration_s": 2.56,
|
||||
"pcm_sha256": "c8b5d1af4328d0ee202d592b4c8b48603f7f5505fffba8d8e513407d866ec90a"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-14",
|
||||
"stage_id": 0,
|
||||
"epoch": 16,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 86.03154140000697,
|
||||
"peak_sampled_rss_bytes": 15296053248,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-16-f66088565b534d1ab2749109959f8d63.log"
|
||||
}
|
||||
},
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 89.40287510002963,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 53760,
|
||||
"duration_s": 2.24,
|
||||
"pcm_sha256": "053e3a14170269a10218e678ade4223e28b2653a28ea75c15621c02d03a3b041"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-15",
|
||||
"stage_id": 0,
|
||||
"epoch": 17,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 89.39397819992155,
|
||||
"peak_sampled_rss_bytes": 15296118784,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-17-1dbbf8d44c554b0ab0e9d63d3d3c6cfb.log"
|
||||
}
|
||||
},
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 87.37631849991158,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 60480,
|
||||
"duration_s": 2.52,
|
||||
"pcm_sha256": "7b14deb1f5215fc003ec006cf68623644862aa33457d2ad3dcb8663b05472f12"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-16",
|
||||
"stage_id": 0,
|
||||
"epoch": 18,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 87.36507729999721,
|
||||
"peak_sampled_rss_bytes": 15296081920,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-18-0d33268ae2ec4cca9d14bbaa000aed11.log"
|
||||
}
|
||||
},
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 84.44159840000793,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 57600,
|
||||
"duration_s": 2.4,
|
||||
"pcm_sha256": "bd002f117e6873122151b2c4dc8267f5c2de2e98a464c890a076f11c27dcf972"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-17",
|
||||
"stage_id": 0,
|
||||
"epoch": 19,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 84.43294430000242,
|
||||
"peak_sampled_rss_bytes": 15296143360,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-19-a1fe46c104ba46409bbcd1553640fab1.log"
|
||||
}
|
||||
},
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 86.14088329998776,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 51840,
|
||||
"duration_s": 2.16,
|
||||
"pcm_sha256": "7c1a2617b1b315a67bd35854acaed5e1f0fad877858682f2f257e8c7bb8b9cfd"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-18",
|
||||
"stage_id": 0,
|
||||
"epoch": 20,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 86.13427959999535,
|
||||
"peak_sampled_rss_bytes": 15296356352,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-20-60049a03aa98440796d4f3d095fab1da.log"
|
||||
}
|
||||
},
|
||||
{
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 90.84336910001002,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 65280,
|
||||
"duration_s": 2.72,
|
||||
"pcm_sha256": "05552b37ab9df6d0e6c15fa854219b9d3bf53f7cc90fa62e1f66af145f4f3fa3"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-19",
|
||||
"stage_id": 0,
|
||||
"epoch": 21,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 90.83368909999263,
|
||||
"peak_sampled_rss_bytes": 15295959040,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-21-5a4d63dd3dcb4f3c92057150efda2aa9.log"
|
||||
}
|
||||
}
|
||||
],
|
||||
"complete_request": {
|
||||
"text": "The square in the image is red.",
|
||||
"wall_s": 89.67069400008768,
|
||||
"audio_metadata": {
|
||||
"sample_rate_hz": 24000,
|
||||
"channels": 1,
|
||||
"pcm_frames": 59520,
|
||||
"duration_s": 2.48,
|
||||
"pcm_sha256": "28d1c312e3da6e034d1063ccc4f3c927308f42d6c02de197e620afabcc4a53a3"
|
||||
},
|
||||
"stage_event": {
|
||||
"request_id": "minicpmo-measured-0",
|
||||
"stage_id": 0,
|
||||
"epoch": 2,
|
||||
"seq": 1,
|
||||
"kind": "audio",
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"minicpmo_whole_request_wall_s": 89.66583519999404,
|
||||
"peak_sampled_rss_bytes": 15297159168,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-2-765d0081ff11481199ba99c70808d3f7.log"
|
||||
}
|
||||
},
|
||||
"nearest_rank_p50_wall_s": 87.2738595000701,
|
||||
"nearest_rank_p95_wall_s": 91.44574589992408,
|
||||
"output_wav": "\\\\wsl.localhost\\Ubuntu\\home\\zhout\\project\\edge_infer\\vllm-omni-edge\\benchmarks\\edge_harness\\results\\e2e_expansion_20260923\\evidence\\minicpmo_omni_radeon_profile20\\profile20.wav",
|
||||
"execution_plan_after_request": {
|
||||
"backend": "external.minicpmo.gguf.v1",
|
||||
"stage_id": 0,
|
||||
"worker_generation": "1cf3768b5a0d4d09bfa21d21fc93c56d",
|
||||
"model_revision": "db25077c33951fe163b42986fba0132e279872a2",
|
||||
"artifact_sha256": {
|
||||
"MiniCPM-o-4_5-Q4_K_M.gguf": "1237a97ee081b8abebc47aa7dad565701e8f5f904cdc92f6723ac4281bbc0932",
|
||||
"audio/MiniCPM-o-4_5-audio-F16.gguf": "d5b188ac7feaf98e17175c3f9bd14bf269301bfd187439fdaa3e3a494fc32ef7",
|
||||
"vision/MiniCPM-o-4_5-vision-F16.gguf": "1453678cc4e4fe18de241952962e234f265cb8dda780773526103ab8ba82f421",
|
||||
"tts/MiniCPM-o-4_5-tts-F16.gguf": "c7be3748a863dd6966ae7eed42600b7f41ca67affb03729ff245247f0e5ea088",
|
||||
"tts/MiniCPM-o-4_5-projector-F16.gguf": "4b1b5b377358a5e594a304ff6ea5d52df606a9ba7d886c4299d232f0c67dd1fd",
|
||||
"token2wav-gguf/encoder.gguf": "7f8d265da594eaf5e1de2db8f5f1867dbcb0bb75ef5878fadf2952347116f4d0",
|
||||
"token2wav-gguf/flow_matching.gguf": "eda6069f3edeb5dd3a87fbf2aedb2ddd1b46f3273926c4fcf09b24476a39cab8",
|
||||
"token2wav-gguf/flow_extra.gguf": "c67611aa7d02500fe395a7798bf0bfdfb55c74d37ba93934ca74d82b4e63f78d",
|
||||
"token2wav-gguf/hifigan2.gguf": "1b8b3bf5d8d3066aeee4324fdcdd41aefce170d0ee907645858de408d82835c2",
|
||||
"token2wav-gguf/prompt_cache.gguf": "81fe6f541ebe0b67db06a1e395df928da47991dc9637c2cf47d6c59d5b979f2c"
|
||||
},
|
||||
"cli_sha256": "66a512386aafeb708f49a7394a9054d94ec777c362395c8917a3d198aef1b523",
|
||||
"requested_placement": "radeon-hybrid",
|
||||
"ggml_vk_visible_devices": "1",
|
||||
"shared_ram_reservation": {
|
||||
"host_ram": 22548578304
|
||||
},
|
||||
"weights_bytes": 8872297088,
|
||||
"memory_overhead_bytes": 12884901888,
|
||||
"context_tokens": 2048,
|
||||
"max_new_tokens": 96,
|
||||
"request_capacity": 1,
|
||||
"stateful_session": "C++-owned within one request; discarded after completion",
|
||||
"placement_evidence": {
|
||||
"observed": "radeon-hybrid",
|
||||
"worker_pid": 63224,
|
||||
"peak_sampled_rss_bytes": 15295959040,
|
||||
"worker_log": "C:\\Users\\zhout\\w2\\minicpmo_omni_profile20_logs_radeon\\minicpmo-21-5a4d63dd3dcb4f3c92057150efda2aa9.log"
|
||||
}
|
||||
},
|
||||
"abort_check": {
|
||||
"stale_output": false,
|
||||
"ledger": {
|
||||
"capacities": {
|
||||
"host_ram": 22548578304
|
||||
},
|
||||
"reserved": {
|
||||
"host_ram": 0
|
||||
},
|
||||
"owners": [],
|
||||
"quarantined": []
|
||||
}
|
||||
},
|
||||
"status": "passed",
|
||||
"ledger_after_shutdown": {
|
||||
"capacities": {
|
||||
"host_ram": 22548578304
|
||||
},
|
||||
"reserved": {
|
||||
"host_ram": 0
|
||||
},
|
||||
"owners": [],
|
||||
"quarantined": []
|
||||
}
|
||||
}
|
||||
+16
@@ -0,0 +1,16 @@
|
||||
{
|
||||
"scope": "local Whisper tiny.en ASR proxy for generated speech; no perceptual quality claim",
|
||||
"asr_model": "openai/whisper-tiny.en",
|
||||
"asr_revision": "87c7102498dcde7456f24cfd30239ca606ed9063",
|
||||
"asr_weights_sha256": "db59695928ded6043adaef491a53ef4e12da9611184d77c53baa691a60b958ad",
|
||||
"transformers": "5.14.1",
|
||||
"torch": "2.13.0+cpu",
|
||||
"wav_sha256": "cf43ae8608fdd01d11b05705b245bd2ec1e94e3381ae26ac6d2d24a18db452c5",
|
||||
"input_sample_rate": 24000,
|
||||
"input_duration_s": 2.48,
|
||||
"reference_text": "The square in the image is red.",
|
||||
"transcript": "The square in the image is red.",
|
||||
"word_error_rate": 0.0,
|
||||
"asr_load_s": 0.8116924740024842,
|
||||
"asr_inference_s": 0.27089974202681333
|
||||
}
|
||||
+23
@@ -0,0 +1,23 @@
|
||||
W0924 03:04:06.174000 3870014 torch/utils/_pytree.py:630] <enum 'KernelPreference'> is an Enum subclass and is now natively supported by torch.compile as an opaque value type. Calling register_constant() on Enum subclasses is deprecated and will be an error in a future release.
|
||||
W0924 03:04:06.213000 3870014 torch/utils/_pytree.py:630] <enum 'ScaleCalculationMode'> is an Enum subclass and is now natively supported by torch.compile as an opaque value type. Calling register_constant() on Enum subclasses is deprecated and will be an error in a future release.
|
||||
␍Loading weights: 0%| | 0/167 [00:00<?, ?it/s]␍Loading weights: 62%|██████▏ | 104/167 [00:00<00:00, 1037.70it/s]␍Loading weights: 100%|██████████| 167/167 [00:00<00:00, 1156.86it/s]
|
||||
[transformers] Using custom `forced_decoder_ids` from the (generation) config. This is deprecated in favor of the `task` and `language` flags/config options.
|
||||
[transformers] A custom logits processor of type <class 'transformers.generation.logits_process.SuppressTokensLogitsProcessor'> has been passed to `.generate()`, but it was also created in `.generate()`, given its parameterization. The custom <class 'transformers.generation.logits_process.SuppressTokensLogitsProcessor'> will take precedence. Please check the docstring of <class 'transformers.generation.logits_process.SuppressTokensLogitsProcessor'> to see related `.generate()` flags.
|
||||
[transformers] A custom logits processor of type <class 'transformers.generation.logits_process.SuppressTokensAtBeginLogitsProcessor'> has been passed to `.generate()`, but it was also created in `.generate()`, given its parameterization. The custom <class 'transformers.generation.logits_process.SuppressTokensAtBeginLogitsProcessor'> will take precedence. Please check the docstring of <class 'transformers.generation.logits_process.SuppressTokensAtBeginLogitsProcessor'> to see related `.generate()` flags.
|
||||
[transformers] Ignoring clean_up_tokenization_spaces=True for BPE tokenizer WhisperTokenizer. The clean_up_tokenization post-processing step is designed for WordPiece tokenizers and is destructive for BPE (it strips spaces before punctuation). Set clean_up_tokenization_spaces=False to suppress this warning, or set clean_up_tokenization_spaces_for_bpe_even_though_it_will_corrupt_output=True to force cleanup anyway.
|
||||
{
|
||||
"scope": "local Whisper tiny.en ASR proxy for generated speech; no perceptual quality claim",
|
||||
"asr_model": "openai/whisper-tiny.en",
|
||||
"asr_revision": "87c7102498dcde7456f24cfd30239ca606ed9063",
|
||||
"asr_weights_sha256": "db59695928ded6043adaef491a53ef4e12da9611184d77c53baa691a60b958ad",
|
||||
"transformers": "5.14.1",
|
||||
"torch": "2.13.0+cpu",
|
||||
"wav_sha256": "cf43ae8608fdd01d11b05705b245bd2ec1e94e3381ae26ac6d2d24a18db452c5",
|
||||
"input_sample_rate": 24000,
|
||||
"input_duration_s": 2.48,
|
||||
"reference_text": "The square in the image is red.",
|
||||
"transcript": "The square in the image is red.",
|
||||
"word_error_rate": 0.0,
|
||||
"asr_load_s": 0.8116924740024842,
|
||||
"asr_inference_s": 0.27089974202681333
|
||||
}
|
||||
+483
@@ -0,0 +1,483 @@
|
||||
build: 64 (5202b7b) with x86_64-w64-mingw32-gcc-posix (GCC) 13-posix for x86_64-w64-mingw32
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 30720 MiB free
|
||||
llama_model_loader: loaded meta data with 27 key-value pairs and 399 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = qwen3
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Llm
|
||||
llama_model_loader: - kv 3: general.size_label str = 8.2B
|
||||
llama_model_loader: - kv 4: qwen3.block_count u32 = 36
|
||||
llama_model_loader: - kv 5: qwen3.context_length u32 = 40960
|
||||
llama_model_loader: - kv 6: qwen3.embedding_length u32 = 4096
|
||||
llama_model_loader: - kv 7: qwen3.feed_forward_length u32 = 12288
|
||||
llama_model_loader: - kv 8: qwen3.attention.head_count u32 = 32
|
||||
llama_model_loader: - kv 9: qwen3.attention.head_count_kv u32 = 8
|
||||
llama_model_loader: - kv 10: qwen3.rope.freq_base f32 = 1000000.000000
|
||||
llama_model_loader: - kv 11: qwen3.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: qwen3.attention.key_length u32 = 128
|
||||
llama_model_loader: - kv 13: qwen3.attention.value_length u32 = 128
|
||||
llama_model_loader: - kv 14: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 15: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 16: tokenizer.ggml.tokens arr[str,151748] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 17: tokenizer.ggml.token_type arr[i32,151748] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 24: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 25: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 26: general.file_type u32 = 15
|
||||
llama_model_loader: - type f32: 145 tensors
|
||||
llama_model_loader: - type q4_K: 217 tensors
|
||||
llama_model_loader: - type q6_K: 37 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = Q4_K - Medium
|
||||
print_info: file size = 4.68 GiB (4.90 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = qwen3
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 40960
|
||||
print_info: n_embd = 4096
|
||||
print_info: n_layer = 36
|
||||
print_info: n_head = 32
|
||||
print_info: n_head_kv = 8
|
||||
print_info: n_rot = 128
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 128
|
||||
print_info: n_embd_head_v = 128
|
||||
print_info: n_gqa = 4
|
||||
print_info: n_embd_k_gqa = 1024
|
||||
print_info: n_embd_v_gqa = 1024
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 12288
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = -1
|
||||
print_info: rope type = 2
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 1000000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 40960
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = 8B
|
||||
print_info: model params = 8.19 B
|
||||
print_info: general.name = Llm
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 151748
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 36 repeating layers to GPU
|
||||
load_tensors: offloading output layer to GPU
|
||||
load_tensors: offloaded 37/37 layers to GPU
|
||||
load_tensors: Vulkan0 model buffer size = 4454.74 MiB
|
||||
load_tensors: CPU_Mapped model buffer size = 333.43 MiB
|
||||
.....................................................................................
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 1000000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (40960) -- the full capacity of the model will not be utilized
|
||||
llama_context: Vulkan_Host output buffer size = 0.58 MiB
|
||||
llama_kv_cache: Vulkan0 KV buffer size = 288.00 MiB
|
||||
llama_kv_cache: size = 288.00 MiB ( 2048 cells, 36 layers, 1/1 seqs), K (f16): 144.00 MiB, V (f16): 144.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 304.38 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 12.01 MiB
|
||||
llama_context: graph nodes = 1267
|
||||
llama_context: graph splits = 2
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 25673 MiB free
|
||||
llama_model_loader: loaded meta data with 30 key-value pairs and 193 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = llama
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Tts
|
||||
llama_model_loader: - kv 3: general.size_label str = 445M
|
||||
llama_model_loader: - kv 4: llama.block_count u32 = 20
|
||||
llama_model_loader: - kv 5: llama.context_length u32 = 4096
|
||||
llama_model_loader: - kv 6: llama.embedding_length u32 = 768
|
||||
llama_model_loader: - kv 7: llama.feed_forward_length u32 = 3072
|
||||
llama_model_loader: - kv 8: llama.attention.head_count u32 = 12
|
||||
llama_model_loader: - kv 9: llama.attention.head_count_kv u32 = 12
|
||||
llama_model_loader: - kv 10: llama.rope.freq_base f32 = 10000.000000
|
||||
llama_model_loader: - kv 11: llama.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: llama.attention.key_length u32 = 64
|
||||
llama_model_loader: - kv 13: llama.attention.value_length u32 = 64
|
||||
llama_model_loader: - kv 14: general.file_type u32 = 1
|
||||
llama_model_loader: - kv 15: llama.vocab_size u32 = 158626
|
||||
llama_model_loader: - kv 16: llama.rope.dimension_count u32 = 64
|
||||
llama_model_loader: - kv 17: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.tokens arr[str,158626] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.token_type arr[i32,158626] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 24: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 25: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 26: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 27: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 28: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 29: tokenizer.ggml.add_space_prefix bool = false
|
||||
llama_model_loader: - type f32: 50 tensors
|
||||
llama_model_loader: - type f16: 143 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = F16
|
||||
print_info: file size = 1.07 GiB (20.70 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = llama
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 4096
|
||||
print_info: n_embd = 768
|
||||
print_info: n_layer = 20
|
||||
print_info: n_head = 12
|
||||
print_info: n_head_kv = 12
|
||||
print_info: n_rot = 64
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 64
|
||||
print_info: n_embd_head_v = 64
|
||||
print_info: n_gqa = 1
|
||||
print_info: n_embd_k_gqa = 768
|
||||
print_info: n_embd_v_gqa = 768
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 3072
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = 0
|
||||
print_info: rope type = 0
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 10000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 4096
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = ?B
|
||||
print_info: model params = 444.94 M
|
||||
print_info: general.name = Tts
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 158626
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
load_tensors: skipping TTS tensor emb_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor emb_text.weight (size = 467140608 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.weight (size = 12582912 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.weight (size = 2359296 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.weight (size = 6291456 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.weight (size = 1179648 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor head_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 0 repeating layers to GPU
|
||||
load_tensors: offloaded 0/21 layers to GPU
|
||||
load_tensors: CPU_Mapped model buffer size = 592.48 MiB
|
||||
..............................................................
|
||||
llama_init_from_model: model default pooling_type is [0], but [-1] was specified
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 10000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (4096) -- the full capacity of the model will not be utilized
|
||||
llama_context: CPU output buffer size = 0.61 MiB
|
||||
llama_kv_cache: CPU KV buffer size = 120.00 MiB
|
||||
llama_kv_cache: size = 120.00 MiB ( 2048 cells, 20 layers, 1/1 seqs), K (f16): 60.00 MiB, V (f16): 60.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 30.75 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 7.01 MiB
|
||||
llama_context: graph nodes = 613
|
||||
llama_context: graph splits = 216 (with bs=512), 1 (with bs=1)
|
||||
vision_model_loader: model name:
|
||||
vision_model_loader: description: image encoder for MiniCPM-V
|
||||
vision_model_loader: GGUF version: 3
|
||||
vision_model_loader: alignment: 32
|
||||
vision_model_loader: n_tensors: 455
|
||||
vision_model_loader: n_kv: 20
|
||||
|
||||
vision_ctx: vision using Vulkan0 backend
|
||||
load_hparams: model_type: 0
|
||||
load_hparams: n_embd: 1152
|
||||
load_hparams: n_head: 16
|
||||
load_hparams: n_ff: 4304
|
||||
load_hparams: n_layer: 27
|
||||
load_hparams: ffn_op: gelu
|
||||
load_hparams: projection_dim: 0
|
||||
|
||||
--- vision hparams ---
|
||||
load_hparams: image_size: 448
|
||||
load_hparams: patch_size: 14
|
||||
load_hparams: minicpmv_version: 100045
|
||||
load_hparams: proj_scale_factor: 0
|
||||
load_hparams: n_wa_pattern: 0
|
||||
|
||||
load_hparams: model size: 1044.36 MiB
|
||||
load_hparams: metadata size: 0.16 MiB
|
||||
load_tensors: ffn up/down are swapped
|
||||
alloc_compute_meta: Vulkan0 compute buffer size = 100.30 MiB
|
||||
alloc_compute_meta: CPU compute buffer size = 18.30 MiB
|
||||
init tts....init t2w....voc_hg2_model: CPU backend using 8 threads
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=32000
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=200, n_mel=80, total_size=16000
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=200, n_mel=80, total_elements=16000
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=200, n_tokens=100, current_iter=0, current_total=0, new_total=100, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 1 (total_tokens=100, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 20]
|
||||
audition_audio_batch_encode: Final output: 20 tokens x 4096 dims
|
||||
=== Model Paths ===
|
||||
Base dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf
|
||||
LLM: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf [OK]
|
||||
Vision: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/vision/MiniCPM-o-4_5-vision-F16.gguf [OK]
|
||||
Audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf [OK]
|
||||
TTS: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf [OK]
|
||||
Projector: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf [OK]
|
||||
===================
|
||||
=== Initializing Omni Context ===
|
||||
Media type: 2 (omni: audio+vision)
|
||||
TTS enabled: yes
|
||||
Context size: 2048
|
||||
GPU layers: 99
|
||||
Token2Wav device: cpu
|
||||
Vision backend: metal
|
||||
TTS bin dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts
|
||||
Ref audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\vllm-omni-edge\benchmarks\edge_harness\results\e2e_expansion_20260923\evidence\minicpmo_cpp_radeon890m_prompted\fixture_0000.wav
|
||||
02:28:25.627 === omni_init start
|
||||
02:28:25.627 media_type = 2, duplex_mode = 0, base_output_dir = ./tools/omni/output
|
||||
02:28:25.627 === omni_init: loading new LLM model
|
||||
02:29:14.654 === omni_init: loading TTS model
|
||||
02:29:14.654 TTS model: loading with n_gpu_layers=0
|
||||
02:29:17.447 TTS sampler: temp=0.80, top_p=0.85, top_k=25, rep_penalty=1.05
|
||||
02:29:17.447 TTS: loading weights from GGUF (emb_code, emb_text, projector_semantic, head_code)...
|
||||
02:29:23.362 TTS: head_code shape: dim0=6562, dim1=768, will transpose to [6562, 768]
|
||||
02:29:23.369 TTS: head_code loaded, verifying first few values:
|
||||
02:29:23.369 head_code_weight[0] = 0.01361084 (expect ~0.01385498)
|
||||
02:29:23.369 head_code_weight[1] = -0.01660156 (expect ~-0.01647949)
|
||||
02:29:23.369 head_code_weight[768] = 0.04101562 (this is weight[1, 0], expect ~0.04150391)
|
||||
02:29:23.483 TTS: weights loaded successfully
|
||||
02:29:23.483 Projector: loading from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf
|
||||
02:29:23.589 Projector: loaded successfully
|
||||
02:29:23.589 === omni_init: loading APM model
|
||||
02:29:26.709 APM: init from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf
|
||||
02:29:33.167 Token2Wav: found models in \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf
|
||||
02:29:33.219 Token2Wav: all model files found, initializing session...
|
||||
02:29:33.220 Token2Wav: non-CUDA backend, vocoder using CPU for better performance
|
||||
02:29:33.220 Token2Wav: using prompt_cache from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf/prompt_cache.gguf
|
||||
02:29:38.104 Token2Wav: initialized successfully
|
||||
02:29:38.104 Python T2W: script_dir=./tools/omni/pyt2w, model_dir=./tools/omni/pyt2w/token2wav
|
||||
02:29:38.104 Token2Wav: 浣跨敤 C++ 瀹炵幇
|
||||
02:29:38.104 === omni_init success: ctx_llama = 000001ea2e4ffe80
|
||||
=== Running test case ===
|
||||
Audio prefix: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-1-_ykdyex8\fixture_
|
||||
Count: 2
|
||||
02:29:38.104 stream_prefill: n_past = 0
|
||||
voice_clone_prompt = <|im_start|>system
|
||||
妯′豢闊抽�鏍锋湰鐨勯煶鑹插苟鐢熸垚鏂扮殑鍐呭�銆�
|
||||
<|audio_start|>
|
||||
assistant_prompt = <|audio_end|>浣犵殑浠诲姟鏄�敤杩欑�澹伴煶妯″紡鏉ュ綋涓€涓�姪鎵嬨€傝�璁ょ湡銆侀珮璐ㄩ噺鍦板洖澶嶇敤鎴风殑闂��銆傝�鐢ㄩ珮鑷�劧搴︾殑鏂瑰紡鍜岀敤鎴疯亰澶┿€�<|im_end|>
|
||||
<|im_start|>user
|
||||
|
||||
02:29:38.104 system prompt ref_audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-1-_ykdyex8\fixture_0000.wav
|
||||
02:29:38.528 system prompt ref_audio embedding: n_pos=20
|
||||
02:29:39.123 馃敀 n_keep 璁剧疆涓� 72 (system prompt tokens)锛岃繖閮ㄥ垎姘歌繙涓嶄細琚�粦鍔ㄧ獥鍙ｅ垹闄�
|
||||
02:29:39.124 stream_prefill(index=0): system prompt 鍒濆�鍖栧畬鎴愶紝ref_audio 宸插湪鍏朵腑 prefill
|
||||
02:29:39.124 [SW] system_prompt registered: preserve_length=72 (will be protected from sliding)
|
||||
02:29:39.124 n_past = 72
|
||||
02:29:39.124
|
||||
|
||||
c++ finish stream_prefill(index=0). n_past=72, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 0 (audio) : 1.01983 s
|
||||
encode_image_with_vision_chunks: step 1 of 1 encoded in 1113.66 ms
|
||||
encode_image_with_vision_chunks: all 1 chunks encoded in 1131.64 ms (grid: 1x1)
|
||||
|
||||
encode_image_with_vision_chunks: image encoded in 1131.65 ms by vision ( 1131.65 ms per chunk, 64 total tokens)
|
||||
omni_image_embed_make_chunks_with_filename: created 1 vision chunks from C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-1-_ykdyex8\fixture_0001.jpg
|
||||
stream_prefill: prefilled 1 vision chunks (64 tokens each)
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=49339
|
||||
audition_audio_preprocess: Audio not aligned to 100ms (49339 samples = 3083.7ms), padded with silence to 49600 samples (3100ms)
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=310, n_mel=80, total_size=24800
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=310, n_mel=80, total_elements=24800
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=310, n_tokens=155, current_iter=1, current_total=155, new_total=310, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 2 (total_tokens=310, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 31]
|
||||
audition_audio_batch_encode: Final output: 31 tokens x 4096 dims
|
||||
02:29:40.833 stream_prefill(index=1): processing user audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-1-_ykdyex8\fixture_0001.wav
|
||||
02:29:41.139 stream_prefill(index=1): user audio embedding: n_pos=31
|
||||
02:29:41.501
|
||||
|
||||
c++ finish stream_prefill(index=1). n_past=172, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 1 (audio+vision) : 2.37453 s
|
||||
02:29:41.502 馃搷 stream_decode 寮€濮�: n_past=172, n_keep=72, n_ctx=2048, duplex_mode=0
|
||||
02:29:41.502 stream_decode: create tts thread (simplex mode)
|
||||
02:29:41.502 stream_decode: create t2w thread
|
||||
02:29:41.502 wait prefill done
|
||||
02:29:41.502 馃搷 [鍗曞伐TTS] 娣诲姞 assistant prompt: "<|im_end|>
|
||||
<|im_start|>assistant
|
||||
<think>
|
||||
|
||||
</think>
|
||||
|
||||
<|tts_bos|>", n_past=172
|
||||
02:29:41.502 T2W thread (C++) started
|
||||
<user>
|
||||
<assistant>02:29:41.507 TTS: 鍒涘缓鍗曞伐妯″紡杈撳嚭鐩�綍: ./tools/omni/output/round_000
|
||||
02:29:41.508 TTS thread started
|
||||
02:29:41.692 馃搷 [鍗曞伐TTS] assistant prompt 瀹屾垚, n_past=182
|
||||
02:29:41.692 LLM decode: max_tgt_len = 96, n_predict = 96, n_ctx = 2048
|
||||
02:29:42.216 LLM: detected end token, set llm_generation_done=true
|
||||
02:29:42.216 LLM->TTS: text='The square in the image is red.', n_tokens=8, hidden_size=32768, n_embd=4096, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:29:42.216 馃搷 Decode 缁撴潫: n_past=191, 鍓╀綑绌洪棿=1857, 鏃犻渶婊戠獥
|
||||
02:29:42.216 馃搷 杞�� 1 缁撴潫锛岃�褰曡竟鐣屼簬 n_past=191
|
||||
02:29:42.216 TTS<-LLM: chunk_idx=0, text='The square in the image is red.', n_tokens=8, hidden_size=32768, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:29:42.216 TTS: after queue pop - speek_done=0, llm_finish=1, llm_text.empty=0, token_ids.size=8
|
||||
02:29:42.216 TTS: before empty check - response.empty=0, response='The square in the image is red.', llm_finish=1
|
||||
Waiting for audio generation to complete...
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_0.wav does not exist or is empty
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_1.wav does not exist or is empty
|
||||
TTS: no valid WAV files to merge
|
||||
02:29:42.216 TTS: DEBUG before has_llm_data - token_ids.size=8, hidden_states.size=32768, n_embd=4096
|
||||
02:29:42.216 TTS: has_llm_data=1, token_ids=8, hidden_states=32768, n_embd=4096, llm_finish=1
|
||||
02:29:42.217 TTS: n_tokens_orig=8, n_tokens_filtered=8 (filtered 0 special tokens)
|
||||
02:29:42.246 TTS: saved merged embeddings (text) to ./tools/omni/output/round_000/llm_debug/chunk_0/merged_embeddings.txt
|
||||
02:29:42.246 TTS Simplex: generating audio tokens for chunk 0 (n_tokens=8, tts_n_embd=768)
|
||||
02:29:42.246 TTS Simplex: 鍦� prefill 鍓嶆坊鍔� audio_bos (chunk_idx=0, new_size=9)
|
||||
02:29:42.246 TTS Simplex: first chunk - cleared KV cache
|
||||
02:29:42.246 TTS Simplex: first chunk - cleared tts_all_generated_tokens and tts_token_buffer
|
||||
02:29:42.310 馃搷 涓轰笅涓€杞�噯澶�: eval <|im_end|>\n<|im_start|>user\n, n_past=196
|
||||
02:29:44.498 TTS Simplex: prefill completed, n_past_tts=9
|
||||
02:29:44.498 TTS Simplex: sampler created
|
||||
02:29:44.498 TTS simplex: is_audio_bos=true (first audio token)
|
||||
02:29:44.526 TTS Simplex Phase1: token 1/500: rel_id=1496
|
||||
02:29:44.537 TTS Simplex Phase1: token 2/500: rel_id=4137
|
||||
02:29:44.548 TTS Simplex Phase1: token 3/500: rel_id=4218
|
||||
02:29:44.560 TTS Simplex Phase1: token 4/500: rel_id=4218
|
||||
02:29:44.570 TTS Simplex Phase1: token 5/500: rel_id=4137
|
||||
02:29:44.790 TTS Simplex Phase1: yield 25 tokens 鍒� T2W (step 25, buffer=25)
|
||||
02:29:44.790 TTS Simplex Phase1: token 25/500: rel_id=1034
|
||||
02:29:44.790 T2W绾跨▼(C++): 杞��鍒囨崲 (-1 -> 0)锛屾洿鏂拌緭鍑虹洰褰�
|
||||
02:29:44.790 T2W绾跨▼(C++): 鏂拌緭鍑虹洰褰� ./tools/omni/output/round_000/tts_wav
|
||||
02:29:44.882 TTS Simplex Phase1: EOS token at step 34
|
||||
02:29:44.882 TTS Simplex: is_final_text_chunk=true, will enter Phase 2 for text_eos_embed
|
||||
02:29:44.882 TTS Simplex Phase2: injecting text_eos_embed + audio_bos at n_past=42
|
||||
02:29:44.889 TTS Simplex Phase2: text_eos_embed injected, n_past=43
|
||||
02:29:44.897 TTS Simplex Phase2: audio_bos injected, n_past=44, starting final generation
|
||||
02:29:45.083 TTS Simplex Phase2: yield 25 tokens 鍒� T2W
|
||||
02:29:45.159 TTS Simplex Phase2: EOS at step 24 (final end)
|
||||
02:29:45.159 TTS Simplex: chunk 缁撴潫锛宼ts_token_buffer 鍓╀綑 6 tokens (淇濈暀绛変笅涓€涓� chunk)
|
||||
02:29:45.159 TTS Simplex: is_final_text_chunk=true, flush 鍓╀綑 6 tokens 浠� tts_token_buffer
|
||||
02:29:45.159 TTS Simplex: generated 56 audio tokens for chunk 0
|
||||
02:29:45.159 TTS Simplex: chunk 0 first tokens: [1496 4137 4218 4218 4137 ], last tokens: [1461 1458 4218 4218 6324 ]
|
||||
02:29:45.159 TTS Simplex: updated tts_n_past_accumulated=68, total_generated_tokens=56
|
||||
02:29:45.160 TTS: finished processing all chunks
|
||||
02:29:45.160 TTS: 鍗曞伐妯″紡杞��缁撴潫锛屼笅涓€杞� round_idx=1
|
||||
02:29:45.161 TTS: sent is_final=true to T2W queue (turn end)
|
||||
02:29:47.234 馃帀 棣栧搷鏃堕棿 (First Audio Response): 5732ms
|
||||
02:29:47.234 T2W绾跨▼: wav_0.wav | 0.84s audio | 2442.8ms inference | RTF=2.91 | t=5732ms
|
||||
02:29:49.517 T2W绾跨▼: wav_1.wav | 1.00s audio | 2272.1ms inference | RTF=2.27 | t=8015ms
|
||||
02:29:51.617 T2W绾跨▼: wav_2.wav | 0.52s audio | 2099.8ms inference | RTF=4.04 | t=10115ms
|
||||
02:29:51.626 T2W绾跨▼: 鍐欏叆缁撴潫鏍囪� ./tools/omni/output/round_000/tts_wav/generation_done.flag (last_wav=2)
|
||||
02:29:51.626 T2W绾跨▼: 杞��缁撴潫鍚庢洿鏂拌緭鍑虹洰褰曚负 ./tools/omni/output/round_001/tts_wav
|
||||
02:29:02:29:51.700 T2W(C++) 绾跨▼: 鍋滄�
|
||||
[timing] call=0 tokens=28 final=0 token2mel=2105.818ms vocoder=336.718ms total=2442.556ms audio=20160
|
||||
[timing] call=1 tokens=28 final=0 token2mel=1889.656ms vocoder=382.374ms total=2272.042ms audio=24000
|
||||
[timing] call=2 tokens=9 final=1 token2mel=1911.280ms vocoder=188.460ms total=2099.750ms audio=12480
|
||||
Audio generation completed.
|
||||
llama_perf_context_print: load time = 26762.24 ms
|
||||
llama_perf_context_print: prompt eval time = 3579.99 ms / 182 tokens ( 19.67 ms per token, 50.84 tokens per second)
|
||||
llama_perf_context_print: eval time = 520.73 ms / 9 runs ( 57.86 ms per token, 17.28 tokens per second)
|
||||
llama_perf_context_print: total time = 36769.94 ms / 191 tokens
|
||||
llama_perf_context_print: graphs reused = 10
|
||||
Token2Wav (C++): session released
|
||||
51.700 omni_stop_threads: stop signals sent
|
||||
tts thread end
|
||||
t2w thread end
|
||||
+483
@@ -0,0 +1,483 @@
|
||||
build: 64 (5202b7b) with x86_64-w64-mingw32-gcc-posix (GCC) 13-posix for x86_64-w64-mingw32
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 30720 MiB free
|
||||
llama_model_loader: loaded meta data with 27 key-value pairs and 399 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = qwen3
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Llm
|
||||
llama_model_loader: - kv 3: general.size_label str = 8.2B
|
||||
llama_model_loader: - kv 4: qwen3.block_count u32 = 36
|
||||
llama_model_loader: - kv 5: qwen3.context_length u32 = 40960
|
||||
llama_model_loader: - kv 6: qwen3.embedding_length u32 = 4096
|
||||
llama_model_loader: - kv 7: qwen3.feed_forward_length u32 = 12288
|
||||
llama_model_loader: - kv 8: qwen3.attention.head_count u32 = 32
|
||||
llama_model_loader: - kv 9: qwen3.attention.head_count_kv u32 = 8
|
||||
llama_model_loader: - kv 10: qwen3.rope.freq_base f32 = 1000000.000000
|
||||
llama_model_loader: - kv 11: qwen3.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: qwen3.attention.key_length u32 = 128
|
||||
llama_model_loader: - kv 13: qwen3.attention.value_length u32 = 128
|
||||
llama_model_loader: - kv 14: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 15: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 16: tokenizer.ggml.tokens arr[str,151748] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 17: tokenizer.ggml.token_type arr[i32,151748] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 24: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 25: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 26: general.file_type u32 = 15
|
||||
llama_model_loader: - type f32: 145 tensors
|
||||
llama_model_loader: - type q4_K: 217 tensors
|
||||
llama_model_loader: - type q6_K: 37 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = Q4_K - Medium
|
||||
print_info: file size = 4.68 GiB (4.90 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = qwen3
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 40960
|
||||
print_info: n_embd = 4096
|
||||
print_info: n_layer = 36
|
||||
print_info: n_head = 32
|
||||
print_info: n_head_kv = 8
|
||||
print_info: n_rot = 128
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 128
|
||||
print_info: n_embd_head_v = 128
|
||||
print_info: n_gqa = 4
|
||||
print_info: n_embd_k_gqa = 1024
|
||||
print_info: n_embd_v_gqa = 1024
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 12288
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = -1
|
||||
print_info: rope type = 2
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 1000000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 40960
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = 8B
|
||||
print_info: model params = 8.19 B
|
||||
print_info: general.name = Llm
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 151748
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 36 repeating layers to GPU
|
||||
load_tensors: offloading output layer to GPU
|
||||
load_tensors: offloaded 37/37 layers to GPU
|
||||
load_tensors: Vulkan0 model buffer size = 4454.74 MiB
|
||||
load_tensors: CPU_Mapped model buffer size = 333.43 MiB
|
||||
.....................................................................................
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 1000000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (40960) -- the full capacity of the model will not be utilized
|
||||
llama_context: Vulkan_Host output buffer size = 0.58 MiB
|
||||
llama_kv_cache: Vulkan0 KV buffer size = 288.00 MiB
|
||||
llama_kv_cache: size = 288.00 MiB ( 2048 cells, 36 layers, 1/1 seqs), K (f16): 144.00 MiB, V (f16): 144.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 304.38 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 12.01 MiB
|
||||
llama_context: graph nodes = 1267
|
||||
llama_context: graph splits = 2
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 25673 MiB free
|
||||
llama_model_loader: loaded meta data with 30 key-value pairs and 193 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = llama
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Tts
|
||||
llama_model_loader: - kv 3: general.size_label str = 445M
|
||||
llama_model_loader: - kv 4: llama.block_count u32 = 20
|
||||
llama_model_loader: - kv 5: llama.context_length u32 = 4096
|
||||
llama_model_loader: - kv 6: llama.embedding_length u32 = 768
|
||||
llama_model_loader: - kv 7: llama.feed_forward_length u32 = 3072
|
||||
llama_model_loader: - kv 8: llama.attention.head_count u32 = 12
|
||||
llama_model_loader: - kv 9: llama.attention.head_count_kv u32 = 12
|
||||
llama_model_loader: - kv 10: llama.rope.freq_base f32 = 10000.000000
|
||||
llama_model_loader: - kv 11: llama.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: llama.attention.key_length u32 = 64
|
||||
llama_model_loader: - kv 13: llama.attention.value_length u32 = 64
|
||||
llama_model_loader: - kv 14: general.file_type u32 = 1
|
||||
llama_model_loader: - kv 15: llama.vocab_size u32 = 158626
|
||||
llama_model_loader: - kv 16: llama.rope.dimension_count u32 = 64
|
||||
llama_model_loader: - kv 17: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.tokens arr[str,158626] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.token_type arr[i32,158626] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 24: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 25: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 26: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 27: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 28: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 29: tokenizer.ggml.add_space_prefix bool = false
|
||||
llama_model_loader: - type f32: 50 tensors
|
||||
llama_model_loader: - type f16: 143 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = F16
|
||||
print_info: file size = 1.07 GiB (20.70 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = llama
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 4096
|
||||
print_info: n_embd = 768
|
||||
print_info: n_layer = 20
|
||||
print_info: n_head = 12
|
||||
print_info: n_head_kv = 12
|
||||
print_info: n_rot = 64
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 64
|
||||
print_info: n_embd_head_v = 64
|
||||
print_info: n_gqa = 1
|
||||
print_info: n_embd_k_gqa = 768
|
||||
print_info: n_embd_v_gqa = 768
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 3072
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = 0
|
||||
print_info: rope type = 0
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 10000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 4096
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = ?B
|
||||
print_info: model params = 444.94 M
|
||||
print_info: general.name = Tts
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 158626
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
load_tensors: skipping TTS tensor emb_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor emb_text.weight (size = 467140608 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.weight (size = 12582912 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.weight (size = 2359296 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.weight (size = 6291456 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.weight (size = 1179648 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor head_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 0 repeating layers to GPU
|
||||
load_tensors: offloaded 0/21 layers to GPU
|
||||
load_tensors: CPU_Mapped model buffer size = 592.48 MiB
|
||||
..............................................................
|
||||
llama_init_from_model: model default pooling_type is [0], but [-1] was specified
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 10000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (4096) -- the full capacity of the model will not be utilized
|
||||
llama_context: CPU output buffer size = 0.61 MiB
|
||||
llama_kv_cache: CPU KV buffer size = 120.00 MiB
|
||||
llama_kv_cache: size = 120.00 MiB ( 2048 cells, 20 layers, 1/1 seqs), K (f16): 60.00 MiB, V (f16): 60.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 30.75 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 7.01 MiB
|
||||
llama_context: graph nodes = 613
|
||||
llama_context: graph splits = 216 (with bs=512), 1 (with bs=1)
|
||||
vision_model_loader: model name:
|
||||
vision_model_loader: description: image encoder for MiniCPM-V
|
||||
vision_model_loader: GGUF version: 3
|
||||
vision_model_loader: alignment: 32
|
||||
vision_model_loader: n_tensors: 455
|
||||
vision_model_loader: n_kv: 20
|
||||
|
||||
vision_ctx: vision using Vulkan0 backend
|
||||
load_hparams: model_type: 0
|
||||
load_hparams: n_embd: 1152
|
||||
load_hparams: n_head: 16
|
||||
load_hparams: n_ff: 4304
|
||||
load_hparams: n_layer: 27
|
||||
load_hparams: ffn_op: gelu
|
||||
load_hparams: projection_dim: 0
|
||||
|
||||
--- vision hparams ---
|
||||
load_hparams: image_size: 448
|
||||
load_hparams: patch_size: 14
|
||||
load_hparams: minicpmv_version: 100045
|
||||
load_hparams: proj_scale_factor: 0
|
||||
load_hparams: n_wa_pattern: 0
|
||||
|
||||
load_hparams: model size: 1044.36 MiB
|
||||
load_hparams: metadata size: 0.16 MiB
|
||||
load_tensors: ffn up/down are swapped
|
||||
alloc_compute_meta: Vulkan0 compute buffer size = 100.30 MiB
|
||||
alloc_compute_meta: CPU compute buffer size = 18.30 MiB
|
||||
init tts....init t2w....voc_hg2_model: CPU backend using 8 threads
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=32000
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=200, n_mel=80, total_size=16000
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=200, n_mel=80, total_elements=16000
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=200, n_tokens=100, current_iter=0, current_total=0, new_total=100, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 1 (total_tokens=100, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 20]
|
||||
audition_audio_batch_encode: Final output: 20 tokens x 4096 dims
|
||||
=== Model Paths ===
|
||||
Base dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf
|
||||
LLM: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf [OK]
|
||||
Vision: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/vision/MiniCPM-o-4_5-vision-F16.gguf [OK]
|
||||
Audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf [OK]
|
||||
TTS: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf [OK]
|
||||
Projector: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf [OK]
|
||||
===================
|
||||
=== Initializing Omni Context ===
|
||||
Media type: 2 (omni: audio+vision)
|
||||
TTS enabled: yes
|
||||
Context size: 2048
|
||||
GPU layers: 99
|
||||
Token2Wav device: cpu
|
||||
Vision backend: metal
|
||||
TTS bin dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts
|
||||
Ref audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\vllm-omni-edge\benchmarks\edge_harness\results\e2e_expansion_20260923\evidence\minicpmo_cpp_radeon890m_prompted\fixture_0000.wav
|
||||
02:41:30.515 === omni_init start
|
||||
02:41:30.515 media_type = 2, duplex_mode = 0, base_output_dir = ./tools/omni/output
|
||||
02:41:30.515 === omni_init: loading new LLM model
|
||||
02:42:23.148 === omni_init: loading TTS model
|
||||
02:42:23.148 TTS model: loading with n_gpu_layers=0
|
||||
02:42:26.809 TTS sampler: temp=0.80, top_p=0.85, top_k=25, rep_penalty=1.05
|
||||
02:42:26.809 TTS: loading weights from GGUF (emb_code, emb_text, projector_semantic, head_code)...
|
||||
02:42:32.344 TTS: head_code shape: dim0=6562, dim1=768, will transpose to [6562, 768]
|
||||
02:42:32.349 TTS: head_code loaded, verifying first few values:
|
||||
02:42:32.349 head_code_weight[0] = 0.01361084 (expect ~0.01385498)
|
||||
02:42:32.350 head_code_weight[1] = -0.01660156 (expect ~-0.01647949)
|
||||
02:42:32.350 head_code_weight[768] = 0.04101562 (this is weight[1, 0], expect ~0.04150391)
|
||||
02:42:32.398 TTS: weights loaded successfully
|
||||
02:42:32.398 Projector: loading from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf
|
||||
02:42:32.499 Projector: loaded successfully
|
||||
02:42:32.499 === omni_init: loading APM model
|
||||
02:42:35.557 APM: init from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf
|
||||
02:42:40.839 Token2Wav: found models in \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf
|
||||
02:42:40.892 Token2Wav: all model files found, initializing session...
|
||||
02:42:40.892 Token2Wav: non-CUDA backend, vocoder using CPU for better performance
|
||||
02:42:40.892 Token2Wav: using prompt_cache from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf/prompt_cache.gguf
|
||||
02:42:45.418 Token2Wav: initialized successfully
|
||||
02:42:45.418 Python T2W: script_dir=./tools/omni/pyt2w, model_dir=./tools/omni/pyt2w/token2wav
|
||||
02:42:45.418 Token2Wav: 浣跨敤 C++ 瀹炵幇
|
||||
02:42:45.418 === omni_init success: ctx_llama = 000002216638f830
|
||||
=== Running test case ===
|
||||
Audio prefix: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-10-u5i2eod1\fixture_
|
||||
Count: 2
|
||||
02:42:45.419 stream_prefill: n_past = 0
|
||||
voice_clone_prompt = <|im_start|>system
|
||||
妯′豢闊抽�鏍锋湰鐨勯煶鑹插苟鐢熸垚鏂扮殑鍐呭�銆�
|
||||
<|audio_start|>
|
||||
assistant_prompt = <|audio_end|>浣犵殑浠诲姟鏄�敤杩欑�澹伴煶妯″紡鏉ュ綋涓€涓�姪鎵嬨€傝�璁ょ湡銆侀珮璐ㄩ噺鍦板洖澶嶇敤鎴风殑闂��銆傝�鐢ㄩ珮鑷�劧搴︾殑鏂瑰紡鍜岀敤鎴疯亰澶┿€�<|im_end|>
|
||||
<|im_start|>user
|
||||
|
||||
02:42:45.419 system prompt ref_audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-10-u5i2eod1\fixture_0000.wav
|
||||
02:42:46.038 system prompt ref_audio embedding: n_pos=20
|
||||
02:42:46.539 馃敀 n_keep 璁剧疆涓� 72 (system prompt tokens)锛岃繖閮ㄥ垎姘歌繙涓嶄細琚�粦鍔ㄧ獥鍙ｅ垹闄�
|
||||
02:42:46.539 stream_prefill(index=0): system prompt 鍒濆�鍖栧畬鎴愶紝ref_audio 宸插湪鍏朵腑 prefill
|
||||
02:42:46.539 [SW] system_prompt registered: preserve_length=72 (will be protected from sliding)
|
||||
02:42:46.539 n_past = 72
|
||||
02:42:46.539
|
||||
|
||||
c++ finish stream_prefill(index=0). n_past=72, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 0 (audio) : 1.12075 s
|
||||
encode_image_with_vision_chunks: step 1 of 1 encoded in 612.65 ms
|
||||
encode_image_with_vision_chunks: all 1 chunks encoded in 630.46 ms (grid: 1x1)
|
||||
|
||||
encode_image_with_vision_chunks: image encoded in 630.46 ms by vision ( 630.46 ms per chunk, 64 total tokens)
|
||||
omni_image_embed_make_chunks_with_filename: created 1 vision chunks from C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-10-u5i2eod1\fixture_0001.jpg
|
||||
stream_prefill: prefilled 1 vision chunks (64 tokens each)
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=49339
|
||||
audition_audio_preprocess: Audio not aligned to 100ms (49339 samples = 3083.7ms), padded with silence to 49600 samples (3100ms)
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=310, n_mel=80, total_size=24800
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=310, n_mel=80, total_elements=24800
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=310, n_tokens=155, current_iter=1, current_total=155, new_total=310, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 2 (total_tokens=310, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 31]
|
||||
audition_audio_batch_encode: Final output: 31 tokens x 4096 dims
|
||||
02:42:47.697 stream_prefill(index=1): processing user audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-10-u5i2eod1\fixture_0001.wav
|
||||
02:42:48.036 stream_prefill(index=1): user audio embedding: n_pos=31
|
||||
02:42:48.357
|
||||
|
||||
c++ finish stream_prefill(index=1). n_past=172, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 1 (audio+vision) : 1.81548 s
|
||||
02:42:48.357 馃搷 stream_decode 寮€濮�: n_past=172, n_keep=72, n_ctx=2048, duplex_mode=0
|
||||
02:42:48.357 stream_decode: create tts thread (simplex mode)
|
||||
02:42:48.357 stream_decode: create t2w thread
|
||||
02:42:48.357 wait prefill done
|
||||
02:42:48.357 馃搷 [鍗曞伐TTS] 娣诲姞 assistant prompt: "<|im_end|>
|
||||
<|im_start|>assistant
|
||||
<think>
|
||||
|
||||
</think>
|
||||
|
||||
<|tts_bos|>", n_past=172
|
||||
02:42:48.358 T2W thread (C++) started
|
||||
<user>
|
||||
<assistant>02:42:48.362 TTS: 鍒涘缓鍗曞伐妯″紡杈撳嚭鐩�綍: ./tools/omni/output/round_000
|
||||
02:42:48.362 TTS thread started
|
||||
02:42:48.522 馃搷 [鍗曞伐TTS] assistant prompt 瀹屾垚, n_past=182
|
||||
02:42:48.522 LLM decode: max_tgt_len = 96, n_predict = 96, n_ctx = 2048
|
||||
02:42:49.065 LLM: detected end token, set llm_generation_done=true
|
||||
02:42:49.065 LLM->TTS: text='The square in the image is red.', n_tokens=8, hidden_size=32768, n_embd=4096, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:42:49.065 馃搷 Decode 缁撴潫: n_past=191, 鍓╀綑绌洪棿=1857, 鏃犻渶婊戠獥
|
||||
02:42:49.065 馃搷 杞�� 1 缁撴潫锛岃�褰曡竟鐣屼簬 n_past=191
|
||||
02:42:49.065 TTS<-LLM: chunk_idx=0, text='The square in the image is red.', n_tokens=8, hidden_size=32768, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:42:49.065 TTS: after queue pop - speek_done=0, llm_finish=1, llm_text.empty=0, token_ids.size=8
|
||||
02:42:49.065 TTS: before empty check - response.empty=0, response='The square in the image is red.', llm_finish=1
|
||||
Waiting for audio generation to complete...
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_0.wav does not exist or is empty
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_1.wav does not exist or is empty
|
||||
TTS: no valid WAV files to merge
|
||||
02:42:49.065 TTS: DEBUG before has_llm_data - token_ids.size=8, hidden_states.size=32768, n_embd=4096
|
||||
02:42:49.065 TTS: has_llm_data=1, token_ids=8, hidden_states=32768, n_embd=4096, llm_finish=1
|
||||
02:42:49.066 TTS: n_tokens_orig=8, n_tokens_filtered=8 (filtered 0 special tokens)
|
||||
02:42:49.097 TTS: saved merged embeddings (text) to ./tools/omni/output/round_000/llm_debug/chunk_0/merged_embeddings.txt
|
||||
02:42:49.097 TTS Simplex: generating audio tokens for chunk 0 (n_tokens=8, tts_n_embd=768)
|
||||
02:42:49.097 TTS Simplex: 鍦� prefill 鍓嶆坊鍔� audio_bos (chunk_idx=0, new_size=9)
|
||||
02:42:49.097 TTS Simplex: first chunk - cleared KV cache
|
||||
02:42:49.097 TTS Simplex: first chunk - cleared tts_all_generated_tokens and tts_token_buffer
|
||||
02:42:49.160 馃搷 涓轰笅涓€杞�噯澶�: eval <|im_end|>\n<|im_start|>user\n, n_past=196
|
||||
02:42:49.379 TTS Simplex: prefill completed, n_past_tts=9
|
||||
02:42:49.379 TTS Simplex: sampler created
|
||||
02:42:49.379 TTS simplex: is_audio_bos=true (first audio token)
|
||||
02:42:49.405 TTS Simplex Phase1: token 1/500: rel_id=770
|
||||
02:42:49.416 TTS Simplex Phase1: token 2/500: rel_id=4137
|
||||
02:42:49.428 TTS Simplex Phase1: token 3/500: rel_id=4137
|
||||
02:42:49.439 TTS Simplex Phase1: token 4/500: rel_id=4137
|
||||
02:42:49.450 TTS Simplex Phase1: token 5/500: rel_id=4137
|
||||
02:02:42:42:49.4900.670 TTS Simplex Phase1: yield 25 tokens 鍒� T2W (step 25, buffer=25)
|
||||
670 T2W绾跨▼(C++): 杞��鍒囨崲 (-1 -> 0)锛屾洿鏂拌緭鍑虹洰褰�
|
||||
02:42:49.670 TTS Simplex Phase1: token 25/500: rel_id=1480
|
||||
02:42:49.670 T2W绾跨▼(C++): 鏂拌緭鍑虹洰褰� ./tools/omni/output/round_000/tts_wav
|
||||
02:42:49.927 TTS Simplex Phase1: EOS token at step 49
|
||||
02:42:49.927 TTS Simplex: is_final_text_chunk=true, will enter Phase 2 for text_eos_embed
|
||||
02:42:49.927 TTS Simplex Phase2: injecting text_eos_embed + audio_bos at n_past=57
|
||||
02:42:49.935 TTS Simplex Phase2: text_eos_embed injected, n_past=58
|
||||
02:42:49.943 TTS Simplex Phase2: audio_bos injected, n_past=59, starting final generation
|
||||
02:42:49.964 TTS Simplex Phase2: yield 25 tokens 鍒� T2W
|
||||
02:42:50.097 TTS Simplex Phase2: EOS at step 14 (final end)
|
||||
02:42:50.097 TTS Simplex: chunk 缁撴潫锛宼ts_token_buffer 鍓╀綑 11 tokens (淇濈暀绛変笅涓€涓� chunk)
|
||||
02:42:50.097 TTS Simplex: is_final_text_chunk=true, flush 鍓╀綑 11 tokens 浠� tts_token_buffer
|
||||
02:42:50.097 TTS Simplex: generated 61 audio tokens for chunk 0
|
||||
02:42:50.097 TTS Simplex: chunk 0 first tokens: [770 4137 4137 4137 4137 ], last tokens: [2919 1461 4137 4218 4137 ]
|
||||
02:42:50.097 TTS Simplex: updated tts_n_past_accumulated=73, total_generated_tokens=61
|
||||
02:42:50.098 TTS: finished processing all chunks
|
||||
02:42:50.099 TTS: 鍗曞伐妯″紡杞��缁撴潫锛屼笅涓€杞� round_idx=1
|
||||
02:42:50.099 TTS: sent is_final=true to T2W queue (turn end)
|
||||
02:42:51.993 馃帀 棣栧搷鏃堕棿 (First Audio Response): 3636ms
|
||||
02:42:51.994 T2W绾跨▼: wav_0.wav | 0.84s audio | 2322.6ms inference | RTF=2.76 | t=3636ms
|
||||
02:42:54.226 T2W绾跨▼: wav_1.wav | 1.00s audio | 2231.5ms inference | RTF=2.23 | t=5868ms
|
||||
02:42:56.338 T2W绾跨▼: wav_2.wav | 0.72s audio | 2111.7ms inference | RTF=2.93 | t=7981ms
|
||||
02:42:56.338 T2W绾跨▼: 鍐欏叆缁撴潫鏍囪� ./tools/omni/output/round_000/tts_wav/generation_done.flag (last_wav=2)
|
||||
02:42:56.338 T2W绾跨▼: 杞��缁撴潫鍚庢洿鏂拌緭鍑虹洰褰曚负 ./tools/omni/output/round_001/tts_wav
|
||||
02:42:56.420 omni_stop_threads: stop signals sent
|
||||
02:42:56.420 T2W(C++) 绾跨▼: 鍋滄�
|
||||
[timing] call=0 tokens=28 final=0 token2mel=1994.472ms vocoder=328.065ms total=2322.552ms audio=20160
|
||||
[timing] call=1 tokens=28 final=0 token2mel=1848.511ms vocoder=382.957ms total=2231.481ms audio=24000
|
||||
[timing] call=2 tokens=14 final=1 token2mel=1863.372ms vocoder=248.320ms total=2111.702ms audio=17280
|
||||
Audio generation completed.
|
||||
llama_perf_context_print: load time = 25247.74 ms
|
||||
llama_perf_context_print: prompt eval time = 3102.69 ms / 182 tokens ( 17.05 ms per token, 58.66 tokens per second)
|
||||
llama_perf_context_print: eval time = 539.95 ms / 9 runs ( 59.99 ms per token, 16.67 tokens per second)
|
||||
llama_perf_context_print: total time = 33145.17 ms / 191 tokens
|
||||
llama_perf_context_print: graphs reused = 10
|
||||
Token2Wav (C++): session released
|
||||
tts thread end
|
||||
t2w thread end
|
||||
+483
@@ -0,0 +1,483 @@
|
||||
build: 64 (5202b7b) with x86_64-w64-mingw32-gcc-posix (GCC) 13-posix for x86_64-w64-mingw32
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 30720 MiB free
|
||||
llama_model_loader: loaded meta data with 27 key-value pairs and 399 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = qwen3
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Llm
|
||||
llama_model_loader: - kv 3: general.size_label str = 8.2B
|
||||
llama_model_loader: - kv 4: qwen3.block_count u32 = 36
|
||||
llama_model_loader: - kv 5: qwen3.context_length u32 = 40960
|
||||
llama_model_loader: - kv 6: qwen3.embedding_length u32 = 4096
|
||||
llama_model_loader: - kv 7: qwen3.feed_forward_length u32 = 12288
|
||||
llama_model_loader: - kv 8: qwen3.attention.head_count u32 = 32
|
||||
llama_model_loader: - kv 9: qwen3.attention.head_count_kv u32 = 8
|
||||
llama_model_loader: - kv 10: qwen3.rope.freq_base f32 = 1000000.000000
|
||||
llama_model_loader: - kv 11: qwen3.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: qwen3.attention.key_length u32 = 128
|
||||
llama_model_loader: - kv 13: qwen3.attention.value_length u32 = 128
|
||||
llama_model_loader: - kv 14: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 15: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 16: tokenizer.ggml.tokens arr[str,151748] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 17: tokenizer.ggml.token_type arr[i32,151748] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 24: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 25: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 26: general.file_type u32 = 15
|
||||
llama_model_loader: - type f32: 145 tensors
|
||||
llama_model_loader: - type q4_K: 217 tensors
|
||||
llama_model_loader: - type q6_K: 37 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = Q4_K - Medium
|
||||
print_info: file size = 4.68 GiB (4.90 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = qwen3
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 40960
|
||||
print_info: n_embd = 4096
|
||||
print_info: n_layer = 36
|
||||
print_info: n_head = 32
|
||||
print_info: n_head_kv = 8
|
||||
print_info: n_rot = 128
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 128
|
||||
print_info: n_embd_head_v = 128
|
||||
print_info: n_gqa = 4
|
||||
print_info: n_embd_k_gqa = 1024
|
||||
print_info: n_embd_v_gqa = 1024
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 12288
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = -1
|
||||
print_info: rope type = 2
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 1000000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 40960
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = 8B
|
||||
print_info: model params = 8.19 B
|
||||
print_info: general.name = Llm
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 151748
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 36 repeating layers to GPU
|
||||
load_tensors: offloading output layer to GPU
|
||||
load_tensors: offloaded 37/37 layers to GPU
|
||||
load_tensors: Vulkan0 model buffer size = 4454.74 MiB
|
||||
load_tensors: CPU_Mapped model buffer size = 333.43 MiB
|
||||
.....................................................................................
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 1000000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (40960) -- the full capacity of the model will not be utilized
|
||||
llama_context: Vulkan_Host output buffer size = 0.58 MiB
|
||||
llama_kv_cache: Vulkan0 KV buffer size = 288.00 MiB
|
||||
llama_kv_cache: size = 288.00 MiB ( 2048 cells, 36 layers, 1/1 seqs), K (f16): 144.00 MiB, V (f16): 144.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 304.38 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 12.01 MiB
|
||||
llama_context: graph nodes = 1267
|
||||
llama_context: graph splits = 2
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 25673 MiB free
|
||||
llama_model_loader: loaded meta data with 30 key-value pairs and 193 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = llama
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Tts
|
||||
llama_model_loader: - kv 3: general.size_label str = 445M
|
||||
llama_model_loader: - kv 4: llama.block_count u32 = 20
|
||||
llama_model_loader: - kv 5: llama.context_length u32 = 4096
|
||||
llama_model_loader: - kv 6: llama.embedding_length u32 = 768
|
||||
llama_model_loader: - kv 7: llama.feed_forward_length u32 = 3072
|
||||
llama_model_loader: - kv 8: llama.attention.head_count u32 = 12
|
||||
llama_model_loader: - kv 9: llama.attention.head_count_kv u32 = 12
|
||||
llama_model_loader: - kv 10: llama.rope.freq_base f32 = 10000.000000
|
||||
llama_model_loader: - kv 11: llama.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: llama.attention.key_length u32 = 64
|
||||
llama_model_loader: - kv 13: llama.attention.value_length u32 = 64
|
||||
llama_model_loader: - kv 14: general.file_type u32 = 1
|
||||
llama_model_loader: - kv 15: llama.vocab_size u32 = 158626
|
||||
llama_model_loader: - kv 16: llama.rope.dimension_count u32 = 64
|
||||
llama_model_loader: - kv 17: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.tokens arr[str,158626] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.token_type arr[i32,158626] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 24: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 25: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 26: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 27: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 28: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 29: tokenizer.ggml.add_space_prefix bool = false
|
||||
llama_model_loader: - type f32: 50 tensors
|
||||
llama_model_loader: - type f16: 143 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = F16
|
||||
print_info: file size = 1.07 GiB (20.70 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = llama
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 4096
|
||||
print_info: n_embd = 768
|
||||
print_info: n_layer = 20
|
||||
print_info: n_head = 12
|
||||
print_info: n_head_kv = 12
|
||||
print_info: n_rot = 64
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 64
|
||||
print_info: n_embd_head_v = 64
|
||||
print_info: n_gqa = 1
|
||||
print_info: n_embd_k_gqa = 768
|
||||
print_info: n_embd_v_gqa = 768
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 3072
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = 0
|
||||
print_info: rope type = 0
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 10000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 4096
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = ?B
|
||||
print_info: model params = 444.94 M
|
||||
print_info: general.name = Tts
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 158626
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
load_tensors: skipping TTS tensor emb_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor emb_text.weight (size = 467140608 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.weight (size = 12582912 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.weight (size = 2359296 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.weight (size = 6291456 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.weight (size = 1179648 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor head_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 0 repeating layers to GPU
|
||||
load_tensors: offloaded 0/21 layers to GPU
|
||||
load_tensors: CPU_Mapped model buffer size = 592.48 MiB
|
||||
..............................................................
|
||||
llama_init_from_model: model default pooling_type is [0], but [-1] was specified
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 10000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (4096) -- the full capacity of the model will not be utilized
|
||||
llama_context: CPU output buffer size = 0.61 MiB
|
||||
llama_kv_cache: CPU KV buffer size = 120.00 MiB
|
||||
llama_kv_cache: size = 120.00 MiB ( 2048 cells, 20 layers, 1/1 seqs), K (f16): 60.00 MiB, V (f16): 60.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 30.75 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 7.01 MiB
|
||||
llama_context: graph nodes = 613
|
||||
llama_context: graph splits = 216 (with bs=512), 1 (with bs=1)
|
||||
vision_model_loader: model name:
|
||||
vision_model_loader: description: image encoder for MiniCPM-V
|
||||
vision_model_loader: GGUF version: 3
|
||||
vision_model_loader: alignment: 32
|
||||
vision_model_loader: n_tensors: 455
|
||||
vision_model_loader: n_kv: 20
|
||||
|
||||
vision_ctx: vision using Vulkan0 backend
|
||||
load_hparams: model_type: 0
|
||||
load_hparams: n_embd: 1152
|
||||
load_hparams: n_head: 16
|
||||
load_hparams: n_ff: 4304
|
||||
load_hparams: n_layer: 27
|
||||
load_hparams: ffn_op: gelu
|
||||
load_hparams: projection_dim: 0
|
||||
|
||||
--- vision hparams ---
|
||||
load_hparams: image_size: 448
|
||||
load_hparams: patch_size: 14
|
||||
load_hparams: minicpmv_version: 100045
|
||||
load_hparams: proj_scale_factor: 0
|
||||
load_hparams: n_wa_pattern: 0
|
||||
|
||||
load_hparams: model size: 1044.36 MiB
|
||||
load_hparams: metadata size: 0.16 MiB
|
||||
load_tensors: ffn up/down are swapped
|
||||
alloc_compute_meta: Vulkan0 compute buffer size = 100.30 MiB
|
||||
alloc_compute_meta: CPU compute buffer size = 18.30 MiB
|
||||
init tts....init t2w....voc_hg2_model: CPU backend using 8 threads
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=32000
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=200, n_mel=80, total_size=16000
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=200, n_mel=80, total_elements=16000
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=200, n_tokens=100, current_iter=0, current_total=0, new_total=100, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 1 (total_tokens=100, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 20]
|
||||
audition_audio_batch_encode: Final output: 20 tokens x 4096 dims
|
||||
=== Model Paths ===
|
||||
Base dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf
|
||||
LLM: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf [OK]
|
||||
Vision: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/vision/MiniCPM-o-4_5-vision-F16.gguf [OK]
|
||||
Audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf [OK]
|
||||
TTS: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf [OK]
|
||||
Projector: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf [OK]
|
||||
===================
|
||||
=== Initializing Omni Context ===
|
||||
Media type: 2 (omni: audio+vision)
|
||||
TTS enabled: yes
|
||||
Context size: 2048
|
||||
GPU layers: 99
|
||||
Token2Wav device: cpu
|
||||
Vision backend: metal
|
||||
TTS bin dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts
|
||||
Ref audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\vllm-omni-edge\benchmarks\edge_harness\results\e2e_expansion_20260923\evidence\minicpmo_cpp_radeon890m_prompted\fixture_0000.wav
|
||||
02:42:58.124 === omni_init start
|
||||
02:42:58.125 media_type = 2, duplex_mode = 0, base_output_dir = ./tools/omni/output
|
||||
02:42:58.125 === omni_init: loading new LLM model
|
||||
02:43:50.206 === omni_init: loading TTS model
|
||||
02:43:50.206 TTS model: loading with n_gpu_layers=0
|
||||
02:43:53.869 TTS sampler: temp=0.80, top_p=0.85, top_k=25, rep_penalty=1.05
|
||||
02:43:53.869 TTS: loading weights from GGUF (emb_code, emb_text, projector_semantic, head_code)...
|
||||
02:43:59.383 TTS: head_code shape: dim0=6562, dim1=768, will transpose to [6562, 768]
|
||||
02:43:59.389 TTS: head_code loaded, verifying first few values:
|
||||
02:43:59.389 head_code_weight[0] = 0.01361084 (expect ~0.01385498)
|
||||
02:43:59.389 head_code_weight[1] = -0.01660156 (expect ~-0.01647949)
|
||||
02:43:59.389 head_code_weight[768] = 0.04101562 (this is weight[1, 0], expect ~0.04150391)
|
||||
02:43:59.436 TTS: weights loaded successfully
|
||||
02:43:59.436 Projector: loading from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf
|
||||
02:43:59.538 Projector: loaded successfully
|
||||
02:43:59.538 === omni_init: loading APM model
|
||||
02:44:02.572 APM: init from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf
|
||||
02:44:08.113 Token2Wav: found models in \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf
|
||||
02:44:08.176 Token2Wav: all model files found, initializing session...
|
||||
02:44:08.176 Token2Wav: non-CUDA backend, vocoder using CPU for better performance
|
||||
02:44:08.176 Token2Wav: using prompt_cache from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf/prompt_cache.gguf
|
||||
02:44:12.764 Token2Wav: initialized successfully
|
||||
02:44:12.764 Python T2W: script_dir=./tools/omni/pyt2w, model_dir=./tools/omni/pyt2w/token2wav
|
||||
02:44:12.764 Token2Wav: 浣跨敤 C++ 瀹炵幇
|
||||
02:44:12.764 === omni_init success: ctx_llama = 000002757e4cf830
|
||||
=== Running test case ===
|
||||
Audio prefix: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-11-ea3tyiaz\fixture_
|
||||
Count: 2
|
||||
02:44:12.764 stream_prefill: n_past = 0
|
||||
voice_clone_prompt = <|im_start|>system
|
||||
妯′豢闊抽�鏍锋湰鐨勯煶鑹插苟鐢熸垚鏂扮殑鍐呭�銆�
|
||||
<|audio_start|>
|
||||
assistant_prompt = <|audio_end|>浣犵殑浠诲姟鏄�敤杩欑�澹伴煶妯″紡鏉ュ綋涓€涓�姪鎵嬨€傝�璁ょ湡銆侀珮璐ㄩ噺鍦板洖澶嶇敤鎴风殑闂��銆傝�鐢ㄩ珮鑷�劧搴︾殑鏂瑰紡鍜岀敤鎴疯亰澶┿€�<|im_end|>
|
||||
<|im_start|>user
|
||||
|
||||
02:44:12.764 system prompt ref_audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-11-ea3tyiaz\fixture_0000.wav
|
||||
02:44:13.341 system prompt ref_audio embedding: n_pos=20
|
||||
02:44:13.846 馃敀 n_keep 璁剧疆涓� 72 (system prompt tokens)锛岃繖閮ㄥ垎姘歌繙涓嶄細琚�粦鍔ㄧ獥鍙ｅ垹闄�
|
||||
02:44:13.846 stream_prefill(index=0): system prompt 鍒濆�鍖栧畬鎴愶紝ref_audio 宸插湪鍏朵腑 prefill
|
||||
02:44:13.846 [SW] system_prompt registered: preserve_length=72 (will be protected from sliding)
|
||||
02:44:13.846 n_past = 72
|
||||
02:44:13.846
|
||||
|
||||
c++ finish stream_prefill(index=0). n_past=72, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 0 (audio) : 1.08176 s
|
||||
encode_image_with_vision_chunks: step 1 of 1 encoded in 634.25 ms
|
||||
encode_image_with_vision_chunks: all 1 chunks encoded in 652.30 ms (grid: 1x1)
|
||||
|
||||
encode_image_with_vision_chunks: image encoded in 652.31 ms by vision ( 652.31 ms per chunk, 64 total tokens)
|
||||
omni_image_embed_make_chunks_with_filename: created 1 vision chunks from C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-11-ea3tyiaz\fixture_0001.jpg
|
||||
stream_prefill: prefilled 1 vision chunks (64 tokens each)
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=49339
|
||||
audition_audio_preprocess: Audio not aligned to 100ms (49339 samples = 3083.7ms), padded with silence to 49600 samples (3100ms)
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=310, n_mel=80, total_size=24800
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=310, n_mel=80, total_elements=24800
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=310, n_tokens=155, current_iter=1, current_total=155, new_total=310, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 2 (total_tokens=310, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 31]
|
||||
audition_audio_batch_encode: Final output: 31 tokens x 4096 dims
|
||||
02:44:15.023 stream_prefill(index=1): processing user audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-11-ea3tyiaz\fixture_0001.wav
|
||||
02:44:15.322 stream_prefill(index=1): user audio embedding: n_pos=31
|
||||
02:44:15.635
|
||||
|
||||
c++ finish stream_prefill(index=1). n_past=172, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 1 (audio+vision) : 1.78776 s
|
||||
02:44:15.635 馃搷 stream_decode 寮€濮�: n_past=172, n_keep=72, n_ctx=2048, duplex_mode=0
|
||||
02:44:15.635 stream_decode: create tts thread (simplex mode)
|
||||
02:44:15.635 stream_decode: create t2w thread
|
||||
02:44:15.635 wait prefill done
|
||||
02:44:15.635 馃搷 [鍗曞伐TTS] 娣诲姞 assistant prompt: "<|im_end|>
|
||||
<|im_start|>assistant
|
||||
<think>
|
||||
|
||||
</think>
|
||||
|
||||
<|tts_bos|>", n_past=172
|
||||
02:44:15.636 T2W thread (C++) started
|
||||
<user>
|
||||
<assistant>02:44:15.639 TTS: 鍒涘缓鍗曞伐妯″紡杈撳嚭鐩�綍: ./tools/omni/output/round_000
|
||||
02:44:15.639 TTS thread started
|
||||
02:44:15.798 馃搷 [鍗曞伐TTS] assistant prompt 瀹屾垚, n_past=182
|
||||
02:44:15.798 LLM decode: max_tgt_len = 96, n_predict = 96, n_ctx = 2048
|
||||
02:44:16.326 LLM: detected end token, set llm_generation_done=true
|
||||
02:44:16.327 LLM->TTS: text='The square in the image is red.', n_tokens=8, hidden_size=32768, n_embd=4096, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:44:16.327 馃搷 Decode 缁撴潫: n_past=191, 鍓╀綑绌洪棿=1857, 鏃犻渶婊戠獥
|
||||
02:44:16.327 馃搷 杞�� 1 缁撴潫锛岃�褰曡竟鐣屼簬 n_past=191
|
||||
02:44:16.327 TTS<-LLM: chunk_idx=0, text='The square in the image is red.', n_tokens=8, hidden_size=32768, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:44:16.327 TTS: after queue pop - speek_done=0, llm_finish=1, llm_text.empty=0, token_ids.size=8
|
||||
02:44:16.327 TTS: before empty check - response.empty=0, response='The square in the image is red.', llm_finish=1
|
||||
Waiting for audio generation to complete...
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_0.wav does not exist or is empty
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_1.wav does not exist or is empty
|
||||
TTS: no valid WAV files to merge
|
||||
02:44:16.327 TTS: DEBUG before has_llm_data - token_ids.size=8, hidden_states.size=32768, n_embd=4096
|
||||
02:44:16.327 TTS: has_llm_data=1, token_ids=8, hidden_states=32768, n_embd=4096, llm_finish=1
|
||||
02:44:16.328 TTS: n_tokens_orig=8, n_tokens_filtered=8 (filtered 0 special tokens)
|
||||
02:44:16.360 TTS: saved merged embeddings (text) to ./tools/omni/output/round_000/llm_debug/chunk_0/merged_embeddings.txt
|
||||
02:44:16.360 TTS Simplex: generating audio tokens for chunk 0 (n_tokens=8, tts_n_embd=768)
|
||||
02:44:16.360 TTS Simplex: 鍦� prefill 鍓嶆坊鍔� audio_bos (chunk_idx=0, new_size=9)
|
||||
02:44:16.360 TTS Simplex: first chunk - cleared KV cache
|
||||
02:44:16.360 TTS Simplex: first chunk - cleared tts_all_generated_tokens and tts_token_buffer
|
||||
02:44:16.422 馃搷 涓轰笅涓€杞�噯澶�: eval <|im_end|>\n<|im_start|>user\n, n_past=196
|
||||
02:44:16.635 TTS Simplex: prefill completed, n_past_tts=9
|
||||
02:44:16.635 TTS Simplex: sampler created
|
||||
02:44:16.635 TTS simplex: is_audio_bos=true (first audio token)
|
||||
02:44:16.662 TTS Simplex Phase1: token 1/500: rel_id=767
|
||||
02:44:16.673 TTS Simplex Phase1: token 2/500: rel_id=1704
|
||||
02:44:16.684 TTS Simplex Phase1: token 3/500: rel_id=3648
|
||||
02:44:16.695 TTS Simplex Phase1: token 4/500: rel_id=3645
|
||||
02:44:16.706 TTS Simplex Phase1: token 5/500: rel_id=2031
|
||||
02:44:16.925 TTS Simplex Phase1: yield 25 tokens 鍒� T2W (step 25, buffer=25)
|
||||
02:44:16.925 TTS Simplex Phase1: token 25/500: rel_id=5934
|
||||
02:44:16.925 T2W绾跨▼(C++): 杞��鍒囨崲 (-1 -> 0)锛屾洿鏂拌緭鍑虹洰褰�
|
||||
02:44:16.925 T2W绾跨▼(C++): 鏂拌緭鍑虹洰褰� ./tools/omni/output/round_000/tts_wav
|
||||
02:44:17.087 TTS Simplex Phase1: EOS token at step 40
|
||||
02:44:17.087 TTS Simplex: is_final_text_chunk=true, will enter Phase 2 for text_eos_embed
|
||||
02:44:17.087 TTS Simplex Phase2: injecting text_eos_embed + audio_bos at n_past=48
|
||||
02:44:17.095 TTS Simplex Phase2: text_eos_embed injected, n_past=49
|
||||
02:44:17.103 TTS Simplex Phase2: audio_bos injected, n_past=50, starting final generation
|
||||
02:44:17.223 TTS Simplex Phase2: yield 25 tokens 鍒� T2W
|
||||
02:44:17.311 TTS Simplex Phase2: EOS at step 19 (final end)
|
||||
02:44:17.311 TTS Simplex: chunk 缁撴潫锛宼ts_token_buffer 鍓╀綑 7 tokens (淇濈暀绛変笅涓€涓� chunk)
|
||||
02:44:17.311 TTS Simplex: is_final_text_chunk=true, flush 鍓╀綑 7 tokens 浠� tts_token_buffer
|
||||
02:44:17.311 TTS Simplex: generated 57 audio tokens for chunk 0
|
||||
02:44:17.311 TTS Simplex: chunk 0 first tokens: [767 1704 3648 3645 2031 ], last tokens: [735 4218 4137 4218 6081 ]
|
||||
02:44:17.311 TTS Simplex: updated tts_n_past_accumulated=69, total_generated_tokens=57
|
||||
02:44:17.312 TTS: finished processing all chunks
|
||||
02:44:17.312 TTS: 鍗曞伐妯″紡杞��缁撴潫锛屼笅涓€杞� round_idx=1
|
||||
02:44:17.312 TTS: sent is_final=true to T2W queue (turn end)
|
||||
02:44:19.261 馃帀 棣栧搷鏃堕棿 (First Audio Response): 3625ms
|
||||
02:44:19.261 T2W绾跨▼: wav_0.wav | 0.84s audio | 2334.4ms inference | RTF=2.78 | t=3625ms
|
||||
02:44:21.496 T2W绾跨▼: wav_1.wav | 1.00s audio | 2235.1ms inference | RTF=2.24 | t=5861ms
|
||||
02:44:23.555 T2W绾跨▼: wav_2.wav | 0.56s audio | 2057.6ms inference | RTF=3.67 | t=7919ms
|
||||
02:44:23.565 T2W绾跨▼: 鍐欏叆缁撴潫鏍囪� ./tools/omni/output/round_000/tts_wav/generation_done.flag (last_wav=2)
|
||||
02:44:23.565 T2W绾跨▼: 杞��缁撴潫鍚庢洿鏂拌緭鍑虹洰褰曚负 ./tools/omni/output/round_001/tts_wav
|
||||
02:44:23.60602 :T2W(C++) 绾跨▼: 鍋滄�
|
||||
[timing] call=0 tokens=28 final=0 token2mel=2002.770ms vocoder=331.609ms total=2334.394ms audio=20160
|
||||
[timing] call=1 tokens=28 final=0 token2mel=1854.383ms vocoder=380.674ms total=2235.069ms audio=24000
|
||||
[timing] call=2 tokens=10 final=1 token2mel=1860.585ms vocoder=196.983ms total=2057.579ms audio=13440
|
||||
Audio generation completed.
|
||||
llama_perf_context_print: load time = 25463.47 ms
|
||||
llama_perf_context_print: prompt eval time = 3032.64 ms / 182 tokens ( 16.66 ms per token, 60.01 tokens per second)
|
||||
llama_perf_context_print: eval time = 526.40 ms / 9 runs ( 58.49 ms per token, 17.10 tokens per second)
|
||||
llama_perf_context_print: total time = 33271.53 ms / 191 tokens
|
||||
llama_perf_context_print: graphs reused = 10
|
||||
Token2Wav (C++): session released
|
||||
44:23.606 omni_stop_threads: stop signals sent
|
||||
tts thread end
|
||||
t2w thread end
|
||||
+483
@@ -0,0 +1,483 @@
|
||||
build: 64 (5202b7b) with x86_64-w64-mingw32-gcc-posix (GCC) 13-posix for x86_64-w64-mingw32
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 30720 MiB free
|
||||
llama_model_loader: loaded meta data with 27 key-value pairs and 399 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = qwen3
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Llm
|
||||
llama_model_loader: - kv 3: general.size_label str = 8.2B
|
||||
llama_model_loader: - kv 4: qwen3.block_count u32 = 36
|
||||
llama_model_loader: - kv 5: qwen3.context_length u32 = 40960
|
||||
llama_model_loader: - kv 6: qwen3.embedding_length u32 = 4096
|
||||
llama_model_loader: - kv 7: qwen3.feed_forward_length u32 = 12288
|
||||
llama_model_loader: - kv 8: qwen3.attention.head_count u32 = 32
|
||||
llama_model_loader: - kv 9: qwen3.attention.head_count_kv u32 = 8
|
||||
llama_model_loader: - kv 10: qwen3.rope.freq_base f32 = 1000000.000000
|
||||
llama_model_loader: - kv 11: qwen3.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: qwen3.attention.key_length u32 = 128
|
||||
llama_model_loader: - kv 13: qwen3.attention.value_length u32 = 128
|
||||
llama_model_loader: - kv 14: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 15: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 16: tokenizer.ggml.tokens arr[str,151748] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 17: tokenizer.ggml.token_type arr[i32,151748] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 24: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 25: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 26: general.file_type u32 = 15
|
||||
llama_model_loader: - type f32: 145 tensors
|
||||
llama_model_loader: - type q4_K: 217 tensors
|
||||
llama_model_loader: - type q6_K: 37 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = Q4_K - Medium
|
||||
print_info: file size = 4.68 GiB (4.90 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = qwen3
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 40960
|
||||
print_info: n_embd = 4096
|
||||
print_info: n_layer = 36
|
||||
print_info: n_head = 32
|
||||
print_info: n_head_kv = 8
|
||||
print_info: n_rot = 128
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 128
|
||||
print_info: n_embd_head_v = 128
|
||||
print_info: n_gqa = 4
|
||||
print_info: n_embd_k_gqa = 1024
|
||||
print_info: n_embd_v_gqa = 1024
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 12288
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = -1
|
||||
print_info: rope type = 2
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 1000000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 40960
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = 8B
|
||||
print_info: model params = 8.19 B
|
||||
print_info: general.name = Llm
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 151748
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 36 repeating layers to GPU
|
||||
load_tensors: offloading output layer to GPU
|
||||
load_tensors: offloaded 37/37 layers to GPU
|
||||
load_tensors: Vulkan0 model buffer size = 4454.74 MiB
|
||||
load_tensors: CPU_Mapped model buffer size = 333.43 MiB
|
||||
.....................................................................................
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 1000000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (40960) -- the full capacity of the model will not be utilized
|
||||
llama_context: Vulkan_Host output buffer size = 0.58 MiB
|
||||
llama_kv_cache: Vulkan0 KV buffer size = 288.00 MiB
|
||||
llama_kv_cache: size = 288.00 MiB ( 2048 cells, 36 layers, 1/1 seqs), K (f16): 144.00 MiB, V (f16): 144.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 304.38 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 12.01 MiB
|
||||
llama_context: graph nodes = 1267
|
||||
llama_context: graph splits = 2
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 25673 MiB free
|
||||
llama_model_loader: loaded meta data with 30 key-value pairs and 193 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = llama
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Tts
|
||||
llama_model_loader: - kv 3: general.size_label str = 445M
|
||||
llama_model_loader: - kv 4: llama.block_count u32 = 20
|
||||
llama_model_loader: - kv 5: llama.context_length u32 = 4096
|
||||
llama_model_loader: - kv 6: llama.embedding_length u32 = 768
|
||||
llama_model_loader: - kv 7: llama.feed_forward_length u32 = 3072
|
||||
llama_model_loader: - kv 8: llama.attention.head_count u32 = 12
|
||||
llama_model_loader: - kv 9: llama.attention.head_count_kv u32 = 12
|
||||
llama_model_loader: - kv 10: llama.rope.freq_base f32 = 10000.000000
|
||||
llama_model_loader: - kv 11: llama.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: llama.attention.key_length u32 = 64
|
||||
llama_model_loader: - kv 13: llama.attention.value_length u32 = 64
|
||||
llama_model_loader: - kv 14: general.file_type u32 = 1
|
||||
llama_model_loader: - kv 15: llama.vocab_size u32 = 158626
|
||||
llama_model_loader: - kv 16: llama.rope.dimension_count u32 = 64
|
||||
llama_model_loader: - kv 17: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.tokens arr[str,158626] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.token_type arr[i32,158626] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 24: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 25: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 26: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 27: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 28: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 29: tokenizer.ggml.add_space_prefix bool = false
|
||||
llama_model_loader: - type f32: 50 tensors
|
||||
llama_model_loader: - type f16: 143 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = F16
|
||||
print_info: file size = 1.07 GiB (20.70 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = llama
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 4096
|
||||
print_info: n_embd = 768
|
||||
print_info: n_layer = 20
|
||||
print_info: n_head = 12
|
||||
print_info: n_head_kv = 12
|
||||
print_info: n_rot = 64
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 64
|
||||
print_info: n_embd_head_v = 64
|
||||
print_info: n_gqa = 1
|
||||
print_info: n_embd_k_gqa = 768
|
||||
print_info: n_embd_v_gqa = 768
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 3072
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = 0
|
||||
print_info: rope type = 0
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 10000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 4096
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = ?B
|
||||
print_info: model params = 444.94 M
|
||||
print_info: general.name = Tts
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 158626
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
load_tensors: skipping TTS tensor emb_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor emb_text.weight (size = 467140608 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.weight (size = 12582912 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.weight (size = 2359296 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.weight (size = 6291456 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.weight (size = 1179648 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor head_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 0 repeating layers to GPU
|
||||
load_tensors: offloaded 0/21 layers to GPU
|
||||
load_tensors: CPU_Mapped model buffer size = 592.48 MiB
|
||||
..............................................................
|
||||
llama_init_from_model: model default pooling_type is [0], but [-1] was specified
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 10000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (4096) -- the full capacity of the model will not be utilized
|
||||
llama_context: CPU output buffer size = 0.61 MiB
|
||||
llama_kv_cache: CPU KV buffer size = 120.00 MiB
|
||||
llama_kv_cache: size = 120.00 MiB ( 2048 cells, 20 layers, 1/1 seqs), K (f16): 60.00 MiB, V (f16): 60.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 30.75 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 7.01 MiB
|
||||
llama_context: graph nodes = 613
|
||||
llama_context: graph splits = 216 (with bs=512), 1 (with bs=1)
|
||||
vision_model_loader: model name:
|
||||
vision_model_loader: description: image encoder for MiniCPM-V
|
||||
vision_model_loader: GGUF version: 3
|
||||
vision_model_loader: alignment: 32
|
||||
vision_model_loader: n_tensors: 455
|
||||
vision_model_loader: n_kv: 20
|
||||
|
||||
vision_ctx: vision using Vulkan0 backend
|
||||
load_hparams: model_type: 0
|
||||
load_hparams: n_embd: 1152
|
||||
load_hparams: n_head: 16
|
||||
load_hparams: n_ff: 4304
|
||||
load_hparams: n_layer: 27
|
||||
load_hparams: ffn_op: gelu
|
||||
load_hparams: projection_dim: 0
|
||||
|
||||
--- vision hparams ---
|
||||
load_hparams: image_size: 448
|
||||
load_hparams: patch_size: 14
|
||||
load_hparams: minicpmv_version: 100045
|
||||
load_hparams: proj_scale_factor: 0
|
||||
load_hparams: n_wa_pattern: 0
|
||||
|
||||
load_hparams: model size: 1044.36 MiB
|
||||
load_hparams: metadata size: 0.16 MiB
|
||||
load_tensors: ffn up/down are swapped
|
||||
alloc_compute_meta: Vulkan0 compute buffer size = 100.30 MiB
|
||||
alloc_compute_meta: CPU compute buffer size = 18.30 MiB
|
||||
init tts....init t2w....voc_hg2_model: CPU backend using 8 threads
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=32000
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=200, n_mel=80, total_size=16000
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=200, n_mel=80, total_elements=16000
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=200, n_tokens=100, current_iter=0, current_total=0, new_total=100, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 1 (total_tokens=100, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 20]
|
||||
audition_audio_batch_encode: Final output: 20 tokens x 4096 dims
|
||||
=== Model Paths ===
|
||||
Base dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf
|
||||
LLM: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf [OK]
|
||||
Vision: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/vision/MiniCPM-o-4_5-vision-F16.gguf [OK]
|
||||
Audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf [OK]
|
||||
TTS: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf [OK]
|
||||
Projector: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf [OK]
|
||||
===================
|
||||
=== Initializing Omni Context ===
|
||||
Media type: 2 (omni: audio+vision)
|
||||
TTS enabled: yes
|
||||
Context size: 2048
|
||||
GPU layers: 99
|
||||
Token2Wav device: cpu
|
||||
Vision backend: metal
|
||||
TTS bin dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts
|
||||
Ref audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\vllm-omni-edge\benchmarks\edge_harness\results\e2e_expansion_20260923\evidence\minicpmo_cpp_radeon890m_prompted\fixture_0000.wav
|
||||
02:44:25.400 === omni_init start
|
||||
02:44:25.400 media_type = 2, duplex_mode = 0, base_output_dir = ./tools/omni/output
|
||||
02:44:25.400 === omni_init: loading new LLM model
|
||||
02:45:10.901 === omni_init: loading TTS model
|
||||
02:45:10.901 TTS model: loading with n_gpu_layers=0
|
||||
02:45:14.552 TTS sampler: temp=0.80, top_p=0.85, top_k=25, rep_penalty=1.05
|
||||
02:45:14.552 TTS: loading weights from GGUF (emb_code, emb_text, projector_semantic, head_code)...
|
||||
02:45:20.095 TTS: head_code shape: dim0=6562, dim1=768, will transpose to [6562, 768]
|
||||
02:45:20.101 TTS: head_code loaded, verifying first few values:
|
||||
02:45:20.101 head_code_weight[0] = 0.01361084 (expect ~0.01385498)
|
||||
02:45:20.101 head_code_weight[1] = -0.01660156 (expect ~-0.01647949)
|
||||
02:45:20.101 head_code_weight[768] = 0.04101562 (this is weight[1, 0], expect ~0.04150391)
|
||||
02:45:20.149 TTS: weights loaded successfully
|
||||
02:45:20.149 Projector: loading from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf
|
||||
02:45:20.253 Projector: loaded successfully
|
||||
02:45:20.253 === omni_init: loading APM model
|
||||
02:45:23.311 APM: init from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf
|
||||
02:45:28.667 Token2Wav: found models in \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf
|
||||
02:45:28.722 Token2Wav: all model files found, initializing session...
|
||||
02:45:28.722 Token2Wav: non-CUDA backend, vocoder using CPU for better performance
|
||||
02:45:28.722 Token2Wav: using prompt_cache from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf/prompt_cache.gguf
|
||||
02:45:33.241 Token2Wav: initialized successfully
|
||||
02:45:33.241 Python T2W: script_dir=./tools/omni/pyt2w, model_dir=./tools/omni/pyt2w/token2wav
|
||||
02:45:33.241 Token2Wav: 浣跨敤 C++ 瀹炵幇
|
||||
02:45:33.241 === omni_init success: ctx_llama = 000002ae2acef830
|
||||
=== Running test case ===
|
||||
Audio prefix: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-12-xr5v5uhs\fixture_
|
||||
Count: 2
|
||||
02:45:33.241 stream_prefill: n_past = 0
|
||||
voice_clone_prompt = <|im_start|>system
|
||||
妯′豢闊抽�鏍锋湰鐨勯煶鑹插苟鐢熸垚鏂扮殑鍐呭�銆�
|
||||
<|audio_start|>
|
||||
assistant_prompt = <|audio_end|>浣犵殑浠诲姟鏄�敤杩欑�澹伴煶妯″紡鏉ュ綋涓€涓�姪鎵嬨€傝�璁ょ湡銆侀珮璐ㄩ噺鍦板洖澶嶇敤鎴风殑闂��銆傝�鐢ㄩ珮鑷�劧搴︾殑鏂瑰紡鍜岀敤鎴疯亰澶┿€�<|im_end|>
|
||||
<|im_start|>user
|
||||
|
||||
02:45:33.241 system prompt ref_audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-12-xr5v5uhs\fixture_0000.wav
|
||||
02:45:33.883 system prompt ref_audio embedding: n_pos=20
|
||||
02:45:34.380 馃敀 n_keep 璁剧疆涓� 72 (system prompt tokens)锛岃繖閮ㄥ垎姘歌繙涓嶄細琚�粦鍔ㄧ獥鍙ｅ垹闄�
|
||||
02:45:34.380 stream_prefill(index=0): system prompt 鍒濆�鍖栧畬鎴愶紝ref_audio 宸插湪鍏朵腑 prefill
|
||||
02:45:34.380 [SW] system_prompt registered: preserve_length=72 (will be protected from sliding)
|
||||
02:45:34.380 n_past = 72
|
||||
02:45:34.380
|
||||
|
||||
c++ finish stream_prefill(index=0). n_past=72, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 0 (audio) : 1.13862 s
|
||||
encode_image_with_vision_chunks: step 1 of 1 encoded in 621.77 ms
|
||||
encode_image_with_vision_chunks: all 1 chunks encoded in 639.77 ms (grid: 1x1)
|
||||
|
||||
encode_image_with_vision_chunks: image encoded in 639.77 ms by vision ( 639.77 ms per chunk, 64 total tokens)
|
||||
omni_image_embed_make_chunks_with_filename: created 1 vision chunks from C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-12-xr5v5uhs\fixture_0001.jpg
|
||||
stream_prefill: prefilled 1 vision chunks (64 tokens each)
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=49339
|
||||
audition_audio_preprocess: Audio not aligned to 100ms (49339 samples = 3083.7ms), padded with silence to 49600 samples (3100ms)
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=310, n_mel=80, total_size=24800
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=310, n_mel=80, total_elements=24800
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=310, n_tokens=155, current_iter=1, current_total=155, new_total=310, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 2 (total_tokens=310, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 31]
|
||||
audition_audio_batch_encode: Final output: 31 tokens x 4096 dims
|
||||
02:45:35.534 stream_prefill(index=1): processing user audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-12-xr5v5uhs\fixture_0001.wav
|
||||
02:45:35.835 stream_prefill(index=1): user audio embedding: n_pos=31
|
||||
02:45:36.147
|
||||
|
||||
c++ finish stream_prefill(index=1). n_past=172, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 1 (audio+vision) : 1.76615 s
|
||||
02:45:36.147 馃搷 stream_decode 寮€濮�: n_past=172, n_keep=72, n_ctx=2048, duplex_mode=0
|
||||
02:45:36.148 stream_decode: create tts thread (simplex mode)
|
||||
02:45:36.148 stream_decode: create t2w thread
|
||||
02:45:36.148 wait prefill done
|
||||
02:45:36.148 馃搷 [鍗曞伐TTS] 娣诲姞 assistant prompt: "<|im_end|>
|
||||
<|im_start|>assistant
|
||||
<think>
|
||||
|
||||
</think>
|
||||
|
||||
<|tts_bos|>", n_past=172
|
||||
02:45:36.148 T2W thread (C++) started
|
||||
<user>
|
||||
<assistant>02:45:36.151 TTS: 鍒涘缓鍗曞伐妯″紡杈撳嚭鐩�綍: ./tools/omni/output/round_000
|
||||
02:45:36.151 TTS thread started
|
||||
02:45:36.310 馃搷 [鍗曞伐TTS] assistant prompt 瀹屾垚, n_past=182
|
||||
02:45:36.313 LLM decode: max_tgt_len = 96, n_predict = 96, n_ctx = 2048
|
||||
02:45:36.830 LLM: detected end token, set llm_generation_done=true
|
||||
02:45:36.831 LLM->TTS: text='The square in the image is red.', n_tokens=8, hidden_size=32768, n_embd=4096, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:45:36.831 馃搷 Decode 缁撴潫: n_past=191, 鍓╀綑绌洪棿=1857, 鏃犻渶婊戠獥
|
||||
02:45:36.831 馃搷 杞�� 1 缁撴潫锛岃�褰曡竟鐣屼簬 n_past=191
|
||||
02:45:36.831 TTS<-LLM: chunk_idx=0, text='The square in the image is red.', n_tokens=8, hidden_size=32768, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:45:36.831 TTS: after queue pop - speek_done=0, llm_finish=1, llm_text.empty=0, token_ids.size=8
|
||||
02:45:36.831 TTS: before empty check - response.empty=0, response='The square in the image is red.', llm_finish=1
|
||||
Waiting for audio generation to complete...
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_0.wav does not exist or is empty
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_1.wav does not exist or is empty
|
||||
TTS: no valid WAV files to merge
|
||||
02:45:36.831 TTS: DEBUG before has_llm_data - token_ids.size=8, hidden_states.size=32768, n_embd=4096
|
||||
02:45:36.831 TTS: has_llm_data=1, token_ids=8, hidden_states=32768, n_embd=4096, llm_finish=1
|
||||
02:45:36.832 TTS: n_tokens_orig=8, n_tokens_filtered=8 (filtered 0 special tokens)
|
||||
02:45:36.869 TTS: saved merged embeddings (text) to ./tools/omni/output/round_000/llm_debug/chunk_0/merged_embeddings.txt
|
||||
02:45:36.869 TTS Simplex: generating audio tokens for chunk 0 (n_tokens=8, tts_n_embd=768)
|
||||
02:45:36.869 TTS Simplex: 鍦� prefill 鍓嶆坊鍔� audio_bos (chunk_idx=0, new_size=9)
|
||||
02:45:36.869 TTS Simplex: first chunk - cleared KV cache
|
||||
02:45:36.869 TTS Simplex: first chunk - cleared tts_all_generated_tokens and tts_token_buffer
|
||||
02:45:36.922 馃搷 涓轰笅涓€杞�噯澶�: eval <|im_end|>\n<|im_start|>user\n, n_past=196
|
||||
02:45:37.143 TTS Simplex: prefill completed, n_past_tts=9
|
||||
02:45:37.143 TTS Simplex: sampler created
|
||||
02:45:37.143 TTS simplex: is_audio_bos=true (first audio token)
|
||||
02:45:37.169 TTS Simplex Phase1: token 1/500: rel_id=1472
|
||||
02:45:37.180 TTS Simplex Phase1: token 2/500: rel_id=6486
|
||||
02:45:37.191 TTS Simplex Phase1: token 3/500: rel_id=6486
|
||||
02:45:37.202 TTS Simplex Phase1: token 4/500: rel_id=6486
|
||||
02:45:37.213 TTS Simplex Phase1: token 5/500: rel_id=4299
|
||||
02:45:37.434 TTS Simplex Phase1: yield 25 tokens 鍒� T2W (step 25, buffer=25)
|
||||
02:45:37.434 TTS Simplex Phase1: token 25/500: rel_id=1806
|
||||
02:45:37.434 T2W绾跨▼(C++): 杞��鍒囨崲 (-1 -> 0)锛屾洿鏂拌緭鍑虹洰褰�
|
||||
02:45:37.434 T2W绾跨▼(C++): 鏂拌緭鍑虹洰褰� ./tools/omni/output/round_000/tts_wav
|
||||
02:45:37.658 TTS Simplex Phase1: EOS token at step 46
|
||||
02:45:37.658 TTS Simplex: is_final_text_chunk=true, will enter Phase 2 for text_eos_embed
|
||||
02:45:37.658 TTS Simplex Phase2: injecting text_eos_embed + audio_bos at n_past=54
|
||||
02:45:37.666 TTS Simplex Phase2: text_eos_embed injected, n_past=55
|
||||
02:45:37.674 TTS Simplex Phase2: audio_bos injected, n_past=56, starting final generation
|
||||
02:45:37.733 TTS Simplex Phase2: yield 25 tokens 鍒� T2W
|
||||
02:45:37.898 TTS Simplex Phase2: EOS at step 20 (final end)
|
||||
02:45:37.898 TTS Simplex: chunk 缁撴潫锛宼ts_token_buffer 鍓╀綑 14 tokens (淇濈暀绛変笅涓€涓� chunk)
|
||||
02:45:37.898 TTS Simplex: is_final_text_chunk=true, flush 鍓╀綑 14 tokens 浠� tts_token_buffer
|
||||
02:45:37.898 TTS Simplex: generated 64 audio tokens for chunk 0
|
||||
02:45:37.898 TTS Simplex: chunk 0 first tokens: [1472 6486 6486 6486 4299 ], last tokens: [4299 6486 6486 4299 4299 ]
|
||||
02:45:37.898 TTS Simplex: updated tts_n_past_accumulated=76, total_generated_tokens=64
|
||||
02:45:37.908 TTS: finished processing all chunks
|
||||
02:45:37.908 TTS: 鍗曞伐妯″紡杞��缁撴潫锛屼笅涓€杞� round_idx=1
|
||||
02:45:37.908 TTS: sent is_final=true to T2W queue (turn end)
|
||||
02:45:39.767 馃帀 棣栧搷鏃堕棿 (First Audio Response): 3619ms
|
||||
02:45:39.767 T2W绾跨▼: wav_0.wav | 0.84s audio | 2331.8ms inference | RTF=2.78 | t=3619ms
|
||||
02:45:42.000 T2W绾跨▼: wav_1.wav | 1.00s audio | 2232.1ms inference | RTF=2.23 | t=5852ms
|
||||
02:45:44.157 T2W绾跨▼: wav_2.wav | 0.84s audio | 2156.4ms inference | RTF=2.57 | t=8009ms
|
||||
02:45:44.157 T2W绾跨▼: 鍐欏叆缁撴潫鏍囪� ./tools/omni/output/round_000/tts_wav/generation_done.flag (last_wav=2)
|
||||
02:45:44.157 T2W绾跨▼: 杞��缁撴潫鍚庢洿鏂拌緭鍑虹洰褰曚负 ./tools/omni/output/round_001/tts_wav
|
||||
02:45:02:44.198 45:omni_stop_threads: stop signals sent
|
||||
44.198 T2W(C++) 绾跨▼: 鍋滄�
|
||||
[timing] call=0 tokens=28 final=0 token2mel=2004.545ms vocoder=327.177ms total=2331.737ms audio=20160
|
||||
[timing] call=1 tokens=28 final=0 token2mel=1850.819ms vocoder=381.245ms total=2232.086ms audio=24000
|
||||
[timing] call=2 tokens=17 final=1 token2mel=1871.335ms vocoder=285.029ms total=2156.374ms audio=20160
|
||||
Audio generation completed.
|
||||
llama_perf_context_print: load time = 25053.93 ms
|
||||
llama_perf_context_print: prompt eval time = 3071.36 ms / 182 tokens ( 16.88 ms per token, 59.26 tokens per second)
|
||||
llama_perf_context_print: eval time = 514.66 ms / 9 runs ( 57.18 ms per token, 17.49 tokens per second)
|
||||
llama_perf_context_print: total time = 32938.18 ms / 191 tokens
|
||||
llama_perf_context_print: graphs reused = 10
|
||||
Token2Wav (C++): session released
|
||||
tts thread end
|
||||
t2w thread end
|
||||
+483
@@ -0,0 +1,483 @@
|
||||
build: 64 (5202b7b) with x86_64-w64-mingw32-gcc-posix (GCC) 13-posix for x86_64-w64-mingw32
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 30720 MiB free
|
||||
llama_model_loader: loaded meta data with 27 key-value pairs and 399 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = qwen3
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Llm
|
||||
llama_model_loader: - kv 3: general.size_label str = 8.2B
|
||||
llama_model_loader: - kv 4: qwen3.block_count u32 = 36
|
||||
llama_model_loader: - kv 5: qwen3.context_length u32 = 40960
|
||||
llama_model_loader: - kv 6: qwen3.embedding_length u32 = 4096
|
||||
llama_model_loader: - kv 7: qwen3.feed_forward_length u32 = 12288
|
||||
llama_model_loader: - kv 8: qwen3.attention.head_count u32 = 32
|
||||
llama_model_loader: - kv 9: qwen3.attention.head_count_kv u32 = 8
|
||||
llama_model_loader: - kv 10: qwen3.rope.freq_base f32 = 1000000.000000
|
||||
llama_model_loader: - kv 11: qwen3.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: qwen3.attention.key_length u32 = 128
|
||||
llama_model_loader: - kv 13: qwen3.attention.value_length u32 = 128
|
||||
llama_model_loader: - kv 14: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 15: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 16: tokenizer.ggml.tokens arr[str,151748] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 17: tokenizer.ggml.token_type arr[i32,151748] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 24: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 25: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 26: general.file_type u32 = 15
|
||||
llama_model_loader: - type f32: 145 tensors
|
||||
llama_model_loader: - type q4_K: 217 tensors
|
||||
llama_model_loader: - type q6_K: 37 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = Q4_K - Medium
|
||||
print_info: file size = 4.68 GiB (4.90 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = qwen3
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 40960
|
||||
print_info: n_embd = 4096
|
||||
print_info: n_layer = 36
|
||||
print_info: n_head = 32
|
||||
print_info: n_head_kv = 8
|
||||
print_info: n_rot = 128
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 128
|
||||
print_info: n_embd_head_v = 128
|
||||
print_info: n_gqa = 4
|
||||
print_info: n_embd_k_gqa = 1024
|
||||
print_info: n_embd_v_gqa = 1024
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 12288
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = -1
|
||||
print_info: rope type = 2
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 1000000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 40960
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = 8B
|
||||
print_info: model params = 8.19 B
|
||||
print_info: general.name = Llm
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 151748
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 36 repeating layers to GPU
|
||||
load_tensors: offloading output layer to GPU
|
||||
load_tensors: offloaded 37/37 layers to GPU
|
||||
load_tensors: Vulkan0 model buffer size = 4454.74 MiB
|
||||
load_tensors: CPU_Mapped model buffer size = 333.43 MiB
|
||||
.....................................................................................
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 1000000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (40960) -- the full capacity of the model will not be utilized
|
||||
llama_context: Vulkan_Host output buffer size = 0.58 MiB
|
||||
llama_kv_cache: Vulkan0 KV buffer size = 288.00 MiB
|
||||
llama_kv_cache: size = 288.00 MiB ( 2048 cells, 36 layers, 1/1 seqs), K (f16): 144.00 MiB, V (f16): 144.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 304.38 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 12.01 MiB
|
||||
llama_context: graph nodes = 1267
|
||||
llama_context: graph splits = 2
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 25673 MiB free
|
||||
llama_model_loader: loaded meta data with 30 key-value pairs and 193 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = llama
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Tts
|
||||
llama_model_loader: - kv 3: general.size_label str = 445M
|
||||
llama_model_loader: - kv 4: llama.block_count u32 = 20
|
||||
llama_model_loader: - kv 5: llama.context_length u32 = 4096
|
||||
llama_model_loader: - kv 6: llama.embedding_length u32 = 768
|
||||
llama_model_loader: - kv 7: llama.feed_forward_length u32 = 3072
|
||||
llama_model_loader: - kv 8: llama.attention.head_count u32 = 12
|
||||
llama_model_loader: - kv 9: llama.attention.head_count_kv u32 = 12
|
||||
llama_model_loader: - kv 10: llama.rope.freq_base f32 = 10000.000000
|
||||
llama_model_loader: - kv 11: llama.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: llama.attention.key_length u32 = 64
|
||||
llama_model_loader: - kv 13: llama.attention.value_length u32 = 64
|
||||
llama_model_loader: - kv 14: general.file_type u32 = 1
|
||||
llama_model_loader: - kv 15: llama.vocab_size u32 = 158626
|
||||
llama_model_loader: - kv 16: llama.rope.dimension_count u32 = 64
|
||||
llama_model_loader: - kv 17: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.tokens arr[str,158626] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.token_type arr[i32,158626] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 24: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 25: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 26: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 27: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 28: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 29: tokenizer.ggml.add_space_prefix bool = false
|
||||
llama_model_loader: - type f32: 50 tensors
|
||||
llama_model_loader: - type f16: 143 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = F16
|
||||
print_info: file size = 1.07 GiB (20.70 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = llama
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 4096
|
||||
print_info: n_embd = 768
|
||||
print_info: n_layer = 20
|
||||
print_info: n_head = 12
|
||||
print_info: n_head_kv = 12
|
||||
print_info: n_rot = 64
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 64
|
||||
print_info: n_embd_head_v = 64
|
||||
print_info: n_gqa = 1
|
||||
print_info: n_embd_k_gqa = 768
|
||||
print_info: n_embd_v_gqa = 768
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 3072
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = 0
|
||||
print_info: rope type = 0
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 10000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 4096
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = ?B
|
||||
print_info: model params = 444.94 M
|
||||
print_info: general.name = Tts
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 158626
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
load_tensors: skipping TTS tensor emb_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor emb_text.weight (size = 467140608 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.weight (size = 12582912 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.weight (size = 2359296 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.weight (size = 6291456 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.weight (size = 1179648 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor head_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 0 repeating layers to GPU
|
||||
load_tensors: offloaded 0/21 layers to GPU
|
||||
load_tensors: CPU_Mapped model buffer size = 592.48 MiB
|
||||
..............................................................
|
||||
llama_init_from_model: model default pooling_type is [0], but [-1] was specified
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 10000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (4096) -- the full capacity of the model will not be utilized
|
||||
llama_context: CPU output buffer size = 0.61 MiB
|
||||
llama_kv_cache: CPU KV buffer size = 120.00 MiB
|
||||
llama_kv_cache: size = 120.00 MiB ( 2048 cells, 20 layers, 1/1 seqs), K (f16): 60.00 MiB, V (f16): 60.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 30.75 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 7.01 MiB
|
||||
llama_context: graph nodes = 613
|
||||
llama_context: graph splits = 216 (with bs=512), 1 (with bs=1)
|
||||
vision_model_loader: model name:
|
||||
vision_model_loader: description: image encoder for MiniCPM-V
|
||||
vision_model_loader: GGUF version: 3
|
||||
vision_model_loader: alignment: 32
|
||||
vision_model_loader: n_tensors: 455
|
||||
vision_model_loader: n_kv: 20
|
||||
|
||||
vision_ctx: vision using Vulkan0 backend
|
||||
load_hparams: model_type: 0
|
||||
load_hparams: n_embd: 1152
|
||||
load_hparams: n_head: 16
|
||||
load_hparams: n_ff: 4304
|
||||
load_hparams: n_layer: 27
|
||||
load_hparams: ffn_op: gelu
|
||||
load_hparams: projection_dim: 0
|
||||
|
||||
--- vision hparams ---
|
||||
load_hparams: image_size: 448
|
||||
load_hparams: patch_size: 14
|
||||
load_hparams: minicpmv_version: 100045
|
||||
load_hparams: proj_scale_factor: 0
|
||||
load_hparams: n_wa_pattern: 0
|
||||
|
||||
load_hparams: model size: 1044.36 MiB
|
||||
load_hparams: metadata size: 0.16 MiB
|
||||
load_tensors: ffn up/down are swapped
|
||||
alloc_compute_meta: Vulkan0 compute buffer size = 100.30 MiB
|
||||
alloc_compute_meta: CPU compute buffer size = 18.30 MiB
|
||||
init tts....init t2w....voc_hg2_model: CPU backend using 8 threads
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=32000
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=200, n_mel=80, total_size=16000
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=200, n_mel=80, total_elements=16000
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=200, n_tokens=100, current_iter=0, current_total=0, new_total=100, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 1 (total_tokens=100, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 20]
|
||||
audition_audio_batch_encode: Final output: 20 tokens x 4096 dims
|
||||
=== Model Paths ===
|
||||
Base dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf
|
||||
LLM: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf [OK]
|
||||
Vision: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/vision/MiniCPM-o-4_5-vision-F16.gguf [OK]
|
||||
Audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf [OK]
|
||||
TTS: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf [OK]
|
||||
Projector: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf [OK]
|
||||
===================
|
||||
=== Initializing Omni Context ===
|
||||
Media type: 2 (omni: audio+vision)
|
||||
TTS enabled: yes
|
||||
Context size: 2048
|
||||
GPU layers: 99
|
||||
Token2Wav device: cpu
|
||||
Vision backend: metal
|
||||
TTS bin dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts
|
||||
Ref audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\vllm-omni-edge\benchmarks\edge_harness\results\e2e_expansion_20260923\evidence\minicpmo_cpp_radeon890m_prompted\fixture_0000.wav
|
||||
02:45:45.895 === omni_init start
|
||||
02:45:45.896 media_type = 2, duplex_mode = 0, base_output_dir = ./tools/omni/output
|
||||
02:45:45.896 === omni_init: loading new LLM model
|
||||
02:46:36.401 === omni_init: loading TTS model
|
||||
02:46:36.401 TTS model: loading with n_gpu_layers=0
|
||||
02:46:40.071 TTS sampler: temp=0.80, top_p=0.85, top_k=25, rep_penalty=1.05
|
||||
02:46:40.071 TTS: loading weights from GGUF (emb_code, emb_text, projector_semantic, head_code)...
|
||||
02:46:45.542 TTS: head_code shape: dim0=6562, dim1=768, will transpose to [6562, 768]
|
||||
02:46:45.548 TTS: head_code loaded, verifying first few values:
|
||||
02:46:45.548 head_code_weight[0] = 0.01361084 (expect ~0.01385498)
|
||||
02:46:45.548 head_code_weight[1] = -0.01660156 (expect ~-0.01647949)
|
||||
02:46:45.548 head_code_weight[768] = 0.04101562 (this is weight[1, 0], expect ~0.04150391)
|
||||
02:46:45.598 TTS: weights loaded successfully
|
||||
02:46:45.598 Projector: loading from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf
|
||||
02:46:45.702 Projector: loaded successfully
|
||||
02:46:45.703 === omni_init: loading APM model
|
||||
02:46:48.730 APM: init from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf
|
||||
02:46:54.119 Token2Wav: found models in \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf
|
||||
02:46:54.176 Token2Wav: all model files found, initializing session...
|
||||
02:46:54.176 Token2Wav: non-CUDA backend, vocoder using CPU for better performance
|
||||
02:46:54.176 Token2Wav: using prompt_cache from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf/prompt_cache.gguf
|
||||
02:46:58.656 Token2Wav: initialized successfully
|
||||
02:46:58.656 Python T2W: script_dir=./tools/omni/pyt2w, model_dir=./tools/omni/pyt2w/token2wav
|
||||
02:46:58.656 Token2Wav: 浣跨敤 C++ 瀹炵幇
|
||||
02:46:58.656 === omni_init success: ctx_llama = 0000018922e4f830
|
||||
=== Running test case ===
|
||||
Audio prefix: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-13-sdf8l572\fixture_
|
||||
Count: 2
|
||||
02:46:58.657 stream_prefill: n_past = 0
|
||||
voice_clone_prompt = <|im_start|>system
|
||||
妯′豢闊抽�鏍锋湰鐨勯煶鑹插苟鐢熸垚鏂扮殑鍐呭�銆�
|
||||
<|audio_start|>
|
||||
assistant_prompt = <|audio_end|>浣犵殑浠诲姟鏄�敤杩欑�澹伴煶妯″紡鏉ュ綋涓€涓�姪鎵嬨€傝�璁ょ湡銆侀珮璐ㄩ噺鍦板洖澶嶇敤鎴风殑闂��銆傝�鐢ㄩ珮鑷�劧搴︾殑鏂瑰紡鍜岀敤鎴疯亰澶┿€�<|im_end|>
|
||||
<|im_start|>user
|
||||
|
||||
02:46:58.657 system prompt ref_audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-13-sdf8l572\fixture_0000.wav
|
||||
02:46:59.314 system prompt ref_audio embedding: n_pos=20
|
||||
02:46:59.815 馃敀 n_keep 璁剧疆涓� 72 (system prompt tokens)锛岃繖閮ㄥ垎姘歌繙涓嶄細琚�粦鍔ㄧ獥鍙ｅ垹闄�
|
||||
02:46:59.815 stream_prefill(index=0): system prompt 鍒濆�鍖栧畬鎴愶紝ref_audio 宸插湪鍏朵腑 prefill
|
||||
02:46:59.815 [SW] system_prompt registered: preserve_length=72 (will be protected from sliding)
|
||||
02:46:59.815 n_past = 72
|
||||
02:46:59.815
|
||||
|
||||
c++ finish stream_prefill(index=0). n_past=72, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 0 (audio) : 1.15864 s
|
||||
encode_image_with_vision_chunks: step 1 of 1 encoded in 621.38 ms
|
||||
encode_image_with_vision_chunks: all 1 chunks encoded in 640.12 ms (grid: 1x1)
|
||||
|
||||
encode_image_with_vision_chunks: image encoded in 640.12 ms by vision ( 640.12 ms per chunk, 64 total tokens)
|
||||
omni_image_embed_make_chunks_with_filename: created 1 vision chunks from C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-13-sdf8l572\fixture_0001.jpg
|
||||
stream_prefill: prefilled 1 vision chunks (64 tokens each)
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=49339
|
||||
audition_audio_preprocess: Audio not aligned to 100ms (49339 samples = 3083.7ms), padded with silence to 49600 samples (3100ms)
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=310, n_mel=80, total_size=24800
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=310, n_mel=80, total_elements=24800
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=310, n_tokens=155, current_iter=1, current_total=155, new_total=310, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 2 (total_tokens=310, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 31]
|
||||
audition_audio_batch_encode: Final output: 31 tokens x 4096 dims
|
||||
02:47:00.973 stream_prefill(index=1): processing user audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-13-sdf8l572\fixture_0001.wav
|
||||
02:47:01.249 stream_prefill(index=1): user audio embedding: n_pos=31
|
||||
02:47:01.561
|
||||
|
||||
c++ finish stream_prefill(index=1). n_past=172, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 1 (audio+vision) : 1.74432 s
|
||||
02:47:01.561 馃搷 stream_decode 寮€濮�: n_past=172, n_keep=72, n_ctx=2048, duplex_mode=0
|
||||
02:47:01.561 stream_decode: create tts thread (simplex mode)
|
||||
02:47:01.561 stream_decode: create t2w thread
|
||||
02:47:01.561 wait prefill done
|
||||
02:47:01.561 馃搷 [鍗曞伐TTS] 娣诲姞 assistant prompt: "<|im_end|>
|
||||
<|im_start|>assistant
|
||||
<think>
|
||||
|
||||
</think>
|
||||
|
||||
<|tts_bos|>", n_past=172
|
||||
02:47:01.562 T2W thread (C++) started
|
||||
<user>
|
||||
<assistant>02:47:01.565 TTS: 鍒涘缓鍗曞伐妯″紡杈撳嚭鐩�綍: ./tools/omni/output/round_000
|
||||
02:47:01.566 TTS thread started
|
||||
02:47:01.723 馃搷 [鍗曞伐TTS] assistant prompt 瀹屾垚, n_past=182
|
||||
02:47:01.724 LLM decode: max_tgt_len = 96, n_predict = 96, n_ctx = 2048
|
||||
02:47:02.244 LLM: detected end token, set llm_generation_done=true
|
||||
02:47:02.244 LLM->TTS: text='The square in the image is red.', n_tokens=8, hidden_size=32768, n_embd=4096, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:47:02.244 馃搷 Decode 缁撴潫: n_past=191, 鍓╀綑绌洪棿=1857, 鏃犻渶婊戠獥
|
||||
02:47:02.244 馃搷 杞�� 1 缁撴潫锛岃�褰曡竟鐣屼簬 n_past=191
|
||||
02:47:02.244 TTS<-LLM: chunk_idx=0, text='The square in the image is red.', n_tokens=8, hidden_size=32768, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:47:02.244 TTS: after queue pop - speek_done=0, llm_finish=1, llm_text.empty=0, token_ids.size=8
|
||||
02:47:02.244 TTS: before empty check - response.empty=0, response='The square in the image is red.', llm_finish=1
|
||||
Waiting for audio generation to complete...
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_0.wav does not exist or is empty
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_1.wav does not exist or is empty
|
||||
TTS: no valid WAV files to merge
|
||||
02:47:02.244 TTS: DEBUG before has_llm_data - token_ids.size=8, hidden_states.size=32768, n_embd=4096
|
||||
02:47:02.244 TTS: has_llm_data=1, token_ids=8, hidden_states=32768, n_embd=4096, llm_finish=1
|
||||
02:47:02.245 TTS: n_tokens_orig=8, n_tokens_filtered=8 (filtered 0 special tokens)
|
||||
02:47:02.275 TTS: saved merged embeddings (text) to ./tools/omni/output/round_000/llm_debug/chunk_0/merged_embeddings.txt
|
||||
02:47:02.275 TTS Simplex: generating audio tokens for chunk 0 (n_tokens=8, tts_n_embd=768)
|
||||
02:47:02.276 TTS Simplex: 鍦� prefill 鍓嶆坊鍔� audio_bos (chunk_idx=0, new_size=9)
|
||||
02:47:02.276 TTS Simplex: first chunk - cleared KV cache
|
||||
02:47:02.276 TTS Simplex: first chunk - cleared tts_all_generated_tokens and tts_token_buffer
|
||||
02:47:02.336 馃搷 涓轰笅涓€杞�噯澶�: eval <|im_end|>\n<|im_start|>user\n, n_past=196
|
||||
02:47:02.534 TTS Simplex: prefill completed, n_past_tts=9
|
||||
02:47:02.534 TTS Simplex: sampler created
|
||||
02:47:02.534 TTS simplex: is_audio_bos=true (first audio token)
|
||||
02:47:02.559 TTS Simplex Phase1: token 1/500: rel_id=1496
|
||||
02:47:02.570 TTS Simplex Phase1: token 2/500: rel_id=4218
|
||||
02:47:02.580 TTS Simplex Phase1: token 3/500: rel_id=4218
|
||||
02:47:02.591 TTS Simplex Phase1: token 4/500: rel_id=4137
|
||||
02:47:02.602 TTS Simplex Phase1: token 5/500: rel_id=4137
|
||||
02:47:02.817 TTS Simplex Phase1: yield 25 tokens 鍒� T2W (step 25, buffer=25)
|
||||
02:47:02.817 TTS Simplex Phase1: token 25/500: rel_id=1749
|
||||
02:47:02.817 T2W绾跨▼(C++): 杞��鍒囨崲 (-1 -> 0)锛屾洿鏂拌緭鍑虹洰褰�
|
||||
02:47:02.817 T2W绾跨▼(C++): 鏂拌緭鍑虹洰褰� ./tools/omni/output/round_000/tts_wav
|
||||
02:47:02.963 TTS Simplex Phase1: EOS token at step 39
|
||||
02:47:02.963 TTS Simplex: is_final_text_chunk=true, will enter Phase 2 for text_eos_embed
|
||||
02:47:02.963 TTS Simplex Phase2: injecting text_eos_embed + audio_bos at n_past=47
|
||||
02:47:02.972 TTS Simplex Phase2: text_eos_embed injected, n_past=48
|
||||
02:47:02.980 TTS Simplex Phase2: audio_bos injected, n_past=49, starting final generation
|
||||
02:47:03.112 TTS Simplex Phase2: yield 25 tokens 鍒� T2W
|
||||
02:47:03.224 TTS Simplex Phase2: EOS at step 22 (final end)
|
||||
02:47:03.224 TTS Simplex: chunk 缁撴潫锛宼ts_token_buffer 鍓╀綑 9 tokens (淇濈暀绛変笅涓€涓� chunk)
|
||||
02:47:03.224 TTS Simplex: is_final_text_chunk=true, flush 鍓╀綑 9 tokens 浠� tts_token_buffer
|
||||
02:47:03.224 TTS Simplex: generated 59 audio tokens for chunk 0
|
||||
02:47:03.224 TTS Simplex: chunk 0 first tokens: [1496 4218 4218 4137 4137 ], last tokens: [1489 4137 4218 4218 4137 ]
|
||||
02:47:03.224 TTS Simplex: updated tts_n_past_accumulated=71, total_generated_tokens=59
|
||||
02:47:03.235 TTS: finished processing all chunks
|
||||
02:47:03.235 TTS: 鍗曞伐妯″紡杞��缁撴潫锛屼笅涓€杞� round_idx=1
|
||||
02:47:03.235 TTS: sent is_final=true to T2W queue (turn end)
|
||||
02:47:05.148 馃帀 棣栧搷鏃堕棿 (First Audio Response): 3586ms
|
||||
02:47:05.148 T2W绾跨▼: wav_0.wav | 0.84s audio | 2329.7ms inference | RTF=2.77 | t=3586ms
|
||||
02:47:07.395 T2W绾跨▼: wav_1.wav | 1.00s audio | 2246.6ms inference | RTF=2.25 | t=5834ms
|
||||
02:47:09.504 T2W绾跨▼: wav_2.wav | 0.64s audio | 2108.0ms inference | RTF=3.29 | t=7942ms
|
||||
02:47:09.504 T2W绾跨▼: 鍐欏叆缁撴潫鏍囪� ./tools/omni/output/round_000/tts_wav/generation_done.flag (last_wav=2)
|
||||
02:47:09.504 T2W绾跨▼: 杞��缁撴潫鍚庢洿鏂拌緭鍑虹洰褰曚负 ./tools/omni/output/round_001/tts_wav
|
||||
02:47:09.606 omni_stop_threads: stop signals sent
|
||||
02:47:09.606 T2W(C++) 绾跨▼: 鍋滄�
|
||||
[timing] call=0 tokens=28 final=0 token2mel=2000.289ms vocoder=329.330ms total=2329.634ms audio=20160
|
||||
[timing] call=1 tokens=28 final=0 token2mel=1874.519ms vocoder=372.070ms total=2246.599ms audio=24000
|
||||
[timing] call=2 tokens=12 final=1 token2mel=1887.511ms vocoder=220.482ms total=2108.008ms audio=15360
|
||||
Audio generation completed.
|
||||
llama_perf_context_print: load time = 25194.31 ms
|
||||
llama_perf_context_print: prompt eval time = 3065.64 ms / 182 tokens ( 16.84 ms per token, 59.37 tokens per second)
|
||||
llama_perf_context_print: eval time = 517.68 ms / 9 runs ( 57.52 ms per token, 17.39 tokens per second)
|
||||
llama_perf_context_print: total time = 33077.12 ms / 191 tokens
|
||||
llama_perf_context_print: graphs reused = 10
|
||||
Token2Wav (C++): session released
|
||||
tts thread end
|
||||
t2w thread end
|
||||
+483
@@ -0,0 +1,483 @@
|
||||
build: 64 (5202b7b) with x86_64-w64-mingw32-gcc-posix (GCC) 13-posix for x86_64-w64-mingw32
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 30720 MiB free
|
||||
llama_model_loader: loaded meta data with 27 key-value pairs and 399 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = qwen3
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Llm
|
||||
llama_model_loader: - kv 3: general.size_label str = 8.2B
|
||||
llama_model_loader: - kv 4: qwen3.block_count u32 = 36
|
||||
llama_model_loader: - kv 5: qwen3.context_length u32 = 40960
|
||||
llama_model_loader: - kv 6: qwen3.embedding_length u32 = 4096
|
||||
llama_model_loader: - kv 7: qwen3.feed_forward_length u32 = 12288
|
||||
llama_model_loader: - kv 8: qwen3.attention.head_count u32 = 32
|
||||
llama_model_loader: - kv 9: qwen3.attention.head_count_kv u32 = 8
|
||||
llama_model_loader: - kv 10: qwen3.rope.freq_base f32 = 1000000.000000
|
||||
llama_model_loader: - kv 11: qwen3.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: qwen3.attention.key_length u32 = 128
|
||||
llama_model_loader: - kv 13: qwen3.attention.value_length u32 = 128
|
||||
llama_model_loader: - kv 14: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 15: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 16: tokenizer.ggml.tokens arr[str,151748] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 17: tokenizer.ggml.token_type arr[i32,151748] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 24: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 25: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 26: general.file_type u32 = 15
|
||||
llama_model_loader: - type f32: 145 tensors
|
||||
llama_model_loader: - type q4_K: 217 tensors
|
||||
llama_model_loader: - type q6_K: 37 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = Q4_K - Medium
|
||||
print_info: file size = 4.68 GiB (4.90 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = qwen3
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 40960
|
||||
print_info: n_embd = 4096
|
||||
print_info: n_layer = 36
|
||||
print_info: n_head = 32
|
||||
print_info: n_head_kv = 8
|
||||
print_info: n_rot = 128
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 128
|
||||
print_info: n_embd_head_v = 128
|
||||
print_info: n_gqa = 4
|
||||
print_info: n_embd_k_gqa = 1024
|
||||
print_info: n_embd_v_gqa = 1024
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 12288
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = -1
|
||||
print_info: rope type = 2
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 1000000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 40960
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = 8B
|
||||
print_info: model params = 8.19 B
|
||||
print_info: general.name = Llm
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 151748
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 36 repeating layers to GPU
|
||||
load_tensors: offloading output layer to GPU
|
||||
load_tensors: offloaded 37/37 layers to GPU
|
||||
load_tensors: Vulkan0 model buffer size = 4454.74 MiB
|
||||
load_tensors: CPU_Mapped model buffer size = 333.43 MiB
|
||||
.....................................................................................
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 1000000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (40960) -- the full capacity of the model will not be utilized
|
||||
llama_context: Vulkan_Host output buffer size = 0.58 MiB
|
||||
llama_kv_cache: Vulkan0 KV buffer size = 288.00 MiB
|
||||
llama_kv_cache: size = 288.00 MiB ( 2048 cells, 36 layers, 1/1 seqs), K (f16): 144.00 MiB, V (f16): 144.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 304.38 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 12.01 MiB
|
||||
llama_context: graph nodes = 1267
|
||||
llama_context: graph splits = 2
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 25673 MiB free
|
||||
llama_model_loader: loaded meta data with 30 key-value pairs and 193 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = llama
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Tts
|
||||
llama_model_loader: - kv 3: general.size_label str = 445M
|
||||
llama_model_loader: - kv 4: llama.block_count u32 = 20
|
||||
llama_model_loader: - kv 5: llama.context_length u32 = 4096
|
||||
llama_model_loader: - kv 6: llama.embedding_length u32 = 768
|
||||
llama_model_loader: - kv 7: llama.feed_forward_length u32 = 3072
|
||||
llama_model_loader: - kv 8: llama.attention.head_count u32 = 12
|
||||
llama_model_loader: - kv 9: llama.attention.head_count_kv u32 = 12
|
||||
llama_model_loader: - kv 10: llama.rope.freq_base f32 = 10000.000000
|
||||
llama_model_loader: - kv 11: llama.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: llama.attention.key_length u32 = 64
|
||||
llama_model_loader: - kv 13: llama.attention.value_length u32 = 64
|
||||
llama_model_loader: - kv 14: general.file_type u32 = 1
|
||||
llama_model_loader: - kv 15: llama.vocab_size u32 = 158626
|
||||
llama_model_loader: - kv 16: llama.rope.dimension_count u32 = 64
|
||||
llama_model_loader: - kv 17: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.tokens arr[str,158626] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.token_type arr[i32,158626] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 24: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 25: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 26: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 27: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 28: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 29: tokenizer.ggml.add_space_prefix bool = false
|
||||
llama_model_loader: - type f32: 50 tensors
|
||||
llama_model_loader: - type f16: 143 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = F16
|
||||
print_info: file size = 1.07 GiB (20.70 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = llama
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 4096
|
||||
print_info: n_embd = 768
|
||||
print_info: n_layer = 20
|
||||
print_info: n_head = 12
|
||||
print_info: n_head_kv = 12
|
||||
print_info: n_rot = 64
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 64
|
||||
print_info: n_embd_head_v = 64
|
||||
print_info: n_gqa = 1
|
||||
print_info: n_embd_k_gqa = 768
|
||||
print_info: n_embd_v_gqa = 768
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 3072
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = 0
|
||||
print_info: rope type = 0
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 10000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 4096
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = ?B
|
||||
print_info: model params = 444.94 M
|
||||
print_info: general.name = Tts
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 158626
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
load_tensors: skipping TTS tensor emb_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor emb_text.weight (size = 467140608 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.weight (size = 12582912 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.weight (size = 2359296 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.weight (size = 6291456 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.weight (size = 1179648 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor head_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 0 repeating layers to GPU
|
||||
load_tensors: offloaded 0/21 layers to GPU
|
||||
load_tensors: CPU_Mapped model buffer size = 592.48 MiB
|
||||
..............................................................
|
||||
llama_init_from_model: model default pooling_type is [0], but [-1] was specified
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 10000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (4096) -- the full capacity of the model will not be utilized
|
||||
llama_context: CPU output buffer size = 0.61 MiB
|
||||
llama_kv_cache: CPU KV buffer size = 120.00 MiB
|
||||
llama_kv_cache: size = 120.00 MiB ( 2048 cells, 20 layers, 1/1 seqs), K (f16): 60.00 MiB, V (f16): 60.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 30.75 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 7.01 MiB
|
||||
llama_context: graph nodes = 613
|
||||
llama_context: graph splits = 216 (with bs=512), 1 (with bs=1)
|
||||
vision_model_loader: model name:
|
||||
vision_model_loader: description: image encoder for MiniCPM-V
|
||||
vision_model_loader: GGUF version: 3
|
||||
vision_model_loader: alignment: 32
|
||||
vision_model_loader: n_tensors: 455
|
||||
vision_model_loader: n_kv: 20
|
||||
|
||||
vision_ctx: vision using Vulkan0 backend
|
||||
load_hparams: model_type: 0
|
||||
load_hparams: n_embd: 1152
|
||||
load_hparams: n_head: 16
|
||||
load_hparams: n_ff: 4304
|
||||
load_hparams: n_layer: 27
|
||||
load_hparams: ffn_op: gelu
|
||||
load_hparams: projection_dim: 0
|
||||
|
||||
--- vision hparams ---
|
||||
load_hparams: image_size: 448
|
||||
load_hparams: patch_size: 14
|
||||
load_hparams: minicpmv_version: 100045
|
||||
load_hparams: proj_scale_factor: 0
|
||||
load_hparams: n_wa_pattern: 0
|
||||
|
||||
load_hparams: model size: 1044.36 MiB
|
||||
load_hparams: metadata size: 0.16 MiB
|
||||
load_tensors: ffn up/down are swapped
|
||||
alloc_compute_meta: Vulkan0 compute buffer size = 100.30 MiB
|
||||
alloc_compute_meta: CPU compute buffer size = 18.30 MiB
|
||||
init tts....init t2w....voc_hg2_model: CPU backend using 8 threads
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=32000
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=200, n_mel=80, total_size=16000
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=200, n_mel=80, total_elements=16000
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=200, n_tokens=100, current_iter=0, current_total=0, new_total=100, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 1 (total_tokens=100, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 20]
|
||||
audition_audio_batch_encode: Final output: 20 tokens x 4096 dims
|
||||
=== Model Paths ===
|
||||
Base dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf
|
||||
LLM: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf [OK]
|
||||
Vision: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/vision/MiniCPM-o-4_5-vision-F16.gguf [OK]
|
||||
Audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf [OK]
|
||||
TTS: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf [OK]
|
||||
Projector: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf [OK]
|
||||
===================
|
||||
=== Initializing Omni Context ===
|
||||
Media type: 2 (omni: audio+vision)
|
||||
TTS enabled: yes
|
||||
Context size: 2048
|
||||
GPU layers: 99
|
||||
Token2Wav device: cpu
|
||||
Vision backend: metal
|
||||
TTS bin dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts
|
||||
Ref audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\vllm-omni-edge\benchmarks\edge_harness\results\e2e_expansion_20260923\evidence\minicpmo_cpp_radeon890m_prompted\fixture_0000.wav
|
||||
02:47:11.339 === omni_init start
|
||||
02:47:11.340 media_type = 2, duplex_mode = 0, base_output_dir = ./tools/omni/output
|
||||
02:47:11.340 === omni_init: loading new LLM model
|
||||
02:48:05.289 === omni_init: loading TTS model
|
||||
02:48:05.289 TTS model: loading with n_gpu_layers=0
|
||||
02:48:09.023 TTS sampler: temp=0.80, top_p=0.85, top_k=25, rep_penalty=1.05
|
||||
02:48:09.023 TTS: loading weights from GGUF (emb_code, emb_text, projector_semantic, head_code)...
|
||||
02:48:14.521 TTS: head_code shape: dim0=6562, dim1=768, will transpose to [6562, 768]
|
||||
02:48:14.527 TTS: head_code loaded, verifying first few values:
|
||||
02:48:14.527 head_code_weight[0] = 0.01361084 (expect ~0.01385498)
|
||||
02:48:14.527 head_code_weight[1] = -0.01660156 (expect ~-0.01647949)
|
||||
02:48:14.527 head_code_weight[768] = 0.04101562 (this is weight[1, 0], expect ~0.04150391)
|
||||
02:48:14.576 TTS: weights loaded successfully
|
||||
02:48:14.577 Projector: loading from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf
|
||||
02:48:14.675 Projector: loaded successfully
|
||||
02:48:14.675 === omni_init: loading APM model
|
||||
02:48:17.727 APM: init from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf
|
||||
02:48:23.029 Token2Wav: found models in \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf
|
||||
02:48:23.085 Token2Wav: all model files found, initializing session...
|
||||
02:48:23.085 Token2Wav: non-CUDA backend, vocoder using CPU for better performance
|
||||
02:48:23.086 Token2Wav: using prompt_cache from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf/prompt_cache.gguf
|
||||
02:48:27.666 Token2Wav: initialized successfully
|
||||
02:48:27.666 Python T2W: script_dir=./tools/omni/pyt2w, model_dir=./tools/omni/pyt2w/token2wav
|
||||
02:48:27.666 Token2Wav: 浣跨敤 C++ 瀹炵幇
|
||||
02:48:27.666 === omni_init success: ctx_llama = 00000177e494f830
|
||||
=== Running test case ===
|
||||
Audio prefix: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-14-vongzq47\fixture_
|
||||
Count: 2
|
||||
02:48:27.666 stream_prefill: n_past = 0
|
||||
voice_clone_prompt = <|im_start|>system
|
||||
妯′豢闊抽�鏍锋湰鐨勯煶鑹插苟鐢熸垚鏂扮殑鍐呭�銆�
|
||||
<|audio_start|>
|
||||
assistant_prompt = <|audio_end|>浣犵殑浠诲姟鏄�敤杩欑�澹伴煶妯″紡鏉ュ綋涓€涓�姪鎵嬨€傝�璁ょ湡銆侀珮璐ㄩ噺鍦板洖澶嶇敤鎴风殑闂��銆傝�鐢ㄩ珮鑷�劧搴︾殑鏂瑰紡鍜岀敤鎴疯亰澶┿€�<|im_end|>
|
||||
<|im_start|>user
|
||||
|
||||
02:48:27.666 system prompt ref_audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-14-vongzq47\fixture_0000.wav
|
||||
02:48:28.231 system prompt ref_audio embedding: n_pos=20
|
||||
02:48:28.732 馃敀 n_keep 璁剧疆涓� 72 (system prompt tokens)锛岃繖閮ㄥ垎姘歌繙涓嶄細琚�粦鍔ㄧ獥鍙ｅ垹闄�
|
||||
02:48:28.732 stream_prefill(index=0): system prompt 鍒濆�鍖栧畬鎴愶紝ref_audio 宸插湪鍏朵腑 prefill
|
||||
02:48:28.732 [SW] system_prompt registered: preserve_length=72 (will be protected from sliding)
|
||||
02:48:28.732 n_past = 72
|
||||
02:48:28.732
|
||||
|
||||
c++ finish stream_prefill(index=0). n_past=72, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 0 (audio) : 1.0657 s
|
||||
encode_image_with_vision_chunks: step 1 of 1 encoded in 616.77 ms
|
||||
encode_image_with_vision_chunks: all 1 chunks encoded in 635.26 ms (grid: 1x1)
|
||||
|
||||
encode_image_with_vision_chunks: image encoded in 635.27 ms by vision ( 635.27 ms per chunk, 64 total tokens)
|
||||
omni_image_embed_make_chunks_with_filename: created 1 vision chunks from C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-14-vongzq47\fixture_0001.jpg
|
||||
stream_prefill: prefilled 1 vision chunks (64 tokens each)
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=49339
|
||||
audition_audio_preprocess: Audio not aligned to 100ms (49339 samples = 3083.7ms), padded with silence to 49600 samples (3100ms)
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=310, n_mel=80, total_size=24800
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=310, n_mel=80, total_elements=24800
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=310, n_tokens=155, current_iter=1, current_total=155, new_total=310, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 2 (total_tokens=310, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 31]
|
||||
audition_audio_batch_encode: Final output: 31 tokens x 4096 dims
|
||||
02:48:29.892 stream_prefill(index=1): processing user audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-14-vongzq47\fixture_0001.wav
|
||||
02:48:30.173 stream_prefill(index=1): user audio embedding: n_pos=31
|
||||
02:48:30.485
|
||||
|
||||
c++ finish stream_prefill(index=1). n_past=172, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 1 (audio+vision) : 1.75167 s
|
||||
02:48:30.485 馃搷 stream_decode 寮€濮�: n_past=172, n_keep=72, n_ctx=2048, duplex_mode=0
|
||||
02:48:30.486 stream_decode: create tts thread (simplex mode)
|
||||
02:48:30.486 stream_decode: create t2w thread
|
||||
02:48:30.486 wait prefill done
|
||||
02:48:30.486 馃搷 [鍗曞伐TTS] 娣诲姞 assistant prompt: "<|im_end|>
|
||||
<|im_start|>assistant
|
||||
<think>
|
||||
|
||||
</think>
|
||||
|
||||
<|tts_bos|>", n_past=172
|
||||
02:48:30.486 T2W thread (C++) started
|
||||
<user>
|
||||
<assistant>02:48:30.489 TTS: 鍒涘缓鍗曞伐妯″紡杈撳嚭鐩�綍: ./tools/omni/output/round_000
|
||||
02:48:30.490 TTS thread started
|
||||
02:48:30.652 馃搷 [鍗曞伐TTS] assistant prompt 瀹屾垚, n_past=182
|
||||
02:48:30.652 LLM decode: max_tgt_len = 96, n_predict = 96, n_ctx = 2048
|
||||
02:48:31.174 LLM: detected end token, set llm_generation_done=true
|
||||
02:48:31.174 LLM->TTS: text='The square in the image is red.', n_tokens=8, hidden_size=32768, n_embd=4096, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:48:31.174 馃搷 Decode 缁撴潫: n_past=191, 鍓╀綑绌洪棿=1857, 鏃犻渶婊戠獥
|
||||
02:48:31.174 馃搷 杞�� 1 缁撴潫锛岃�褰曡竟鐣屼簬 n_past=191
|
||||
02:48:31.174 TTS<-LLM: chunk_idx=0, text='The square in the image is red.', n_tokens=8, hidden_size=32768, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:48:31.174 TTS: after queue pop - speek_done=0, llm_finish=1, llm_text.empty=0, token_ids.size=8
|
||||
02:48:31.174 TTS: before empty check - response.empty=0, response='The square in the image is red.', llm_finish=1
|
||||
Waiting for audio generation to complete...
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_0.wav does not exist or is empty
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_1.wav does not exist or is empty
|
||||
TTS: no valid WAV files to merge
|
||||
02:48:31.174 TTS: DEBUG before has_llm_data - token_ids.size=8, hidden_states.size=32768, n_embd=4096
|
||||
02:48:31.174 TTS: has_llm_data=1, token_ids=8, hidden_states=32768, n_embd=4096, llm_finish=1
|
||||
02:48:31.174 TTS: n_tokens_orig=8, n_tokens_filtered=8 (filtered 0 special tokens)
|
||||
02:48:31.214 TTS: saved merged embeddings (text) to ./tools/omni/output/round_000/llm_debug/chunk_0/merged_embeddings.txt
|
||||
02:48:31.214 TTS Simplex: generating audio tokens for chunk 0 (n_tokens=8, tts_n_embd=768)
|
||||
02:48:31.214 TTS Simplex: 鍦� prefill 鍓嶆坊鍔� audio_bos (chunk_idx=0, new_size=9)
|
||||
02:48:31.214 TTS Simplex: first chunk - cleared KV cache
|
||||
02:48:31.214 TTS Simplex: first chunk - cleared tts_all_generated_tokens and tts_token_buffer
|
||||
02:48:31.266 馃搷 涓轰笅涓€杞�噯澶�: eval <|im_end|>\n<|im_start|>user\n, n_past=196
|
||||
02:48:31.476 TTS Simplex: prefill completed, n_past_tts=9
|
||||
02:48:31.476 TTS Simplex: sampler created
|
||||
02:48:31.476 TTS simplex: is_audio_bos=true (first audio token)
|
||||
02:48:31.501 TTS Simplex Phase1: token 1/500: rel_id=1487
|
||||
02:48:31.512 TTS Simplex Phase1: token 2/500: rel_id=4218
|
||||
02:48:31.523 TTS Simplex Phase1: token 3/500: rel_id=4299
|
||||
02:48:31.533 TTS Simplex Phase1: token 4/500: rel_id=4218
|
||||
02:48:31.544 TTS Simplex Phase1: token 5/500: rel_id=4218
|
||||
02:48:31.760 TTS Simplex Phase1: yield 25 tokens 鍒� T2W (step 25, buffer=25)
|
||||
02:48:31.760 TTS Simplex Phase1: token 25/500: rel_id=1295
|
||||
02:48:31.760 T2W绾跨▼(C++): 杞��鍒囨崲 (-1 -> 0)锛屾洿鏂拌緭鍑虹洰褰�
|
||||
02:48:31.760 T2W绾跨▼(C++): 鏂拌緭鍑虹洰褰� ./tools/omni/output/round_000/tts_wav
|
||||
02:48:31.873 TTS Simplex Phase1: EOS token at step 36
|
||||
02:48:31.873 TTS Simplex: is_final_text_chunk=true, will enter Phase 2 for text_eos_embed
|
||||
02:48:31.873 TTS Simplex Phase2: injecting text_eos_embed + audio_bos at n_past=44
|
||||
02:48:31.881 TTS Simplex Phase2: text_eos_embed injected, n_past=45
|
||||
02:48:31.889 TTS Simplex Phase2: audio_bos injected, n_past=46, starting final generation
|
||||
02:48:32.052 TTS Simplex Phase2: yield 25 tokens 鍒� T2W
|
||||
02:48:32.120 TTS Simplex Phase2: EOS at step 21 (final end)
|
||||
02:48:32.120 TTS Simplex: chunk 缁撴潫锛宼ts_token_buffer 鍓╀綑 5 tokens (淇濈暀绛変笅涓€涓� chunk)
|
||||
02:48:32.120 TTS Simplex: is_final_text_chunk=true, flush 鍓╀綑 5 tokens 浠� tts_token_buffer
|
||||
02:48:32.120 TTS Simplex: generated 55 audio tokens for chunk 0
|
||||
02:48:32.120 TTS Simplex: chunk 0 first tokens: [1487 4218 4299 4218 4218 ], last tokens: [1275 759 4299 4299 6082 ]
|
||||
02:48:32.120 TTS Simplex: updated tts_n_past_accumulated=67, total_generated_tokens=55
|
||||
02:48:32.121 TTS: finished processing all chunks
|
||||
02:48:32.121 TTS: 鍗曞伐妯″紡杞��缁撴潫锛屼笅涓€杞� round_idx=1
|
||||
02:48:32.121 TTS: sent is_final=true to T2W queue (turn end)
|
||||
02:48:34.112 馃帀 棣栧搷鏃堕棿 (First Audio Response): 3626ms
|
||||
02:48:34.112 T2W绾跨▼: wav_0.wav | 0.84s audio | 2350.4ms inference | RTF=2.80 | t=3626ms
|
||||
02:48:36.363 T2W绾跨▼: wav_1.wav | 1.00s audio | 2250.6ms inference | RTF=2.25 | t=5878ms
|
||||
02:48:38.406 T2W绾跨▼: wav_2.wav | 0.48s audio | 2042.0ms inference | RTF=4.25 | t=7920ms
|
||||
02:48:38.407 T2W绾跨▼: 鍐欏叆缁撴潫鏍囪� ./tools/omni/output/round_000/tts_wav/generation_done.flag (last_wav=2)
|
||||
02:48:38.407 T2W绾跨▼: 杞��缁撴潫鍚庢洿鏂拌緭鍑虹洰褰曚负 ./tools/omni/output/round_001/tts_wav
|
||||
02:48:38.413 omni_stop_threads: stop signals sent
|
||||
02:48:38.413 T2W(C++) 绾跨▼: 鍋滄�
|
||||
[timing] call=0 tokens=28 final=0 token2mel=2014.984ms vocoder=335.371ms total=2350.370ms audio=20160
|
||||
[timing] call=1 tokens=28 final=0 token2mel=1862.150ms vocoder=388.435ms total=2250.596ms audio=24000
|
||||
[timing] call=2 tokens=8 final=1 token2mel=1865.175ms vocoder=176.826ms total=2042.011ms audio=11520
|
||||
Audio generation completed.
|
||||
llama_perf_context_print: load time = 25236.50 ms
|
||||
llama_perf_context_print: prompt eval time = 2984.20 ms / 182 tokens ( 16.40 ms per token, 60.99 tokens per second)
|
||||
llama_perf_context_print: eval time = 519.49 ms / 9 runs ( 57.72 ms per token, 17.32 tokens per second)
|
||||
llama_perf_context_print: total time = 32997.92 ms / 191 tokens
|
||||
llama_perf_context_print: graphs reused = 10
|
||||
Token2Wav (C++): session released
|
||||
tts thread end
|
||||
t2w thread end
|
||||
+484
@@ -0,0 +1,484 @@
|
||||
build: 64 (5202b7b) with x86_64-w64-mingw32-gcc-posix (GCC) 13-posix for x86_64-w64-mingw32
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 30720 MiB free
|
||||
llama_model_loader: loaded meta data with 27 key-value pairs and 399 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = qwen3
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Llm
|
||||
llama_model_loader: - kv 3: general.size_label str = 8.2B
|
||||
llama_model_loader: - kv 4: qwen3.block_count u32 = 36
|
||||
llama_model_loader: - kv 5: qwen3.context_length u32 = 40960
|
||||
llama_model_loader: - kv 6: qwen3.embedding_length u32 = 4096
|
||||
llama_model_loader: - kv 7: qwen3.feed_forward_length u32 = 12288
|
||||
llama_model_loader: - kv 8: qwen3.attention.head_count u32 = 32
|
||||
llama_model_loader: - kv 9: qwen3.attention.head_count_kv u32 = 8
|
||||
llama_model_loader: - kv 10: qwen3.rope.freq_base f32 = 1000000.000000
|
||||
llama_model_loader: - kv 11: qwen3.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: qwen3.attention.key_length u32 = 128
|
||||
llama_model_loader: - kv 13: qwen3.attention.value_length u32 = 128
|
||||
llama_model_loader: - kv 14: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 15: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 16: tokenizer.ggml.tokens arr[str,151748] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 17: tokenizer.ggml.token_type arr[i32,151748] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 24: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 25: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 26: general.file_type u32 = 15
|
||||
llama_model_loader: - type f32: 145 tensors
|
||||
llama_model_loader: - type q4_K: 217 tensors
|
||||
llama_model_loader: - type q6_K: 37 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = Q4_K - Medium
|
||||
print_info: file size = 4.68 GiB (4.90 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = qwen3
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 40960
|
||||
print_info: n_embd = 4096
|
||||
print_info: n_layer = 36
|
||||
print_info: n_head = 32
|
||||
print_info: n_head_kv = 8
|
||||
print_info: n_rot = 128
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 128
|
||||
print_info: n_embd_head_v = 128
|
||||
print_info: n_gqa = 4
|
||||
print_info: n_embd_k_gqa = 1024
|
||||
print_info: n_embd_v_gqa = 1024
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 12288
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = -1
|
||||
print_info: rope type = 2
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 1000000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 40960
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = 8B
|
||||
print_info: model params = 8.19 B
|
||||
print_info: general.name = Llm
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 151748
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 36 repeating layers to GPU
|
||||
load_tensors: offloading output layer to GPU
|
||||
load_tensors: offloaded 37/37 layers to GPU
|
||||
load_tensors: Vulkan0 model buffer size = 4454.74 MiB
|
||||
load_tensors: CPU_Mapped model buffer size = 333.43 MiB
|
||||
.....................................................................................
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 1000000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (40960) -- the full capacity of the model will not be utilized
|
||||
llama_context: Vulkan_Host output buffer size = 0.58 MiB
|
||||
llama_kv_cache: Vulkan0 KV buffer size = 288.00 MiB
|
||||
llama_kv_cache: size = 288.00 MiB ( 2048 cells, 36 layers, 1/1 seqs), K (f16): 144.00 MiB, V (f16): 144.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 304.38 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 12.01 MiB
|
||||
llama_context: graph nodes = 1267
|
||||
llama_context: graph splits = 2
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 25673 MiB free
|
||||
llama_model_loader: loaded meta data with 30 key-value pairs and 193 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = llama
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Tts
|
||||
llama_model_loader: - kv 3: general.size_label str = 445M
|
||||
llama_model_loader: - kv 4: llama.block_count u32 = 20
|
||||
llama_model_loader: - kv 5: llama.context_length u32 = 4096
|
||||
llama_model_loader: - kv 6: llama.embedding_length u32 = 768
|
||||
llama_model_loader: - kv 7: llama.feed_forward_length u32 = 3072
|
||||
llama_model_loader: - kv 8: llama.attention.head_count u32 = 12
|
||||
llama_model_loader: - kv 9: llama.attention.head_count_kv u32 = 12
|
||||
llama_model_loader: - kv 10: llama.rope.freq_base f32 = 10000.000000
|
||||
llama_model_loader: - kv 11: llama.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: llama.attention.key_length u32 = 64
|
||||
llama_model_loader: - kv 13: llama.attention.value_length u32 = 64
|
||||
llama_model_loader: - kv 14: general.file_type u32 = 1
|
||||
llama_model_loader: - kv 15: llama.vocab_size u32 = 158626
|
||||
llama_model_loader: - kv 16: llama.rope.dimension_count u32 = 64
|
||||
llama_model_loader: - kv 17: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.tokens arr[str,158626] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.token_type arr[i32,158626] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 24: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 25: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 26: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 27: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 28: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 29: tokenizer.ggml.add_space_prefix bool = false
|
||||
llama_model_loader: - type f32: 50 tensors
|
||||
llama_model_loader: - type f16: 143 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = F16
|
||||
print_info: file size = 1.07 GiB (20.70 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = llama
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 4096
|
||||
print_info: n_embd = 768
|
||||
print_info: n_layer = 20
|
||||
print_info: n_head = 12
|
||||
print_info: n_head_kv = 12
|
||||
print_info: n_rot = 64
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 64
|
||||
print_info: n_embd_head_v = 64
|
||||
print_info: n_gqa = 1
|
||||
print_info: n_embd_k_gqa = 768
|
||||
print_info: n_embd_v_gqa = 768
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 3072
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = 0
|
||||
print_info: rope type = 0
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 10000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 4096
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = ?B
|
||||
print_info: model params = 444.94 M
|
||||
print_info: general.name = Tts
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 158626
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
load_tensors: skipping TTS tensor emb_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor emb_text.weight (size = 467140608 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.weight (size = 12582912 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.weight (size = 2359296 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.weight (size = 6291456 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.weight (size = 1179648 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor head_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 0 repeating layers to GPU
|
||||
load_tensors: offloaded 0/21 layers to GPU
|
||||
load_tensors: CPU_Mapped model buffer size = 592.48 MiB
|
||||
..............................................................
|
||||
llama_init_from_model: model default pooling_type is [0], but [-1] was specified
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 10000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (4096) -- the full capacity of the model will not be utilized
|
||||
llama_context: CPU output buffer size = 0.61 MiB
|
||||
llama_kv_cache: CPU KV buffer size = 120.00 MiB
|
||||
llama_kv_cache: size = 120.00 MiB ( 2048 cells, 20 layers, 1/1 seqs), K (f16): 60.00 MiB, V (f16): 60.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 30.75 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 7.01 MiB
|
||||
llama_context: graph nodes = 613
|
||||
llama_context: graph splits = 216 (with bs=512), 1 (with bs=1)
|
||||
vision_model_loader: model name:
|
||||
vision_model_loader: description: image encoder for MiniCPM-V
|
||||
vision_model_loader: GGUF version: 3
|
||||
vision_model_loader: alignment: 32
|
||||
vision_model_loader: n_tensors: 455
|
||||
vision_model_loader: n_kv: 20
|
||||
|
||||
vision_ctx: vision using Vulkan0 backend
|
||||
load_hparams: model_type: 0
|
||||
load_hparams: n_embd: 1152
|
||||
load_hparams: n_head: 16
|
||||
load_hparams: n_ff: 4304
|
||||
load_hparams: n_layer: 27
|
||||
load_hparams: ffn_op: gelu
|
||||
load_hparams: projection_dim: 0
|
||||
|
||||
--- vision hparams ---
|
||||
load_hparams: image_size: 448
|
||||
load_hparams: patch_size: 14
|
||||
load_hparams: minicpmv_version: 100045
|
||||
load_hparams: proj_scale_factor: 0
|
||||
load_hparams: n_wa_pattern: 0
|
||||
|
||||
load_hparams: model size: 1044.36 MiB
|
||||
load_hparams: metadata size: 0.16 MiB
|
||||
load_tensors: ffn up/down are swapped
|
||||
alloc_compute_meta: Vulkan0 compute buffer size = 100.30 MiB
|
||||
alloc_compute_meta: CPU compute buffer size = 18.30 MiB
|
||||
init tts....init t2w....voc_hg2_model: CPU backend using 8 threads
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=32000
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=200, n_mel=80, total_size=16000
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=200, n_mel=80, total_elements=16000
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=200, n_tokens=100, current_iter=0, current_total=0, new_total=100, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 1 (total_tokens=100, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 20]
|
||||
audition_audio_batch_encode: Final output: 20 tokens x 4096 dims
|
||||
=== Model Paths ===
|
||||
Base dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf
|
||||
LLM: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf [OK]
|
||||
Vision: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/vision/MiniCPM-o-4_5-vision-F16.gguf [OK]
|
||||
Audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf [OK]
|
||||
TTS: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf [OK]
|
||||
Projector: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf [OK]
|
||||
===================
|
||||
=== Initializing Omni Context ===
|
||||
Media type: 2 (omni: audio+vision)
|
||||
TTS enabled: yes
|
||||
Context size: 2048
|
||||
GPU layers: 99
|
||||
Token2Wav device: cpu
|
||||
Vision backend: metal
|
||||
TTS bin dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts
|
||||
Ref audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\vllm-omni-edge\benchmarks\edge_harness\results\e2e_expansion_20260923\evidence\minicpmo_cpp_radeon890m_prompted\fixture_0000.wav
|
||||
02:48:40.543 === omni_init start
|
||||
02:48:40.543 media_type = 2, duplex_mode = 0, base_output_dir = ./tools/omni/output
|
||||
02:48:40.543 === omni_init: loading new LLM model
|
||||
02:49:31.091 === omni_init: loading TTS model
|
||||
02:49:31.091 TTS model: loading with n_gpu_layers=0
|
||||
02:49:34.886 TTS sampler: temp=0.80, top_p=0.85, top_k=25, rep_penalty=1.05
|
||||
02:49:34.886 TTS: loading weights from GGUF (emb_code, emb_text, projector_semantic, head_code)...
|
||||
02:49:40.489 TTS: head_code shape: dim0=6562, dim1=768, will transpose to [6562, 768]
|
||||
02:49:40.495 TTS: head_code loaded, verifying first few values:
|
||||
02:49:40.495 head_code_weight[0] = 0.01361084 (expect ~0.01385498)
|
||||
02:49:40.495 head_code_weight[1] = -0.01660156 (expect ~-0.01647949)
|
||||
02:49:40.495 head_code_weight[768] = 0.04101562 (this is weight[1, 0], expect ~0.04150391)
|
||||
02:49:40.545 TTS: weights loaded successfully
|
||||
02:49:40.545 Projector: loading from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf
|
||||
02:49:40.648 Projector: loaded successfully
|
||||
02:49:40.648 === omni_init: loading APM model
|
||||
02:49:43.804 APM: init from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf
|
||||
02:49:49.272 Token2Wav: found models in \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf
|
||||
02:49:49.324 Token2Wav: all model files found, initializing session...
|
||||
02:49:49.324 Token2Wav: non-CUDA backend, vocoder using CPU for better performance
|
||||
02:49:49.324 Token2Wav: using prompt_cache from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf/prompt_cache.gguf
|
||||
02:49:53.903 Token2Wav: initialized successfully
|
||||
02:49:53.903 Python T2W: script_dir=./tools/omni/pyt2w, model_dir=./tools/omni/pyt2w/token2wav
|
||||
02:49:53.903 Token2Wav: 浣跨敤 C++ 瀹炵幇
|
||||
02:49:53.903 === omni_init success: ctx_llama = 000001aafd49f830
|
||||
=== Running test case ===
|
||||
Audio prefix: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-15-og1p931n\fixture_
|
||||
Count: 2
|
||||
02:49:53.903 stream_prefill: n_past = 0
|
||||
voice_clone_prompt = <|im_start|>system
|
||||
妯′豢闊抽�鏍锋湰鐨勯煶鑹插苟鐢熸垚鏂扮殑鍐呭�銆�
|
||||
<|audio_start|>
|
||||
assistant_prompt = <|audio_end|>浣犵殑浠诲姟鏄�敤杩欑�澹伴煶妯″紡鏉ュ綋涓€涓�姪鎵嬨€傝�璁ょ湡銆侀珮璐ㄩ噺鍦板洖澶嶇敤鎴风殑闂��銆傝�鐢ㄩ珮鑷�劧搴︾殑鏂瑰紡鍜岀敤鎴疯亰澶┿€�<|im_end|>
|
||||
<|im_start|>user
|
||||
|
||||
02:49:53.903 system prompt ref_audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-15-og1p931n\fixture_0000.wav
|
||||
02:49:54.490 system prompt ref_audio embedding: n_pos=20
|
||||
02:49:54.988 馃敀 n_keep 璁剧疆涓� 72 (system prompt tokens)锛岃繖閮ㄥ垎姘歌繙涓嶄細琚�粦鍔ㄧ獥鍙ｅ垹闄�
|
||||
02:49:54.988 stream_prefill(index=0): system prompt 鍒濆�鍖栧畬鎴愶紝ref_audio 宸插湪鍏朵腑 prefill
|
||||
02:49:54.988 [SW] system_prompt registered: preserve_length=72 (will be protected from sliding)
|
||||
02:49:54.988 n_past = 72
|
||||
02:49:54.988
|
||||
|
||||
c++ finish stream_prefill(index=0). n_past=72, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 0 (audio) : 1.0853 s
|
||||
encode_image_with_vision_chunks: step 1 of 1 encoded in 630.00 ms
|
||||
encode_image_with_vision_chunks: all 1 chunks encoded in 648.42 ms (grid: 1x1)
|
||||
|
||||
encode_image_with_vision_chunks: image encoded in 648.42 ms by vision ( 648.42 ms per chunk, 64 total tokens)
|
||||
omni_image_embed_make_chunks_with_filename: created 1 vision chunks from C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-15-og1p931n\fixture_0001.jpg
|
||||
stream_prefill: prefilled 1 vision chunks (64 tokens each)
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=49339
|
||||
audition_audio_preprocess: Audio not aligned to 100ms (49339 samples = 3083.7ms), padded with silence to 49600 samples (3100ms)
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=310, n_mel=80, total_size=24800
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=310, n_mel=80, total_elements=24800
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=310, n_tokens=155, current_iter=1, current_total=155, new_total=310, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 2 (total_tokens=310, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 31]
|
||||
audition_audio_batch_encode: Final output: 31 tokens x 4096 dims
|
||||
02:49:56.158 stream_prefill(index=1): processing user audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-15-og1p931n\fixture_0001.wav
|
||||
02:49:56.458 stream_prefill(index=1): user audio embedding: n_pos=31
|
||||
02:49:56.771
|
||||
|
||||
c++ finish stream_prefill(index=1). n_past=172, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 1 (audio+vision) : 1.78043 s
|
||||
02:49:56.771 馃搷 stream_decode 寮€濮�: n_past=172, n_keep=72, n_ctx=2048, duplex_mode=0
|
||||
02:49:56.771 stream_decode: create tts thread (simplex mode)
|
||||
02:49:56.771 stream_decode: create t2w thread
|
||||
02:49:56.771 wait prefill done
|
||||
02:49:56.771 馃搷 [鍗曞伐TTS] 娣诲姞 assistant prompt: "<|im_end|>
|
||||
<|im_start|>assistant
|
||||
<think>
|
||||
|
||||
</think>
|
||||
|
||||
<|tts_bos|>", n_past=172
|
||||
02:49:56.771 T2W thread (C++) started
|
||||
<user>
|
||||
<assistant>02:49:56.775 TTS: 鍒涘缓鍗曞伐妯″紡杈撳嚭鐩�綍: ./tools/omni/output/round_000
|
||||
02:49:56.776 TTS thread started
|
||||
02:49:56.932 馃搷 [鍗曞伐TTS] assistant prompt 瀹屾垚, n_past=182
|
||||
02:49:56.932 LLM decode: max_tgt_len = 96, n_predict = 96, n_ctx = 2048
|
||||
02:49:57.449 LLM: detected end token, set llm_generation_done=true
|
||||
02:49:57.449 LLM->TTS: text='The square in the image is red.', n_tokens=8, hidden_size=32768, n_embd=4096, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:49:57.449 馃搷 Decode 缁撴潫: n_past=191, 鍓╀綑绌洪棿=1857, 鏃犻渶婊戠獥
|
||||
02:49:57.449 馃搷 杞�� 1 缁撴潫锛岃�褰曡竟鐣屼簬 n_past=191
|
||||
02:49:57.449 TTS<-LLM: chunk_idx=0, text='The square in the image is red.', n_tokens=8, hidden_size=32768, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:49:57.449 TTS: after queue pop - speek_done=0, llm_finish=1, llm_text.empty=0, token_ids.size=8
|
||||
02:49:57.449 TTS: before empty check - response.empty=0, response='The square in the image is red.', llm_finish=1
|
||||
Waiting for audio generation to complete...
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_0.wav does not exist or is empty
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_1.wav does not exist or is empty
|
||||
TTS: no valid WAV files to merge
|
||||
02:49:57.449 TTS: DEBUG before has_llm_data - token_ids.size=8, hidden_states.size=32768, n_embd=4096
|
||||
02:49:57.449 TTS: has_llm_data=1, token_ids=8, hidden_states=32768, n_embd=4096, llm_finish=1
|
||||
02:49:57.450 TTS: n_tokens_orig=8, n_tokens_filtered=8 (filtered 0 special tokens)
|
||||
02:49:57.488 TTS: saved merged embeddings (text) to ./tools/omni/output/round_000/llm_debug/chunk_0/merged_embeddings.txt
|
||||
02:49:57.488 TTS Simplex: generating audio tokens for chunk 0 (n_tokens=8, tts_n_embd=768)
|
||||
02:49:57.488 TTS Simplex: 鍦� prefill 鍓嶆坊鍔� audio_bos (chunk_idx=0, new_size=9)
|
||||
02:49:57.488 TTS Simplex: first chunk - cleared KV cache
|
||||
02:49:57.488 TTS Simplex: first chunk - cleared tts_all_generated_tokens and tts_token_buffer
|
||||
02:49:57.541 馃搷 涓轰笅涓€杞�噯澶�: eval <|im_end|>\n<|im_start|>user\n, n_past=196
|
||||
02:49:57.744 TTS Simplex: prefill completed, n_past_tts=9
|
||||
02:49:57.744 TTS Simplex: sampler created
|
||||
02:49:57.744 TTS simplex: is_audio_bos=true (first audio token)
|
||||
02:49:57.769 TTS Simplex Phase1: token 1/500: rel_id=1460
|
||||
02:49:57.779 TTS Simplex Phase1: token 2/500: rel_id=4218
|
||||
02:49:57.790 TTS Simplex Phase1: token 3/500: rel_id=4218
|
||||
02:49:57.801 TTS Simplex Phase1: token 4/500: rel_id=4218
|
||||
02:49:57.811 TTS Simplex Phase1: token 5/500: rel_id=4218
|
||||
02:49:58.033 TTS Simplex Phase1: yield 25 tokens 鍒� T2W (step 25, buffer=25)
|
||||
02:49:58.033 T2W绾跨▼(C++): 杞��鍒囨崲 (-1 -> 0)锛屾洿鏂拌緭鍑虹洰褰�
|
||||
02:49:58.033 TTS Simplex Phase1: token 25/500: rel_id=3906
|
||||
02:49:58.033 T2W绾跨▼(C++): 鏂拌緭鍑虹洰褰� ./tools/omni/output/round_000/tts_wav
|
||||
02:49:58.306 TTS Simplex Phase1: yield 25 tokens 鍒� T2W (step 50, buffer=25)
|
||||
02:49:58.306 TTS Simplex Phase1: token 50/500: rel_id=6026
|
||||
02:49:58.432 TTS Simplex Phase1: EOS token at step 62
|
||||
02:49:58.432 TTS Simplex: is_final_text_chunk=true, will enter Phase 2 for text_eos_embed
|
||||
02:49:58.432 TTS Simplex Phase2: injecting text_eos_embed + audio_bos at n_past=70
|
||||
02:49:58.439 TTS Simplex Phase2: text_eos_embed injected, n_past=71
|
||||
02:49:58.447 TTS Simplex Phase2: audio_bos injected, n_past=72, starting final generation
|
||||
02:49:58.457 TTS Simplex Phase2: EOS at step 1 (final end)
|
||||
02:49:58.458 TTS Simplex: chunk 缁撴潫锛宼ts_token_buffer 鍓╀綑 11 tokens (淇濈暀绛変笅涓€涓� chunk)
|
||||
02:49:58.458 TTS Simplex: is_final_text_chunk=true, flush 鍓╀綑 11 tokens 浠� tts_token_buffer
|
||||
02:49:58.458 TTS Simplex: generated 61 audio tokens for chunk 0
|
||||
02:49:58.458 TTS Simplex: chunk 0 first tokens: [1460 4218 4218 4218 4218 ], last tokens: [733 1461 4218 4218 6082 ]
|
||||
02:49:58.458 TTS Simplex: updated tts_n_past_accumulated=73, total_generated_tokens=61
|
||||
02:49:58.468 TTS: finished processing all chunks
|
||||
02:49:58.468 TTS: 鍗曞伐妯″紡杞��缁撴潫锛屼笅涓€杞� round_idx=1
|
||||
02:49:58.468 TTS: sent is_final=true to T2W queue (turn end)
|
||||
02:50:00.359 馃帀 棣栧搷鏃堕棿 (First Audio Response): 3587ms
|
||||
02:50:00.359 T2W绾跨▼: wav_0.wav | 0.84s audio | 2324.6ms inference | RTF=2.77 | t=3587ms
|
||||
02:50:02.581 T2W绾跨▼: wav_1.wav | 1.00s audio | 2221.6ms inference | RTF=2.22 | t=5810ms
|
||||
02:50:04.677 T2W绾跨▼: wav_2.wav | 0.72s audio | 2095.5ms inference | RTF=2.91 | t=7906ms
|
||||
02:50:04.678 T2W绾跨▼: 鍐欏叆缁撴潫鏍囪� ./tools/omni/output/round_000/tts_wav/generation_done.flag (last_wav=2)
|
||||
02:50:04.678 T2W绾跨▼: 杞��缁撴潫鍚庢洿鏂拌緭鍑虹洰褰曚负 ./tools/omni/output/round_001/tts_wav
|
||||
02:50:04.732 omni_stop_threads: stop signals sent
|
||||
02:50:04.732 T2W(C++) 绾跨▼: 鍋滄�
|
||||
[timing] call=0 tokens=28 final=0 token2mel=1988.256ms vocoder=336.333ms total=2324.610ms audio=20160
|
||||
[timing] call=1 tokens=28 final=0 token2mel=1841.697ms vocoder=379.857ms total=2221.565ms audio=24000
|
||||
[timing] call=2 tokens=14 final=1 token2mel=1856.846ms vocoder=238.637ms total=2095.494ms audio=17280
|
||||
Audio generation completed.
|
||||
llama_perf_context_print: load time = 25720.03 ms
|
||||
llama_perf_context_print: prompt eval time = 3028.03 ms / 182 tokens ( 16.64 ms per token, 60.11 tokens per second)
|
||||
llama_perf_context_print: eval time = 513.77 ms / 9 runs ( 57.09 ms per token, 17.52 tokens per second)
|
||||
llama_perf_context_print: total time = 33519.64 ms / 191 tokens
|
||||
llama_perf_context_print: graphs reused = 10
|
||||
Token2Wav (C++): session released
|
||||
tts thread end
|
||||
t2w thread end
|
||||
+483
@@ -0,0 +1,483 @@
|
||||
build: 64 (5202b7b) with x86_64-w64-mingw32-gcc-posix (GCC) 13-posix for x86_64-w64-mingw32
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 30720 MiB free
|
||||
llama_model_loader: loaded meta data with 27 key-value pairs and 399 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = qwen3
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Llm
|
||||
llama_model_loader: - kv 3: general.size_label str = 8.2B
|
||||
llama_model_loader: - kv 4: qwen3.block_count u32 = 36
|
||||
llama_model_loader: - kv 5: qwen3.context_length u32 = 40960
|
||||
llama_model_loader: - kv 6: qwen3.embedding_length u32 = 4096
|
||||
llama_model_loader: - kv 7: qwen3.feed_forward_length u32 = 12288
|
||||
llama_model_loader: - kv 8: qwen3.attention.head_count u32 = 32
|
||||
llama_model_loader: - kv 9: qwen3.attention.head_count_kv u32 = 8
|
||||
llama_model_loader: - kv 10: qwen3.rope.freq_base f32 = 1000000.000000
|
||||
llama_model_loader: - kv 11: qwen3.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: qwen3.attention.key_length u32 = 128
|
||||
llama_model_loader: - kv 13: qwen3.attention.value_length u32 = 128
|
||||
llama_model_loader: - kv 14: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 15: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 16: tokenizer.ggml.tokens arr[str,151748] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 17: tokenizer.ggml.token_type arr[i32,151748] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 24: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 25: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 26: general.file_type u32 = 15
|
||||
llama_model_loader: - type f32: 145 tensors
|
||||
llama_model_loader: - type q4_K: 217 tensors
|
||||
llama_model_loader: - type q6_K: 37 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = Q4_K - Medium
|
||||
print_info: file size = 4.68 GiB (4.90 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = qwen3
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 40960
|
||||
print_info: n_embd = 4096
|
||||
print_info: n_layer = 36
|
||||
print_info: n_head = 32
|
||||
print_info: n_head_kv = 8
|
||||
print_info: n_rot = 128
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 128
|
||||
print_info: n_embd_head_v = 128
|
||||
print_info: n_gqa = 4
|
||||
print_info: n_embd_k_gqa = 1024
|
||||
print_info: n_embd_v_gqa = 1024
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 12288
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = -1
|
||||
print_info: rope type = 2
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 1000000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 40960
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = 8B
|
||||
print_info: model params = 8.19 B
|
||||
print_info: general.name = Llm
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 151748
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 36 repeating layers to GPU
|
||||
load_tensors: offloading output layer to GPU
|
||||
load_tensors: offloaded 37/37 layers to GPU
|
||||
load_tensors: Vulkan0 model buffer size = 4454.74 MiB
|
||||
load_tensors: CPU_Mapped model buffer size = 333.43 MiB
|
||||
.....................................................................................
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 1000000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (40960) -- the full capacity of the model will not be utilized
|
||||
llama_context: Vulkan_Host output buffer size = 0.58 MiB
|
||||
llama_kv_cache: Vulkan0 KV buffer size = 288.00 MiB
|
||||
llama_kv_cache: size = 288.00 MiB ( 2048 cells, 36 layers, 1/1 seqs), K (f16): 144.00 MiB, V (f16): 144.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 304.38 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 12.01 MiB
|
||||
llama_context: graph nodes = 1267
|
||||
llama_context: graph splits = 2
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 25673 MiB free
|
||||
llama_model_loader: loaded meta data with 30 key-value pairs and 193 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = llama
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Tts
|
||||
llama_model_loader: - kv 3: general.size_label str = 445M
|
||||
llama_model_loader: - kv 4: llama.block_count u32 = 20
|
||||
llama_model_loader: - kv 5: llama.context_length u32 = 4096
|
||||
llama_model_loader: - kv 6: llama.embedding_length u32 = 768
|
||||
llama_model_loader: - kv 7: llama.feed_forward_length u32 = 3072
|
||||
llama_model_loader: - kv 8: llama.attention.head_count u32 = 12
|
||||
llama_model_loader: - kv 9: llama.attention.head_count_kv u32 = 12
|
||||
llama_model_loader: - kv 10: llama.rope.freq_base f32 = 10000.000000
|
||||
llama_model_loader: - kv 11: llama.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: llama.attention.key_length u32 = 64
|
||||
llama_model_loader: - kv 13: llama.attention.value_length u32 = 64
|
||||
llama_model_loader: - kv 14: general.file_type u32 = 1
|
||||
llama_model_loader: - kv 15: llama.vocab_size u32 = 158626
|
||||
llama_model_loader: - kv 16: llama.rope.dimension_count u32 = 64
|
||||
llama_model_loader: - kv 17: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.tokens arr[str,158626] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.token_type arr[i32,158626] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 24: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 25: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 26: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 27: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 28: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 29: tokenizer.ggml.add_space_prefix bool = false
|
||||
llama_model_loader: - type f32: 50 tensors
|
||||
llama_model_loader: - type f16: 143 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = F16
|
||||
print_info: file size = 1.07 GiB (20.70 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = llama
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 4096
|
||||
print_info: n_embd = 768
|
||||
print_info: n_layer = 20
|
||||
print_info: n_head = 12
|
||||
print_info: n_head_kv = 12
|
||||
print_info: n_rot = 64
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 64
|
||||
print_info: n_embd_head_v = 64
|
||||
print_info: n_gqa = 1
|
||||
print_info: n_embd_k_gqa = 768
|
||||
print_info: n_embd_v_gqa = 768
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 3072
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = 0
|
||||
print_info: rope type = 0
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 10000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 4096
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = ?B
|
||||
print_info: model params = 444.94 M
|
||||
print_info: general.name = Tts
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 158626
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
load_tensors: skipping TTS tensor emb_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor emb_text.weight (size = 467140608 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.weight (size = 12582912 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.weight (size = 2359296 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.weight (size = 6291456 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.weight (size = 1179648 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor head_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 0 repeating layers to GPU
|
||||
load_tensors: offloaded 0/21 layers to GPU
|
||||
load_tensors: CPU_Mapped model buffer size = 592.48 MiB
|
||||
..............................................................
|
||||
llama_init_from_model: model default pooling_type is [0], but [-1] was specified
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 10000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (4096) -- the full capacity of the model will not be utilized
|
||||
llama_context: CPU output buffer size = 0.61 MiB
|
||||
llama_kv_cache: CPU KV buffer size = 120.00 MiB
|
||||
llama_kv_cache: size = 120.00 MiB ( 2048 cells, 20 layers, 1/1 seqs), K (f16): 60.00 MiB, V (f16): 60.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 30.75 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 7.01 MiB
|
||||
llama_context: graph nodes = 613
|
||||
llama_context: graph splits = 216 (with bs=512), 1 (with bs=1)
|
||||
vision_model_loader: model name:
|
||||
vision_model_loader: description: image encoder for MiniCPM-V
|
||||
vision_model_loader: GGUF version: 3
|
||||
vision_model_loader: alignment: 32
|
||||
vision_model_loader: n_tensors: 455
|
||||
vision_model_loader: n_kv: 20
|
||||
|
||||
vision_ctx: vision using Vulkan0 backend
|
||||
load_hparams: model_type: 0
|
||||
load_hparams: n_embd: 1152
|
||||
load_hparams: n_head: 16
|
||||
load_hparams: n_ff: 4304
|
||||
load_hparams: n_layer: 27
|
||||
load_hparams: ffn_op: gelu
|
||||
load_hparams: projection_dim: 0
|
||||
|
||||
--- vision hparams ---
|
||||
load_hparams: image_size: 448
|
||||
load_hparams: patch_size: 14
|
||||
load_hparams: minicpmv_version: 100045
|
||||
load_hparams: proj_scale_factor: 0
|
||||
load_hparams: n_wa_pattern: 0
|
||||
|
||||
load_hparams: model size: 1044.36 MiB
|
||||
load_hparams: metadata size: 0.16 MiB
|
||||
load_tensors: ffn up/down are swapped
|
||||
alloc_compute_meta: Vulkan0 compute buffer size = 100.30 MiB
|
||||
alloc_compute_meta: CPU compute buffer size = 18.30 MiB
|
||||
init tts....init t2w....voc_hg2_model: CPU backend using 8 threads
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=32000
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=200, n_mel=80, total_size=16000
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=200, n_mel=80, total_elements=16000
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=200, n_tokens=100, current_iter=0, current_total=0, new_total=100, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 1 (total_tokens=100, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 20]
|
||||
audition_audio_batch_encode: Final output: 20 tokens x 4096 dims
|
||||
=== Model Paths ===
|
||||
Base dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf
|
||||
LLM: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf [OK]
|
||||
Vision: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/vision/MiniCPM-o-4_5-vision-F16.gguf [OK]
|
||||
Audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf [OK]
|
||||
TTS: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf [OK]
|
||||
Projector: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf [OK]
|
||||
===================
|
||||
=== Initializing Omni Context ===
|
||||
Media type: 2 (omni: audio+vision)
|
||||
TTS enabled: yes
|
||||
Context size: 2048
|
||||
GPU layers: 99
|
||||
Token2Wav device: cpu
|
||||
Vision backend: metal
|
||||
TTS bin dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts
|
||||
Ref audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\vllm-omni-edge\benchmarks\edge_harness\results\e2e_expansion_20260923\evidence\minicpmo_cpp_radeon890m_prompted\fixture_0000.wav
|
||||
02:50:06.426 === omni_init start
|
||||
02:50:06.427 media_type = 2, duplex_mode = 0, base_output_dir = ./tools/omni/output
|
||||
02:50:06.427 === omni_init: loading new LLM model
|
||||
02:50:56.728 === omni_init: loading TTS model
|
||||
02:50:56.728 TTS model: loading with n_gpu_layers=0
|
||||
02:51:00.596 TTS sampler: temp=0.80, top_p=0.85, top_k=25, rep_penalty=1.05
|
||||
02:51:00.596 TTS: loading weights from GGUF (emb_code, emb_text, projector_semantic, head_code)...
|
||||
02:51:06.215 TTS: head_code shape: dim0=6562, dim1=768, will transpose to [6562, 768]
|
||||
02:51:06.221 TTS: head_code loaded, verifying first few values:
|
||||
02:51:06.221 head_code_weight[0] = 0.01361084 (expect ~0.01385498)
|
||||
02:51:06.221 head_code_weight[1] = -0.01660156 (expect ~-0.01647949)
|
||||
02:51:06.221 head_code_weight[768] = 0.04101562 (this is weight[1, 0], expect ~0.04150391)
|
||||
02:51:06.272 TTS: weights loaded successfully
|
||||
02:51:06.272 Projector: loading from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf
|
||||
02:51:06.375 Projector: loaded successfully
|
||||
02:51:06.376 === omni_init: loading APM model
|
||||
02:51:09.496 APM: init from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf
|
||||
02:51:15.048 Token2Wav: found models in \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf
|
||||
02:51:15.099 Token2Wav: all model files found, initializing session...
|
||||
02:51:15.100 Token2Wav: non-CUDA backend, vocoder using CPU for better performance
|
||||
02:51:15.100 Token2Wav: using prompt_cache from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf/prompt_cache.gguf
|
||||
02:51:19.818 Token2Wav: initialized successfully
|
||||
02:51:19.818 Python T2W: script_dir=./tools/omni/pyt2w, model_dir=./tools/omni/pyt2w/token2wav
|
||||
02:51:19.818 Token2Wav: 浣跨敤 C++ 瀹炵幇
|
||||
02:51:19.818 === omni_init success: ctx_llama = 000001b324f7f830
|
||||
=== Running test case ===
|
||||
Audio prefix: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-16-686sircx\fixture_
|
||||
Count: 2
|
||||
02:51:19.819 stream_prefill: n_past = 0
|
||||
voice_clone_prompt = <|im_start|>system
|
||||
妯′豢闊抽�鏍锋湰鐨勯煶鑹插苟鐢熸垚鏂扮殑鍐呭�銆�
|
||||
<|audio_start|>
|
||||
assistant_prompt = <|audio_end|>浣犵殑浠诲姟鏄�敤杩欑�澹伴煶妯″紡鏉ュ綋涓€涓�姪鎵嬨€傝�璁ょ湡銆侀珮璐ㄩ噺鍦板洖澶嶇敤鎴风殑闂��銆傝�鐢ㄩ珮鑷�劧搴︾殑鏂瑰紡鍜岀敤鎴疯亰澶┿€�<|im_end|>
|
||||
<|im_start|>user
|
||||
|
||||
02:51:19.819 system prompt ref_audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-16-686sircx\fixture_0000.wav
|
||||
02:51:20.306 system prompt ref_audio embedding: n_pos=20
|
||||
02:51:20.798 馃敀 n_keep 璁剧疆涓� 72 (system prompt tokens)锛岃繖閮ㄥ垎姘歌繙涓嶄細琚�粦鍔ㄧ獥鍙ｅ垹闄�
|
||||
02:51:20.798 stream_prefill(index=0): system prompt 鍒濆�鍖栧畬鎴愶紝ref_audio 宸插湪鍏朵腑 prefill
|
||||
02:51:20.798 [SW] system_prompt registered: preserve_length=72 (will be protected from sliding)
|
||||
02:51:20.798 n_past = 72
|
||||
02:51:20.798
|
||||
|
||||
c++ finish stream_prefill(index=0). n_past=72, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 0 (audio) : 0.979654 s
|
||||
encode_image_with_vision_chunks: step 1 of 1 encoded in 621.35 ms
|
||||
encode_image_with_vision_chunks: all 1 chunks encoded in 639.49 ms (grid: 1x1)
|
||||
|
||||
encode_image_with_vision_chunks: image encoded in 639.49 ms by vision ( 639.49 ms per chunk, 64 total tokens)
|
||||
omni_image_embed_make_chunks_with_filename: created 1 vision chunks from C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-16-686sircx\fixture_0001.jpg
|
||||
stream_prefill: prefilled 1 vision chunks (64 tokens each)
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=49339
|
||||
audition_audio_preprocess: Audio not aligned to 100ms (49339 samples = 3083.7ms), padded with silence to 49600 samples (3100ms)
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=310, n_mel=80, total_size=24800
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=310, n_mel=80, total_elements=24800
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=310, n_tokens=155, current_iter=1, current_total=155, new_total=310, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 2 (total_tokens=310, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 31]
|
||||
audition_audio_batch_encode: Final output: 31 tokens x 4096 dims
|
||||
02:51:21.957 stream_prefill(index=1): processing user audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-16-686sircx\fixture_0001.wav
|
||||
02:51:22.256 stream_prefill(index=1): user audio embedding: n_pos=31
|
||||
02:51:22.567
|
||||
|
||||
c++ finish stream_prefill(index=1). n_past=172, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 1 (audio+vision) : 1.76747 s
|
||||
02:51:22.567 馃搷 stream_decode 寮€濮�: n_past=172, n_keep=72, n_ctx=2048, duplex_mode=0
|
||||
02:51:22.567 stream_decode: create tts thread (simplex mode)
|
||||
02:51:22.567 stream_decode: create t2w thread
|
||||
02:51:22.567 wait prefill done
|
||||
02:51:22.567 馃搷 [鍗曞伐TTS] 娣诲姞 assistant prompt: "<|im_end|>
|
||||
<|im_start|>assistant
|
||||
<think>
|
||||
|
||||
</think>
|
||||
|
||||
<|tts_bos|>", n_past=172
|
||||
02:51:22.568 T2W thread (C++) started
|
||||
<user>
|
||||
<assistant>02:51:22.573 TTS: 鍒涘缓鍗曞伐妯″紡杈撳嚭鐩�綍: ./tools/omni/output/round_000
|
||||
02:51:22.574 TTS thread started
|
||||
02:51:22.732 馃搷 [鍗曞伐TTS] assistant prompt 瀹屾垚, n_past=182
|
||||
02:51:22.732 LLM decode: max_tgt_len = 96, n_predict = 96, n_ctx = 2048
|
||||
02:51:23.255 LLM: detected end token, set llm_generation_done=true
|
||||
02:51:23.255 LLM->TTS: text='The square in the image is red.', n_tokens=8, hidden_size=32768, n_embd=4096, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:51:23.255 馃搷 Decode 缁撴潫: n_past=191, 鍓╀綑绌洪棿=1857, 鏃犻渶婊戠獥
|
||||
02:51:23.255 馃搷 杞�� 1 缁撴潫锛岃�褰曡竟鐣屼簬 n_past=191
|
||||
02:51:23.255 TTS<-LLM: chunk_idx=0, text='The square in the image is red.', n_tokens=8, hidden_size=32768, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:51:23.255 TTS: after queue pop - speek_done=0, llm_finish=1, llm_text.empty=0, token_ids.size=8
|
||||
02:51:23.256 TTS: before empty check - response.empty=0, response='The square in the image is red.', llm_finish=1
|
||||
Waiting for audio generation to complete...
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_0.wav does not exist or is empty
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_1.wav does not exist or is empty
|
||||
TTS: no valid WAV files to merge
|
||||
02:51:23.256 TTS: DEBUG before has_llm_data - token_ids.size=8, hidden_states.size=32768, n_embd=4096
|
||||
02:51:23.256 TTS: has_llm_data=1, token_ids=8, hidden_states=32768, n_embd=4096, llm_finish=1
|
||||
02:51:23.256 TTS: n_tokens_orig=8, n_tokens_filtered=8 (filtered 0 special tokens)
|
||||
02:51:23.296 TTS: saved merged embeddings (text) to ./tools/omni/output/round_000/llm_debug/chunk_0/merged_embeddings.txt
|
||||
02:51:23.296 TTS Simplex: generating audio tokens for chunk 0 (n_tokens=8, tts_n_embd=768)
|
||||
02:51:23.296 TTS Simplex: 鍦� prefill 鍓嶆坊鍔� audio_bos (chunk_idx=0, new_size=9)
|
||||
02:51:23.296 TTS Simplex: first chunk - cleared KV cache
|
||||
02:51:23.296 TTS Simplex: first chunk - cleared tts_all_generated_tokens and tts_token_buffer
|
||||
02:51:23.349 馃搷 涓轰笅涓€杞�噯澶�: eval <|im_end|>\n<|im_start|>user\n, n_past=196
|
||||
02:51:23.572 TTS Simplex: prefill completed, n_past_tts=9
|
||||
02:51:23.572 TTS Simplex: sampler created
|
||||
02:51:23.572 TTS simplex: is_audio_bos=true (first audio token)
|
||||
02:51:23.598 TTS Simplex Phase1: token 1/500: rel_id=761
|
||||
02:51:23.608 TTS Simplex Phase1: token 2/500: rel_id=4137
|
||||
02:51:23.620 TTS Simplex Phase1: token 3/500: rel_id=4218
|
||||
02:51:23.630 TTS Simplex Phase1: token 4/500: rel_id=4218
|
||||
02:51:23.641 TTS Simplex Phase1: token 5/500: rel_id=4218
|
||||
02:51:23.856 TTS Simplex Phase1: yield 25 tokens 鍒� T2W (step 25, buffer=25)
|
||||
02:51:23.856 TTS Simplex Phase1: token 25/500: rel_id=1978
|
||||
02:51:23.856 T2W绾跨▼(C++): 杞��鍒囨崲 (-1 -> 0)锛屾洿鏂拌緭鍑虹洰褰�
|
||||
02:51:23.856 T2W绾跨▼(C++): 鏂拌緭鍑虹洰褰� ./tools/omni/output/round_000/tts_wav
|
||||
02:51:24.047 TTS Simplex Phase1: EOS token at step 43
|
||||
02:51:24.047 TTS Simplex: is_final_text_chunk=true, will enter Phase 2 for text_eos_embed
|
||||
02:51:24.047 TTS Simplex Phase2: injecting text_eos_embed + audio_bos at n_past=51
|
||||
02:51:24.055 TTS Simplex Phase2: text_eos_embed injected, n_past=52
|
||||
02:51:24.063 TTS Simplex Phase2: audio_bos injected, n_past=53, starting final generation
|
||||
02:51:24.147 TTS Simplex Phase2: yield 25 tokens 鍒� T2W
|
||||
02:51:24.279 TTS Simplex Phase2: EOS at step 20 (final end)
|
||||
02:51:24.279 TTS Simplex: chunk 缁撴潫锛宼ts_token_buffer 鍓╀綑 11 tokens (淇濈暀绛変笅涓€涓� chunk)
|
||||
02:51:24.279 TTS Simplex: is_final_text_chunk=true, flush 鍓╀綑 11 tokens 浠� tts_token_buffer
|
||||
02:51:24.279 TTS Simplex: generated 61 audio tokens for chunk 0
|
||||
02:51:24.279 TTS Simplex: chunk 0 first tokens: [761 4137 4218 4218 4218 ], last tokens: [739 1701 4218 4218 6082 ]
|
||||
02:51:24.279 TTS Simplex: updated tts_n_past_accumulated=73, total_generated_tokens=61
|
||||
02:51:24.280 TTS: finished processing all chunks
|
||||
02:51:24.280 TTS: 鍗曞伐妯″紡杞��缁撴潫锛屼笅涓€杞� round_idx=1
|
||||
02:51:24.280 TTS: sent is_final=true to T2W queue (turn end)
|
||||
02:51:26.234 馃帀 棣栧搷鏃堕棿 (First Audio Response): 3666ms
|
||||
02:51:26.234 T2W绾跨▼: wav_0.wav | 0.84s audio | 2377.2ms inference | RTF=2.83 | t=3666ms
|
||||
02:51:28.463 T2W绾跨▼: wav_1.wav | 1.00s audio | 2227.8ms inference | RTF=2.23 | t=5895ms
|
||||
02:51:30.595 T2W绾跨▼: wav_2.wav | 0.72s audio | 2131.8ms inference | RTF=2.96 | t=8027ms
|
||||
02:51:30.596 T2W绾跨▼: 鍐欏叆缁撴潫鏍囪� ./tools/omni/output/round_000/tts_wav/generation_done.flag (last_wav=2)
|
||||
02:51:30.596 T2W绾跨▼: 杞��缁撴潫鍚庢洿鏂拌緭鍑虹洰褰曚负 ./tools/omni/output/round_001/tts_wav
|
||||
02:02:51:51:30.3000.638 638 omni_stop_threads: stop signals sent
|
||||
T2W(C++) 绾跨▼: 鍋滄�
|
||||
[timing] call=0 tokens=28 final=0 token2mel=2044.605ms vocoder=332.557ms total=2377.176ms audio=20160
|
||||
[timing] call=1 tokens=28 final=0 token2mel=1844.917ms vocoder=382.804ms total=2227.738ms audio=24000
|
||||
[timing] call=2 tokens=14 final=1 token2mel=1881.609ms vocoder=250.147ms total=2131.771ms audio=17280
|
||||
Audio generation completed.
|
||||
llama_perf_context_print: load time = 25880.97 ms
|
||||
llama_perf_context_print: prompt eval time = 2912.20 ms / 182 tokens ( 16.00 ms per token, 62.50 tokens per second)
|
||||
llama_perf_context_print: eval time = 521.12 ms / 9 runs ( 57.90 ms per token, 17.27 tokens per second)
|
||||
llama_perf_context_print: total time = 33786.71 ms / 191 tokens
|
||||
llama_perf_context_print: graphs reused = 10
|
||||
Token2Wav (C++): session released
|
||||
tts thread end
|
||||
t2w thread end
|
||||
+483
@@ -0,0 +1,483 @@
|
||||
build: 64 (5202b7b) with x86_64-w64-mingw32-gcc-posix (GCC) 13-posix for x86_64-w64-mingw32
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 30720 MiB free
|
||||
llama_model_loader: loaded meta data with 27 key-value pairs and 399 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = qwen3
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Llm
|
||||
llama_model_loader: - kv 3: general.size_label str = 8.2B
|
||||
llama_model_loader: - kv 4: qwen3.block_count u32 = 36
|
||||
llama_model_loader: - kv 5: qwen3.context_length u32 = 40960
|
||||
llama_model_loader: - kv 6: qwen3.embedding_length u32 = 4096
|
||||
llama_model_loader: - kv 7: qwen3.feed_forward_length u32 = 12288
|
||||
llama_model_loader: - kv 8: qwen3.attention.head_count u32 = 32
|
||||
llama_model_loader: - kv 9: qwen3.attention.head_count_kv u32 = 8
|
||||
llama_model_loader: - kv 10: qwen3.rope.freq_base f32 = 1000000.000000
|
||||
llama_model_loader: - kv 11: qwen3.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: qwen3.attention.key_length u32 = 128
|
||||
llama_model_loader: - kv 13: qwen3.attention.value_length u32 = 128
|
||||
llama_model_loader: - kv 14: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 15: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 16: tokenizer.ggml.tokens arr[str,151748] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 17: tokenizer.ggml.token_type arr[i32,151748] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 24: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 25: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 26: general.file_type u32 = 15
|
||||
llama_model_loader: - type f32: 145 tensors
|
||||
llama_model_loader: - type q4_K: 217 tensors
|
||||
llama_model_loader: - type q6_K: 37 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = Q4_K - Medium
|
||||
print_info: file size = 4.68 GiB (4.90 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = qwen3
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 40960
|
||||
print_info: n_embd = 4096
|
||||
print_info: n_layer = 36
|
||||
print_info: n_head = 32
|
||||
print_info: n_head_kv = 8
|
||||
print_info: n_rot = 128
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 128
|
||||
print_info: n_embd_head_v = 128
|
||||
print_info: n_gqa = 4
|
||||
print_info: n_embd_k_gqa = 1024
|
||||
print_info: n_embd_v_gqa = 1024
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 12288
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = -1
|
||||
print_info: rope type = 2
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 1000000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 40960
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = 8B
|
||||
print_info: model params = 8.19 B
|
||||
print_info: general.name = Llm
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 151748
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 36 repeating layers to GPU
|
||||
load_tensors: offloading output layer to GPU
|
||||
load_tensors: offloaded 37/37 layers to GPU
|
||||
load_tensors: Vulkan0 model buffer size = 4454.74 MiB
|
||||
load_tensors: CPU_Mapped model buffer size = 333.43 MiB
|
||||
.....................................................................................
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 1000000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (40960) -- the full capacity of the model will not be utilized
|
||||
llama_context: Vulkan_Host output buffer size = 0.58 MiB
|
||||
llama_kv_cache: Vulkan0 KV buffer size = 288.00 MiB
|
||||
llama_kv_cache: size = 288.00 MiB ( 2048 cells, 36 layers, 1/1 seqs), K (f16): 144.00 MiB, V (f16): 144.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 304.38 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 12.01 MiB
|
||||
llama_context: graph nodes = 1267
|
||||
llama_context: graph splits = 2
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 25673 MiB free
|
||||
llama_model_loader: loaded meta data with 30 key-value pairs and 193 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = llama
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Tts
|
||||
llama_model_loader: - kv 3: general.size_label str = 445M
|
||||
llama_model_loader: - kv 4: llama.block_count u32 = 20
|
||||
llama_model_loader: - kv 5: llama.context_length u32 = 4096
|
||||
llama_model_loader: - kv 6: llama.embedding_length u32 = 768
|
||||
llama_model_loader: - kv 7: llama.feed_forward_length u32 = 3072
|
||||
llama_model_loader: - kv 8: llama.attention.head_count u32 = 12
|
||||
llama_model_loader: - kv 9: llama.attention.head_count_kv u32 = 12
|
||||
llama_model_loader: - kv 10: llama.rope.freq_base f32 = 10000.000000
|
||||
llama_model_loader: - kv 11: llama.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: llama.attention.key_length u32 = 64
|
||||
llama_model_loader: - kv 13: llama.attention.value_length u32 = 64
|
||||
llama_model_loader: - kv 14: general.file_type u32 = 1
|
||||
llama_model_loader: - kv 15: llama.vocab_size u32 = 158626
|
||||
llama_model_loader: - kv 16: llama.rope.dimension_count u32 = 64
|
||||
llama_model_loader: - kv 17: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.tokens arr[str,158626] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.token_type arr[i32,158626] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 24: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 25: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 26: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 27: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 28: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 29: tokenizer.ggml.add_space_prefix bool = false
|
||||
llama_model_loader: - type f32: 50 tensors
|
||||
llama_model_loader: - type f16: 143 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = F16
|
||||
print_info: file size = 1.07 GiB (20.70 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = llama
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 4096
|
||||
print_info: n_embd = 768
|
||||
print_info: n_layer = 20
|
||||
print_info: n_head = 12
|
||||
print_info: n_head_kv = 12
|
||||
print_info: n_rot = 64
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 64
|
||||
print_info: n_embd_head_v = 64
|
||||
print_info: n_gqa = 1
|
||||
print_info: n_embd_k_gqa = 768
|
||||
print_info: n_embd_v_gqa = 768
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 3072
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = 0
|
||||
print_info: rope type = 0
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 10000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 4096
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = ?B
|
||||
print_info: model params = 444.94 M
|
||||
print_info: general.name = Tts
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 158626
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
load_tensors: skipping TTS tensor emb_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor emb_text.weight (size = 467140608 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.weight (size = 12582912 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.weight (size = 2359296 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.weight (size = 6291456 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.weight (size = 1179648 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor head_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 0 repeating layers to GPU
|
||||
load_tensors: offloaded 0/21 layers to GPU
|
||||
load_tensors: CPU_Mapped model buffer size = 592.48 MiB
|
||||
..............................................................
|
||||
llama_init_from_model: model default pooling_type is [0], but [-1] was specified
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 10000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (4096) -- the full capacity of the model will not be utilized
|
||||
llama_context: CPU output buffer size = 0.61 MiB
|
||||
llama_kv_cache: CPU KV buffer size = 120.00 MiB
|
||||
llama_kv_cache: size = 120.00 MiB ( 2048 cells, 20 layers, 1/1 seqs), K (f16): 60.00 MiB, V (f16): 60.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 30.75 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 7.01 MiB
|
||||
llama_context: graph nodes = 613
|
||||
llama_context: graph splits = 216 (with bs=512), 1 (with bs=1)
|
||||
vision_model_loader: model name:
|
||||
vision_model_loader: description: image encoder for MiniCPM-V
|
||||
vision_model_loader: GGUF version: 3
|
||||
vision_model_loader: alignment: 32
|
||||
vision_model_loader: n_tensors: 455
|
||||
vision_model_loader: n_kv: 20
|
||||
|
||||
vision_ctx: vision using Vulkan0 backend
|
||||
load_hparams: model_type: 0
|
||||
load_hparams: n_embd: 1152
|
||||
load_hparams: n_head: 16
|
||||
load_hparams: n_ff: 4304
|
||||
load_hparams: n_layer: 27
|
||||
load_hparams: ffn_op: gelu
|
||||
load_hparams: projection_dim: 0
|
||||
|
||||
--- vision hparams ---
|
||||
load_hparams: image_size: 448
|
||||
load_hparams: patch_size: 14
|
||||
load_hparams: minicpmv_version: 100045
|
||||
load_hparams: proj_scale_factor: 0
|
||||
load_hparams: n_wa_pattern: 0
|
||||
|
||||
load_hparams: model size: 1044.36 MiB
|
||||
load_hparams: metadata size: 0.16 MiB
|
||||
load_tensors: ffn up/down are swapped
|
||||
alloc_compute_meta: Vulkan0 compute buffer size = 100.30 MiB
|
||||
alloc_compute_meta: CPU compute buffer size = 18.30 MiB
|
||||
init tts....init t2w....voc_hg2_model: CPU backend using 8 threads
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=32000
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=200, n_mel=80, total_size=16000
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=200, n_mel=80, total_elements=16000
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=200, n_tokens=100, current_iter=0, current_total=0, new_total=100, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 1 (total_tokens=100, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 20]
|
||||
audition_audio_batch_encode: Final output: 20 tokens x 4096 dims
|
||||
=== Model Paths ===
|
||||
Base dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf
|
||||
LLM: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf [OK]
|
||||
Vision: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/vision/MiniCPM-o-4_5-vision-F16.gguf [OK]
|
||||
Audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf [OK]
|
||||
TTS: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf [OK]
|
||||
Projector: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf [OK]
|
||||
===================
|
||||
=== Initializing Omni Context ===
|
||||
Media type: 2 (omni: audio+vision)
|
||||
TTS enabled: yes
|
||||
Context size: 2048
|
||||
GPU layers: 99
|
||||
Token2Wav device: cpu
|
||||
Vision backend: metal
|
||||
TTS bin dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts
|
||||
Ref audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\vllm-omni-edge\benchmarks\edge_harness\results\e2e_expansion_20260923\evidence\minicpmo_cpp_radeon890m_prompted\fixture_0000.wav
|
||||
02:51:32.459 === omni_init start
|
||||
02:51:32.460 media_type = 2, duplex_mode = 0, base_output_dir = ./tools/omni/output
|
||||
02:51:32.460 === omni_init: loading new LLM model
|
||||
02:52:26.995 === omni_init: loading TTS model
|
||||
02:52:26.995 TTS model: loading with n_gpu_layers=0
|
||||
02:52:30.548 TTS sampler: temp=0.80, top_p=0.85, top_k=25, rep_penalty=1.05
|
||||
02:52:30.548 TTS: loading weights from GGUF (emb_code, emb_text, projector_semantic, head_code)...
|
||||
02:52:36.028 TTS: head_code shape: dim0=6562, dim1=768, will transpose to [6562, 768]
|
||||
02:52:36.034 TTS: head_code loaded, verifying first few values:
|
||||
02:52:36.034 head_code_weight[0] = 0.01361084 (expect ~0.01385498)
|
||||
02:52:36.034 head_code_weight[1] = -0.01660156 (expect ~-0.01647949)
|
||||
02:52:36.034 head_code_weight[768] = 0.04101562 (this is weight[1, 0], expect ~0.04150391)
|
||||
02:52:36.085 TTS: weights loaded successfully
|
||||
02:52:36.085 Projector: loading from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf
|
||||
02:52:36.185 Projector: loaded successfully
|
||||
02:52:36.186 === omni_init: loading APM model
|
||||
02:52:39.215 APM: init from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf
|
||||
02:52:44.651 Token2Wav: found models in \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf
|
||||
02:52:44.709 Token2Wav: all model files found, initializing session...
|
||||
02:52:44.709 Token2Wav: non-CUDA backend, vocoder using CPU for better performance
|
||||
02:52:44.709 Token2Wav: using prompt_cache from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf/prompt_cache.gguf
|
||||
02:52:49.253 Token2Wav: initialized successfully
|
||||
02:52:49.253 Python T2W: script_dir=./tools/omni/pyt2w, model_dir=./tools/omni/pyt2w/token2wav
|
||||
02:52:49.253 Token2Wav: 浣跨敤 C++ 瀹炵幇
|
||||
02:52:49.253 === omni_init success: ctx_llama = 000001b52f2bf830
|
||||
=== Running test case ===
|
||||
Audio prefix: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-17-82_a8k_s\fixture_
|
||||
Count: 2
|
||||
02:52:49.253 stream_prefill: n_past = 0
|
||||
voice_clone_prompt = <|im_start|>system
|
||||
妯′豢闊抽�鏍锋湰鐨勯煶鑹插苟鐢熸垚鏂扮殑鍐呭�銆�
|
||||
<|audio_start|>
|
||||
assistant_prompt = <|audio_end|>浣犵殑浠诲姟鏄�敤杩欑�澹伴煶妯″紡鏉ュ綋涓€涓�姪鎵嬨€傝�璁ょ湡銆侀珮璐ㄩ噺鍦板洖澶嶇敤鎴风殑闂��銆傝�鐢ㄩ珮鑷�劧搴︾殑鏂瑰紡鍜岀敤鎴疯亰澶┿€�<|im_end|>
|
||||
<|im_start|>user
|
||||
|
||||
02:52:49.253 system prompt ref_audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-17-82_a8k_s\fixture_0000.wav
|
||||
02:52:49.996 system prompt ref_audio embedding: n_pos=20
|
||||
02:52:50.495 馃敀 n_keep 璁剧疆涓� 72 (system prompt tokens)锛岃繖閮ㄥ垎姘歌繙涓嶄細琚�粦鍔ㄧ獥鍙ｅ垹闄�
|
||||
02:52:50.495 stream_prefill(index=0): system prompt 鍒濆�鍖栧畬鎴愶紝ref_audio 宸插湪鍏朵腑 prefill
|
||||
02:52:50.495 [SW] system_prompt registered: preserve_length=72 (will be protected from sliding)
|
||||
02:52:50.495 n_past = 72
|
||||
02:52:50.495
|
||||
|
||||
c++ finish stream_prefill(index=0). n_past=72, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 0 (audio) : 1.24154 s
|
||||
encode_image_with_vision_chunks: step 1 of 1 encoded in 621.15 ms
|
||||
encode_image_with_vision_chunks: all 1 chunks encoded in 640.51 ms (grid: 1x1)
|
||||
|
||||
encode_image_with_vision_chunks: image encoded in 640.51 ms by vision ( 640.51 ms per chunk, 64 total tokens)
|
||||
omni_image_embed_make_chunks_with_filename: created 1 vision chunks from C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-17-82_a8k_s\fixture_0001.jpg
|
||||
stream_prefill: prefilled 1 vision chunks (64 tokens each)
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=49339
|
||||
audition_audio_preprocess: Audio not aligned to 100ms (49339 samples = 3083.7ms), padded with silence to 49600 samples (3100ms)
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=310, n_mel=80, total_size=24800
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=310, n_mel=80, total_elements=24800
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=310, n_tokens=155, current_iter=1, current_total=155, new_total=310, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 2 (total_tokens=310, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 31]
|
||||
audition_audio_batch_encode: Final output: 31 tokens x 4096 dims
|
||||
02:52:51.652 stream_prefill(index=1): processing user audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-17-82_a8k_s\fixture_0001.wav
|
||||
02:52:51.952 stream_prefill(index=1): user audio embedding: n_pos=31
|
||||
02:52:52.264
|
||||
|
||||
c++ finish stream_prefill(index=1). n_past=172, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 1 (audio+vision) : 1.76643 s
|
||||
02:52:52.264 馃搷 stream_decode 寮€濮�: n_past=172, n_keep=72, n_ctx=2048, duplex_mode=0
|
||||
02:52:52.264 stream_decode: create tts thread (simplex mode)
|
||||
02:52:52.264 stream_decode: create t2w thread
|
||||
02:52:52.264 wait prefill done
|
||||
02:52:52.264 馃搷 [鍗曞伐TTS] 娣诲姞 assistant prompt: "<|im_end|>
|
||||
<|im_start|>assistant
|
||||
<think>
|
||||
|
||||
</think>
|
||||
|
||||
<|tts_bos|>", n_past=172
|
||||
02:52:52.264 T2W thread (C++) started
|
||||
<user>
|
||||
<assistant>02:52:52.268 TTS: 鍒涘缓鍗曞伐妯″紡杈撳嚭鐩�綍: ./tools/omni/output/round_000
|
||||
02:52:52.268 TTS thread started
|
||||
02:52:52.425 馃搷 [鍗曞伐TTS] assistant prompt 瀹屾垚, n_past=182
|
||||
02:52:52.425 LLM decode: max_tgt_len = 96, n_predict = 96, n_ctx = 2048
|
||||
02:52:52.943 LLM: detected end token, set llm_generation_done=true
|
||||
02:52:52.943 LLM->TTS: text='The square in the image is red.', n_tokens=8, hidden_size=32768, n_embd=4096, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:52:52.943 馃搷 Decode 缁撴潫: n_past=191, 鍓╀綑绌洪棿=1857, 鏃犻渶婊戠獥
|
||||
02:52:52.943 馃搷 杞�� 1 缁撴潫锛岃�褰曡竟鐣屼簬 n_past=191
|
||||
02:52:52.943 TTS<-LLM: chunk_idx=0, text='The square in the image is red.', n_tokens=8, hidden_size=32768, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:52:52.943 TTS: after queue pop - speek_done=0, llm_finish=1, llm_text.empty=0, token_ids.size=8
|
||||
02:52:52.943 TTS: before empty check - response.empty=0, response='The square in the image is red.', llm_finish=1
|
||||
Waiting for audio generation to complete...
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_0.wav does not exist or is empty
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_1.wav does not exist or is empty
|
||||
TTS: no valid WAV files to merge
|
||||
02:52:52.943 TTS: DEBUG before has_llm_data - token_ids.size=8, hidden_states.size=32768, n_embd=4096
|
||||
02:52:52.943 TTS: has_llm_data=1, token_ids=8, hidden_states=32768, n_embd=4096, llm_finish=1
|
||||
02:52:52.944 TTS: n_tokens_orig=8, n_tokens_filtered=8 (filtered 0 special tokens)
|
||||
02:52:52.985 TTS: saved merged embeddings (text) to ./tools/omni/output/round_000/llm_debug/chunk_0/merged_embeddings.txt
|
||||
02:52:52.985 TTS Simplex: generating audio tokens for chunk 0 (n_tokens=8, tts_n_embd=768)
|
||||
02:52:52.985 TTS Simplex: 鍦� prefill 鍓嶆坊鍔� audio_bos (chunk_idx=0, new_size=9)
|
||||
02:52:52.985 TTS Simplex: first chunk - cleared KV cache
|
||||
02:52:52.985 TTS Simplex: first chunk - cleared tts_all_generated_tokens and tts_token_buffer
|
||||
02:52:53.039 馃搷 涓轰笅涓€杞�噯澶�: eval <|im_end|>\n<|im_start|>user\n, n_past=196
|
||||
02:52:53.243 TTS Simplex: prefill completed, n_past_tts=9
|
||||
02:52:53.243 TTS Simplex: sampler created
|
||||
02:52:53.243 TTS simplex: is_audio_bos=true (first audio token)
|
||||
02:52:53.268 TTS Simplex Phase1: token 1/500: rel_id=770
|
||||
02:52:53.280 TTS Simplex Phase1: token 2/500: rel_id=1950
|
||||
02:52:53.290 TTS Simplex Phase1: token 3/500: rel_id=4137
|
||||
02:52:53.301 TTS Simplex Phase1: token 4/500: rel_id=4137
|
||||
02:52:53.312 TTS Simplex Phase1: token 5/500: rel_id=4218
|
||||
02:52:53.537 TTS Simplex Phase1: yield 25 tokens 鍒� T2W (step 25, buffer=25)
|
||||
02:52:53.537 TTS Simplex Phase1: token 25/500: rel_id=29
|
||||
02:52:53.537 T2W绾跨▼(C++): 杞��鍒囨崲 (-1 -> 0)锛屾洿鏂拌緭鍑虹洰褰�
|
||||
02:52:53.537 T2W绾跨▼(C++): 鏂拌緭鍑虹洰褰� ./tools/omni/output/round_000/tts_wav
|
||||
02:52:53.689 TTS Simplex Phase1: EOS token at step 39
|
||||
02:52:53.689 TTS Simplex: is_final_text_chunk=true, will enter Phase 2 for text_eos_embed
|
||||
02:52:53.689 TTS Simplex Phase2: injecting text_eos_embed + audio_bos at n_past=47
|
||||
02:52:53.697 TTS Simplex Phase2: text_eos_embed injected, n_past=48
|
||||
02:52:53.705 TTS Simplex Phase2: audio_bos injected, n_past=49, starting final generation
|
||||
02:52:53.845 TTS Simplex Phase2: yield 25 tokens 鍒� T2W
|
||||
02:52:53.890 TTS Simplex Phase2: EOS at step 16 (final end)
|
||||
02:52:53.890 TTS Simplex: chunk 缁撴潫锛宼ts_token_buffer 鍓╀綑 3 tokens (淇濈暀绛変笅涓€涓� chunk)
|
||||
02:52:53.890 TTS Simplex: is_final_text_chunk=true, flush 鍓╀綑 3 tokens 浠� tts_token_buffer
|
||||
02:52:53.890 TTS Simplex: generated 53 audio tokens for chunk 0
|
||||
02:52:53.890 TTS Simplex: chunk 0 first tokens: [770 1950 4137 4137 4218 ], last tokens: [37 729 1950 4218 4137 ]
|
||||
02:52:53.890 TTS Simplex: updated tts_n_past_accumulated=65, total_generated_tokens=53
|
||||
02:52:53.891 TTS: finished processing all chunks
|
||||
02:52:53.891 TTS: 鍗曞伐妯″紡杞��缁撴潫锛屼笅涓€杞� round_idx=1
|
||||
02:52:53.891 TTS: sent is_final=true to T2W queue (turn end)
|
||||
02:52:55.874 馃帀 棣栧搷鏃堕棿 (First Audio Response): 3610ms
|
||||
02:52:55.874 T2W绾跨▼: wav_0.wav | 0.84s audio | 2336.5ms inference | RTF=2.78 | t=3610ms
|
||||
02:52:58.123 T2W绾跨▼: wav_1.wav | 1.00s audio | 2247.7ms inference | RTF=2.25 | t=5859ms
|
||||
02:53:00.120 T2W绾跨▼: wav_2.wav | 0.40s audio | 1996.7ms inference | RTF=4.99 | t=7856ms
|
||||
02:53:00.121 T2W绾跨▼: 鍐欏叆缁撴潫鏍囪� ./tools/omni/output/round_000/tts_wav/generation_done.flag (last_wav=2)
|
||||
02:53:00.121 T2W绾跨▼: 杞��缁撴潫鍚庢洿鏂拌緭鍑虹洰褰曚负 ./tools/omni/output/round_001/tts_wav
|
||||
02:53:00.195 omni_stop_threads: stop signals sent
|
||||
02:53:00.195 T2W(C++) 绾跨▼: 鍋滄�
|
||||
[timing] call=0 tokens=28 final=0 token2mel=1991.471ms vocoder=344.950ms total=2336.435ms audio=20160
|
||||
[timing] call=1 tokens=28 final=0 token2mel=1857.109ms vocoder=390.564ms total=2247.683ms audio=24000
|
||||
[timing] call=2 tokens=6 final=1 token2mel=1851.183ms vocoder=145.531ms total=1996.723ms audio=9600
|
||||
Audio generation completed.
|
||||
llama_perf_context_print: load time = 25018.23 ms
|
||||
llama_perf_context_print: prompt eval time = 3170.99 ms / 182 tokens ( 17.42 ms per token, 57.40 tokens per second)
|
||||
llama_perf_context_print: eval time = 514.99 ms / 9 runs ( 57.22 ms per token, 17.48 tokens per second)
|
||||
llama_perf_context_print: total time = 32788.54 ms / 191 tokens
|
||||
llama_perf_context_print: graphs reused = 10
|
||||
Token2Wav (C++): session released
|
||||
tts thread end
|
||||
t2w thread end
|
||||
+483
@@ -0,0 +1,483 @@
|
||||
build: 64 (5202b7b) with x86_64-w64-mingw32-gcc-posix (GCC) 13-posix for x86_64-w64-mingw32
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 30720 MiB free
|
||||
llama_model_loader: loaded meta data with 27 key-value pairs and 399 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = qwen3
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Llm
|
||||
llama_model_loader: - kv 3: general.size_label str = 8.2B
|
||||
llama_model_loader: - kv 4: qwen3.block_count u32 = 36
|
||||
llama_model_loader: - kv 5: qwen3.context_length u32 = 40960
|
||||
llama_model_loader: - kv 6: qwen3.embedding_length u32 = 4096
|
||||
llama_model_loader: - kv 7: qwen3.feed_forward_length u32 = 12288
|
||||
llama_model_loader: - kv 8: qwen3.attention.head_count u32 = 32
|
||||
llama_model_loader: - kv 9: qwen3.attention.head_count_kv u32 = 8
|
||||
llama_model_loader: - kv 10: qwen3.rope.freq_base f32 = 1000000.000000
|
||||
llama_model_loader: - kv 11: qwen3.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: qwen3.attention.key_length u32 = 128
|
||||
llama_model_loader: - kv 13: qwen3.attention.value_length u32 = 128
|
||||
llama_model_loader: - kv 14: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 15: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 16: tokenizer.ggml.tokens arr[str,151748] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 17: tokenizer.ggml.token_type arr[i32,151748] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 24: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 25: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 26: general.file_type u32 = 15
|
||||
llama_model_loader: - type f32: 145 tensors
|
||||
llama_model_loader: - type q4_K: 217 tensors
|
||||
llama_model_loader: - type q6_K: 37 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = Q4_K - Medium
|
||||
print_info: file size = 4.68 GiB (4.90 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = qwen3
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 40960
|
||||
print_info: n_embd = 4096
|
||||
print_info: n_layer = 36
|
||||
print_info: n_head = 32
|
||||
print_info: n_head_kv = 8
|
||||
print_info: n_rot = 128
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 128
|
||||
print_info: n_embd_head_v = 128
|
||||
print_info: n_gqa = 4
|
||||
print_info: n_embd_k_gqa = 1024
|
||||
print_info: n_embd_v_gqa = 1024
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 12288
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = -1
|
||||
print_info: rope type = 2
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 1000000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 40960
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = 8B
|
||||
print_info: model params = 8.19 B
|
||||
print_info: general.name = Llm
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 151748
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 36 repeating layers to GPU
|
||||
load_tensors: offloading output layer to GPU
|
||||
load_tensors: offloaded 37/37 layers to GPU
|
||||
load_tensors: Vulkan0 model buffer size = 4454.74 MiB
|
||||
load_tensors: CPU_Mapped model buffer size = 333.43 MiB
|
||||
.....................................................................................
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 1000000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (40960) -- the full capacity of the model will not be utilized
|
||||
llama_context: Vulkan_Host output buffer size = 0.58 MiB
|
||||
llama_kv_cache: Vulkan0 KV buffer size = 288.00 MiB
|
||||
llama_kv_cache: size = 288.00 MiB ( 2048 cells, 36 layers, 1/1 seqs), K (f16): 144.00 MiB, V (f16): 144.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 304.38 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 12.01 MiB
|
||||
llama_context: graph nodes = 1267
|
||||
llama_context: graph splits = 2
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 25673 MiB free
|
||||
llama_model_loader: loaded meta data with 30 key-value pairs and 193 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = llama
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Tts
|
||||
llama_model_loader: - kv 3: general.size_label str = 445M
|
||||
llama_model_loader: - kv 4: llama.block_count u32 = 20
|
||||
llama_model_loader: - kv 5: llama.context_length u32 = 4096
|
||||
llama_model_loader: - kv 6: llama.embedding_length u32 = 768
|
||||
llama_model_loader: - kv 7: llama.feed_forward_length u32 = 3072
|
||||
llama_model_loader: - kv 8: llama.attention.head_count u32 = 12
|
||||
llama_model_loader: - kv 9: llama.attention.head_count_kv u32 = 12
|
||||
llama_model_loader: - kv 10: llama.rope.freq_base f32 = 10000.000000
|
||||
llama_model_loader: - kv 11: llama.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: llama.attention.key_length u32 = 64
|
||||
llama_model_loader: - kv 13: llama.attention.value_length u32 = 64
|
||||
llama_model_loader: - kv 14: general.file_type u32 = 1
|
||||
llama_model_loader: - kv 15: llama.vocab_size u32 = 158626
|
||||
llama_model_loader: - kv 16: llama.rope.dimension_count u32 = 64
|
||||
llama_model_loader: - kv 17: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.tokens arr[str,158626] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.token_type arr[i32,158626] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 24: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 25: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 26: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 27: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 28: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 29: tokenizer.ggml.add_space_prefix bool = false
|
||||
llama_model_loader: - type f32: 50 tensors
|
||||
llama_model_loader: - type f16: 143 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = F16
|
||||
print_info: file size = 1.07 GiB (20.70 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = llama
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 4096
|
||||
print_info: n_embd = 768
|
||||
print_info: n_layer = 20
|
||||
print_info: n_head = 12
|
||||
print_info: n_head_kv = 12
|
||||
print_info: n_rot = 64
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 64
|
||||
print_info: n_embd_head_v = 64
|
||||
print_info: n_gqa = 1
|
||||
print_info: n_embd_k_gqa = 768
|
||||
print_info: n_embd_v_gqa = 768
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 3072
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = 0
|
||||
print_info: rope type = 0
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 10000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 4096
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = ?B
|
||||
print_info: model params = 444.94 M
|
||||
print_info: general.name = Tts
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 158626
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
load_tensors: skipping TTS tensor emb_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor emb_text.weight (size = 467140608 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.weight (size = 12582912 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.weight (size = 2359296 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.weight (size = 6291456 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.weight (size = 1179648 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor head_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 0 repeating layers to GPU
|
||||
load_tensors: offloaded 0/21 layers to GPU
|
||||
load_tensors: CPU_Mapped model buffer size = 592.48 MiB
|
||||
..............................................................
|
||||
llama_init_from_model: model default pooling_type is [0], but [-1] was specified
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 10000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (4096) -- the full capacity of the model will not be utilized
|
||||
llama_context: CPU output buffer size = 0.61 MiB
|
||||
llama_kv_cache: CPU KV buffer size = 120.00 MiB
|
||||
llama_kv_cache: size = 120.00 MiB ( 2048 cells, 20 layers, 1/1 seqs), K (f16): 60.00 MiB, V (f16): 60.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 30.75 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 7.01 MiB
|
||||
llama_context: graph nodes = 613
|
||||
llama_context: graph splits = 216 (with bs=512), 1 (with bs=1)
|
||||
vision_model_loader: model name:
|
||||
vision_model_loader: description: image encoder for MiniCPM-V
|
||||
vision_model_loader: GGUF version: 3
|
||||
vision_model_loader: alignment: 32
|
||||
vision_model_loader: n_tensors: 455
|
||||
vision_model_loader: n_kv: 20
|
||||
|
||||
vision_ctx: vision using Vulkan0 backend
|
||||
load_hparams: model_type: 0
|
||||
load_hparams: n_embd: 1152
|
||||
load_hparams: n_head: 16
|
||||
load_hparams: n_ff: 4304
|
||||
load_hparams: n_layer: 27
|
||||
load_hparams: ffn_op: gelu
|
||||
load_hparams: projection_dim: 0
|
||||
|
||||
--- vision hparams ---
|
||||
load_hparams: image_size: 448
|
||||
load_hparams: patch_size: 14
|
||||
load_hparams: minicpmv_version: 100045
|
||||
load_hparams: proj_scale_factor: 0
|
||||
load_hparams: n_wa_pattern: 0
|
||||
|
||||
load_hparams: model size: 1044.36 MiB
|
||||
load_hparams: metadata size: 0.16 MiB
|
||||
load_tensors: ffn up/down are swapped
|
||||
alloc_compute_meta: Vulkan0 compute buffer size = 100.30 MiB
|
||||
alloc_compute_meta: CPU compute buffer size = 18.30 MiB
|
||||
init tts....init t2w....voc_hg2_model: CPU backend using 8 threads
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=32000
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=200, n_mel=80, total_size=16000
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=200, n_mel=80, total_elements=16000
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=200, n_tokens=100, current_iter=0, current_total=0, new_total=100, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 1 (total_tokens=100, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 20]
|
||||
audition_audio_batch_encode: Final output: 20 tokens x 4096 dims
|
||||
=== Model Paths ===
|
||||
Base dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf
|
||||
LLM: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf [OK]
|
||||
Vision: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/vision/MiniCPM-o-4_5-vision-F16.gguf [OK]
|
||||
Audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf [OK]
|
||||
TTS: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf [OK]
|
||||
Projector: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf [OK]
|
||||
===================
|
||||
=== Initializing Omni Context ===
|
||||
Media type: 2 (omni: audio+vision)
|
||||
TTS enabled: yes
|
||||
Context size: 2048
|
||||
GPU layers: 99
|
||||
Token2Wav device: cpu
|
||||
Vision backend: metal
|
||||
TTS bin dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts
|
||||
Ref audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\vllm-omni-edge\benchmarks\edge_harness\results\e2e_expansion_20260923\evidence\minicpmo_cpp_radeon890m_prompted\fixture_0000.wav
|
||||
02:53:01.871 === omni_init start
|
||||
02:53:01.872 media_type = 2, duplex_mode = 0, base_output_dir = ./tools/omni/output
|
||||
02:53:01.872 === omni_init: loading new LLM model
|
||||
02:53:53.980 === omni_init: loading TTS model
|
||||
02:53:53.980 TTS model: loading with n_gpu_layers=0
|
||||
02:53:57.461 TTS sampler: temp=0.80, top_p=0.85, top_k=25, rep_penalty=1.05
|
||||
02:53:57.461 TTS: loading weights from GGUF (emb_code, emb_text, projector_semantic, head_code)...
|
||||
02:54:03.004 TTS: head_code shape: dim0=6562, dim1=768, will transpose to [6562, 768]
|
||||
02:54:03.009 TTS: head_code loaded, verifying first few values:
|
||||
02:54:03.009 head_code_weight[0] = 0.01361084 (expect ~0.01385498)
|
||||
02:54:03.009 head_code_weight[1] = -0.01660156 (expect ~-0.01647949)
|
||||
02:54:03.009 head_code_weight[768] = 0.04101562 (this is weight[1, 0], expect ~0.04150391)
|
||||
02:54:03.058 TTS: weights loaded successfully
|
||||
02:54:03.058 Projector: loading from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf
|
||||
02:54:03.160 Projector: loaded successfully
|
||||
02:54:03.160 === omni_init: loading APM model
|
||||
02:54:06.243 APM: init from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf
|
||||
02:54:11.812 Token2Wav: found models in \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf
|
||||
02:54:11.867 Token2Wav: all model files found, initializing session...
|
||||
02:54:11.867 Token2Wav: non-CUDA backend, vocoder using CPU for better performance
|
||||
02:54:11.867 Token2Wav: using prompt_cache from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf/prompt_cache.gguf
|
||||
02:54:16.373 Token2Wav: initialized successfully
|
||||
02:54:16.373 Python T2W: script_dir=./tools/omni/pyt2w, model_dir=./tools/omni/pyt2w/token2wav
|
||||
02:54:16.373 Token2Wav: 浣跨敤 C++ 瀹炵幇
|
||||
02:54:16.373 === omni_init success: ctx_llama = 0000026a277efb00
|
||||
=== Running test case ===
|
||||
Audio prefix: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-18-j6a2dg5o\fixture_
|
||||
Count: 2
|
||||
02:54:16.374 stream_prefill: n_past = 0
|
||||
voice_clone_prompt = <|im_start|>system
|
||||
妯′豢闊抽�鏍锋湰鐨勯煶鑹插苟鐢熸垚鏂扮殑鍐呭�銆�
|
||||
<|audio_start|>
|
||||
assistant_prompt = <|audio_end|>浣犵殑浠诲姟鏄�敤杩欑�澹伴煶妯″紡鏉ュ綋涓€涓�姪鎵嬨€傝�璁ょ湡銆侀珮璐ㄩ噺鍦板洖澶嶇敤鎴风殑闂��銆傝�鐢ㄩ珮鑷�劧搴︾殑鏂瑰紡鍜岀敤鎴疯亰澶┿€�<|im_end|>
|
||||
<|im_start|>user
|
||||
|
||||
02:54:16.374 system prompt ref_audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-18-j6a2dg5o\fixture_0000.wav
|
||||
02:54:17.036 system prompt ref_audio embedding: n_pos=20
|
||||
02:54:17.543 馃敀 n_keep 璁剧疆涓� 72 (system prompt tokens)锛岃繖閮ㄥ垎姘歌繙涓嶄細琚�粦鍔ㄧ獥鍙ｅ垹闄�
|
||||
02:54:17.543 stream_prefill(index=0): system prompt 鍒濆�鍖栧畬鎴愶紝ref_audio 宸插湪鍏朵腑 prefill
|
||||
02:54:17.543 [SW] system_prompt registered: preserve_length=72 (will be protected from sliding)
|
||||
02:54:17.543 n_past = 72
|
||||
02:54:17.543
|
||||
|
||||
c++ finish stream_prefill(index=0). n_past=72, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 0 (audio) : 1.16958 s
|
||||
encode_image_with_vision_chunks: step 1 of 1 encoded in 623.64 ms
|
||||
encode_image_with_vision_chunks: all 1 chunks encoded in 641.11 ms (grid: 1x1)
|
||||
|
||||
encode_image_with_vision_chunks: image encoded in 641.11 ms by vision ( 641.11 ms per chunk, 64 total tokens)
|
||||
omni_image_embed_make_chunks_with_filename: created 1 vision chunks from C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-18-j6a2dg5o\fixture_0001.jpg
|
||||
stream_prefill: prefilled 1 vision chunks (64 tokens each)
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=49339
|
||||
audition_audio_preprocess: Audio not aligned to 100ms (49339 samples = 3083.7ms), padded with silence to 49600 samples (3100ms)
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=310, n_mel=80, total_size=24800
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=310, n_mel=80, total_elements=24800
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=310, n_tokens=155, current_iter=1, current_total=155, new_total=310, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 2 (total_tokens=310, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 31]
|
||||
audition_audio_batch_encode: Final output: 31 tokens x 4096 dims
|
||||
02:54:18.715 stream_prefill(index=1): processing user audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-18-j6a2dg5o\fixture_0001.wav
|
||||
02:54:19.005 stream_prefill(index=1): user audio embedding: n_pos=31
|
||||
02:54:19.324
|
||||
|
||||
c++ finish stream_prefill(index=1). n_past=172, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 1 (audio+vision) : 1.77855 s
|
||||
02:54:19.324 馃搷 stream_decode 寮€濮�: n_past=172, n_keep=72, n_ctx=2048, duplex_mode=0
|
||||
02:54:19.324 stream_decode: create tts thread (simplex mode)
|
||||
02:54:19.324 stream_decode: create t2w thread
|
||||
02:54:19.324 wait prefill done
|
||||
02:54:19.324 馃搷 [鍗曞伐TTS] 娣诲姞 assistant prompt: "<|im_end|>
|
||||
<|im_start|>assistant
|
||||
<think>
|
||||
|
||||
</think>
|
||||
|
||||
<|tts_bos|>", n_past=172
|
||||
02:54:19.325 T2W thread (C++) started
|
||||
<user>
|
||||
<assistant>02:54:19.327 TTS: 鍒涘缓鍗曞伐妯″紡杈撳嚭鐩�綍: ./tools/omni/output/round_000
|
||||
02:54:19.328 TTS thread started
|
||||
02:54:19.487 馃搷 [鍗曞伐TTS] assistant prompt 瀹屾垚, n_past=182
|
||||
02:54:19.487 LLM decode: max_tgt_len = 96, n_predict = 96, n_ctx = 2048
|
||||
02:54:20.034 LLM: detected end token, set llm_generation_done=true
|
||||
02:54:20.034 LLM->TTS: text='The square in the image is red.', n_tokens=8, hidden_size=32768, n_embd=4096, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:54:20.034 馃搷 Decode 缁撴潫: n_past=191, 鍓╀綑绌洪棿=1857, 鏃犻渶婊戠獥
|
||||
02:54:20.034 馃搷 杞�� 1 缁撴潫锛岃�褰曡竟鐣屼簬 n_past=191
|
||||
02:54:20.034 TTS<-LLM: chunk_idx=0, text='The square in the image is red.', n_tokens=8, hidden_size=32768, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:54:20.034 TTS: after queue pop - speek_done=0, llm_finish=1, llm_text.empty=0, token_ids.size=8
|
||||
02:54:20.034 TTS: before empty check - response.empty=0, response='The square in the image is red.', llm_finish=1
|
||||
Waiting for audio generation to complete...
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_0.wav does not exist or is empty
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_1.wav does not exist or is empty
|
||||
TTS: no valid WAV files to merge
|
||||
02:54:20.034 TTS: DEBUG before has_llm_data - token_ids.size=8, hidden_states.size=32768, n_embd=4096
|
||||
02:54:20.034 TTS: has_llm_data=1, token_ids=8, hidden_states=32768, n_embd=4096, llm_finish=1
|
||||
02:54:20.035 TTS: n_tokens_orig=8, n_tokens_filtered=8 (filtered 0 special tokens)
|
||||
02:54:20.068 TTS: saved merged embeddings (text) to ./tools/omni/output/round_000/llm_debug/chunk_0/merged_embeddings.txt
|
||||
02:54:20.068 TTS Simplex: generating audio tokens for chunk 0 (n_tokens=8, tts_n_embd=768)
|
||||
02:54:20.068 TTS Simplex: 鍦� prefill 鍓嶆坊鍔� audio_bos (chunk_idx=0, new_size=9)
|
||||
02:54:20.068 TTS Simplex: first chunk - cleared KV cache
|
||||
02:54:20.068 TTS Simplex: first chunk - cleared tts_all_generated_tokens and tts_token_buffer
|
||||
02:54:20.129 馃搷 涓轰笅涓€杞�噯澶�: eval <|im_end|>\n<|im_start|>user\n, n_past=196
|
||||
02:54:20.328 TTS Simplex: prefill completed, n_past_tts=9
|
||||
02:54:20.328 TTS Simplex: sampler created
|
||||
02:54:20.328 TTS simplex: is_audio_bos=true (first audio token)
|
||||
02:54:20.353 TTS Simplex Phase1: token 1/500: rel_id=770
|
||||
02:54:20.364 TTS Simplex Phase1: token 2/500: rel_id=4218
|
||||
02:54:20.374 TTS Simplex Phase1: token 3/500: rel_id=4218
|
||||
02:54:20.386 TTS Simplex Phase1: token 4/500: rel_id=4218
|
||||
02:54:20.396 TTS Simplex Phase1: token 5/500: rel_id=4137
|
||||
02:54:20.615 TTS Simplex Phase1: yield 25 tokens 鍒� T2W (step 25, buffer=25)
|
||||
02:54:20.615 TTS Simplex Phase1: token 25/500: rel_id=829
|
||||
02:54:20.615 T2W绾跨▼(C++): 杞��鍒囨崲 (-1 -> 0)锛屾洿鏂拌緭鍑虹洰褰�
|
||||
02:54:20.615 T2W绾跨▼(C++): 鏂拌緭鍑虹洰褰� ./tools/omni/output/round_000/tts_wav
|
||||
02:54:20.793 TTS Simplex Phase1: EOS token at step 42
|
||||
02:54:20.793 TTS Simplex: is_final_text_chunk=true, will enter Phase 2 for text_eos_embed
|
||||
02:54:20.793 TTS Simplex Phase2: injecting text_eos_embed + audio_bos at n_past=50
|
||||
02:54:20.800 TTS Simplex Phase2: text_eos_embed injected, n_past=51
|
||||
02:54:20.808 TTS Simplex Phase2: audio_bos injected, n_past=52, starting final generation
|
||||
02:54:20.904 TTS Simplex Phase2: yield 25 tokens 鍒� T2W
|
||||
02:54:21.026 TTS Simplex Phase2: EOS at step 20 (final end)
|
||||
02:54:21.026 TTS Simplex: chunk 缁撴潫锛宼ts_token_buffer 鍓╀綑 10 tokens (淇濈暀绛変笅涓€涓� chunk)
|
||||
02:54:21.026 TTS Simplex: is_final_text_chunk=true, flush 鍓╀綑 10 tokens 浠� tts_token_buffer
|
||||
02:54:21.026 TTS Simplex: generated 60 audio tokens for chunk 0
|
||||
02:54:21.026 TTS Simplex: chunk 0 first tokens: [770 4218 4218 4218 4137 ], last tokens: [7 735 4218 4218 1950 ]
|
||||
02:54:21.026 TTS Simplex: updated tts_n_past_accumulated=72, total_generated_tokens=60
|
||||
02:54:21.027 TTS: finished processing all chunks
|
||||
02:54:21.027 TTS: 鍗曞伐妯″紡杞��缁撴潫锛屼笅涓€杞� round_idx=1
|
||||
02:54:21.027 TTS: sent is_final=true to T2W queue (turn end)
|
||||
02:54:22.940 馃帀 棣栧搷鏃堕棿 (First Audio Response): 3616ms
|
||||
02:54:22.940 T2W绾跨▼: wav_0.wav | 0.84s audio | 2323.9ms inference | RTF=2.77 | t=3616ms
|
||||
02:54:25.157 T2W绾跨▼: wav_1.wav | 1.00s audio | 2215.5ms inference | RTF=2.22 | t=5832ms
|
||||
02:54:27.277 T2W绾跨▼: wav_2.wav | 0.68s audio | 2119.9ms inference | RTF=3.12 | t=7953ms
|
||||
02:54:27.278 T2W绾跨▼: 鍐欏叆缁撴潫鏍囪� ./tools/omni/output/round_000/tts_wav/generation_done.flag (last_wav=2)
|
||||
02:54:27.278 T2W绾跨▼: 杞��缁撴潫鍚庢洿鏂拌緭鍑虹洰褰曚负 ./tools/omni/output/round_001/tts_wav
|
||||
02:54:02:27.306 omni_stop_threads: stop signals sent
|
||||
54:27.306 T2W(C++) 绾跨▼: 鍋滄�
|
||||
[timing] call=0 tokens=28 final=0 token2mel=1993.339ms vocoder=330.501ms total=2323.855ms audio=20160
|
||||
[timing] call=1 tokens=28 final=0 token2mel=1838.655ms vocoder=376.862ms total=2215.528ms audio=24000
|
||||
[timing] call=2 tokens=13 final=1 token2mel=1867.828ms vocoder=252.075ms total=2119.913ms audio=16320
|
||||
Audio generation completed.
|
||||
llama_perf_context_print: load time = 25382.23 ms
|
||||
llama_perf_context_print: prompt eval time = 3112.86 ms / 182 tokens ( 17.10 ms per token, 58.47 tokens per second)
|
||||
llama_perf_context_print: eval time = 543.75 ms / 9 runs ( 60.42 ms per token, 16.55 tokens per second)
|
||||
llama_perf_context_print: total time = 33201.14 ms / 191 tokens
|
||||
llama_perf_context_print: graphs reused = 10
|
||||
Token2Wav (C++): session released
|
||||
tts thread end
|
||||
t2w thread end
|
||||
+483
@@ -0,0 +1,483 @@
|
||||
build: 64 (5202b7b) with x86_64-w64-mingw32-gcc-posix (GCC) 13-posix for x86_64-w64-mingw32
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 30720 MiB free
|
||||
llama_model_loader: loaded meta data with 27 key-value pairs and 399 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = qwen3
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Llm
|
||||
llama_model_loader: - kv 3: general.size_label str = 8.2B
|
||||
llama_model_loader: - kv 4: qwen3.block_count u32 = 36
|
||||
llama_model_loader: - kv 5: qwen3.context_length u32 = 40960
|
||||
llama_model_loader: - kv 6: qwen3.embedding_length u32 = 4096
|
||||
llama_model_loader: - kv 7: qwen3.feed_forward_length u32 = 12288
|
||||
llama_model_loader: - kv 8: qwen3.attention.head_count u32 = 32
|
||||
llama_model_loader: - kv 9: qwen3.attention.head_count_kv u32 = 8
|
||||
llama_model_loader: - kv 10: qwen3.rope.freq_base f32 = 1000000.000000
|
||||
llama_model_loader: - kv 11: qwen3.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: qwen3.attention.key_length u32 = 128
|
||||
llama_model_loader: - kv 13: qwen3.attention.value_length u32 = 128
|
||||
llama_model_loader: - kv 14: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 15: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 16: tokenizer.ggml.tokens arr[str,151748] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 17: tokenizer.ggml.token_type arr[i32,151748] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 24: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 25: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 26: general.file_type u32 = 15
|
||||
llama_model_loader: - type f32: 145 tensors
|
||||
llama_model_loader: - type q4_K: 217 tensors
|
||||
llama_model_loader: - type q6_K: 37 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = Q4_K - Medium
|
||||
print_info: file size = 4.68 GiB (4.90 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = qwen3
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 40960
|
||||
print_info: n_embd = 4096
|
||||
print_info: n_layer = 36
|
||||
print_info: n_head = 32
|
||||
print_info: n_head_kv = 8
|
||||
print_info: n_rot = 128
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 128
|
||||
print_info: n_embd_head_v = 128
|
||||
print_info: n_gqa = 4
|
||||
print_info: n_embd_k_gqa = 1024
|
||||
print_info: n_embd_v_gqa = 1024
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 12288
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = -1
|
||||
print_info: rope type = 2
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 1000000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 40960
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = 8B
|
||||
print_info: model params = 8.19 B
|
||||
print_info: general.name = Llm
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 151748
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 36 repeating layers to GPU
|
||||
load_tensors: offloading output layer to GPU
|
||||
load_tensors: offloaded 37/37 layers to GPU
|
||||
load_tensors: Vulkan0 model buffer size = 4454.74 MiB
|
||||
load_tensors: CPU_Mapped model buffer size = 333.43 MiB
|
||||
.....................................................................................
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 1000000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (40960) -- the full capacity of the model will not be utilized
|
||||
llama_context: Vulkan_Host output buffer size = 0.58 MiB
|
||||
llama_kv_cache: Vulkan0 KV buffer size = 288.00 MiB
|
||||
llama_kv_cache: size = 288.00 MiB ( 2048 cells, 36 layers, 1/1 seqs), K (f16): 144.00 MiB, V (f16): 144.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 304.38 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 12.01 MiB
|
||||
llama_context: graph nodes = 1267
|
||||
llama_context: graph splits = 2
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 25673 MiB free
|
||||
llama_model_loader: loaded meta data with 30 key-value pairs and 193 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = llama
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Tts
|
||||
llama_model_loader: - kv 3: general.size_label str = 445M
|
||||
llama_model_loader: - kv 4: llama.block_count u32 = 20
|
||||
llama_model_loader: - kv 5: llama.context_length u32 = 4096
|
||||
llama_model_loader: - kv 6: llama.embedding_length u32 = 768
|
||||
llama_model_loader: - kv 7: llama.feed_forward_length u32 = 3072
|
||||
llama_model_loader: - kv 8: llama.attention.head_count u32 = 12
|
||||
llama_model_loader: - kv 9: llama.attention.head_count_kv u32 = 12
|
||||
llama_model_loader: - kv 10: llama.rope.freq_base f32 = 10000.000000
|
||||
llama_model_loader: - kv 11: llama.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: llama.attention.key_length u32 = 64
|
||||
llama_model_loader: - kv 13: llama.attention.value_length u32 = 64
|
||||
llama_model_loader: - kv 14: general.file_type u32 = 1
|
||||
llama_model_loader: - kv 15: llama.vocab_size u32 = 158626
|
||||
llama_model_loader: - kv 16: llama.rope.dimension_count u32 = 64
|
||||
llama_model_loader: - kv 17: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.tokens arr[str,158626] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.token_type arr[i32,158626] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 24: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 25: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 26: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 27: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 28: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 29: tokenizer.ggml.add_space_prefix bool = false
|
||||
llama_model_loader: - type f32: 50 tensors
|
||||
llama_model_loader: - type f16: 143 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = F16
|
||||
print_info: file size = 1.07 GiB (20.70 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = llama
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 4096
|
||||
print_info: n_embd = 768
|
||||
print_info: n_layer = 20
|
||||
print_info: n_head = 12
|
||||
print_info: n_head_kv = 12
|
||||
print_info: n_rot = 64
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 64
|
||||
print_info: n_embd_head_v = 64
|
||||
print_info: n_gqa = 1
|
||||
print_info: n_embd_k_gqa = 768
|
||||
print_info: n_embd_v_gqa = 768
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 3072
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = 0
|
||||
print_info: rope type = 0
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 10000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 4096
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = ?B
|
||||
print_info: model params = 444.94 M
|
||||
print_info: general.name = Tts
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 158626
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
load_tensors: skipping TTS tensor emb_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor emb_text.weight (size = 467140608 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.weight (size = 12582912 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.weight (size = 2359296 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.weight (size = 6291456 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.weight (size = 1179648 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor head_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 0 repeating layers to GPU
|
||||
load_tensors: offloaded 0/21 layers to GPU
|
||||
load_tensors: CPU_Mapped model buffer size = 592.48 MiB
|
||||
..............................................................
|
||||
llama_init_from_model: model default pooling_type is [0], but [-1] was specified
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 10000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (4096) -- the full capacity of the model will not be utilized
|
||||
llama_context: CPU output buffer size = 0.61 MiB
|
||||
llama_kv_cache: CPU KV buffer size = 120.00 MiB
|
||||
llama_kv_cache: size = 120.00 MiB ( 2048 cells, 20 layers, 1/1 seqs), K (f16): 60.00 MiB, V (f16): 60.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 30.75 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 7.01 MiB
|
||||
llama_context: graph nodes = 613
|
||||
llama_context: graph splits = 216 (with bs=512), 1 (with bs=1)
|
||||
vision_model_loader: model name:
|
||||
vision_model_loader: description: image encoder for MiniCPM-V
|
||||
vision_model_loader: GGUF version: 3
|
||||
vision_model_loader: alignment: 32
|
||||
vision_model_loader: n_tensors: 455
|
||||
vision_model_loader: n_kv: 20
|
||||
|
||||
vision_ctx: vision using Vulkan0 backend
|
||||
load_hparams: model_type: 0
|
||||
load_hparams: n_embd: 1152
|
||||
load_hparams: n_head: 16
|
||||
load_hparams: n_ff: 4304
|
||||
load_hparams: n_layer: 27
|
||||
load_hparams: ffn_op: gelu
|
||||
load_hparams: projection_dim: 0
|
||||
|
||||
--- vision hparams ---
|
||||
load_hparams: image_size: 448
|
||||
load_hparams: patch_size: 14
|
||||
load_hparams: minicpmv_version: 100045
|
||||
load_hparams: proj_scale_factor: 0
|
||||
load_hparams: n_wa_pattern: 0
|
||||
|
||||
load_hparams: model size: 1044.36 MiB
|
||||
load_hparams: metadata size: 0.16 MiB
|
||||
load_tensors: ffn up/down are swapped
|
||||
alloc_compute_meta: Vulkan0 compute buffer size = 100.30 MiB
|
||||
alloc_compute_meta: CPU compute buffer size = 18.30 MiB
|
||||
init tts....init t2w....voc_hg2_model: CPU backend using 8 threads
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=32000
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=200, n_mel=80, total_size=16000
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=200, n_mel=80, total_elements=16000
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=200, n_tokens=100, current_iter=0, current_total=0, new_total=100, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 1 (total_tokens=100, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 20]
|
||||
audition_audio_batch_encode: Final output: 20 tokens x 4096 dims
|
||||
=== Model Paths ===
|
||||
Base dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf
|
||||
LLM: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf [OK]
|
||||
Vision: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/vision/MiniCPM-o-4_5-vision-F16.gguf [OK]
|
||||
Audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf [OK]
|
||||
TTS: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf [OK]
|
||||
Projector: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf [OK]
|
||||
===================
|
||||
=== Initializing Omni Context ===
|
||||
Media type: 2 (omni: audio+vision)
|
||||
TTS enabled: yes
|
||||
Context size: 2048
|
||||
GPU layers: 99
|
||||
Token2Wav device: cpu
|
||||
Vision backend: metal
|
||||
TTS bin dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts
|
||||
Ref audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\vllm-omni-edge\benchmarks\edge_harness\results\e2e_expansion_20260923\evidence\minicpmo_cpp_radeon890m_prompted\fixture_0000.wav
|
||||
02:54:29.284 === omni_init start
|
||||
02:54:29.285 media_type = 2, duplex_mode = 0, base_output_dir = ./tools/omni/output
|
||||
02:54:29.285 === omni_init: loading new LLM model
|
||||
02:55:18.598 === omni_init: loading TTS model
|
||||
02:55:18.598 TTS model: loading with n_gpu_layers=0
|
||||
02:55:22.298 TTS sampler: temp=0.80, top_p=0.85, top_k=25, rep_penalty=1.05
|
||||
02:55:22.298 TTS: loading weights from GGUF (emb_code, emb_text, projector_semantic, head_code)...
|
||||
02:55:27.768 TTS: head_code shape: dim0=6562, dim1=768, will transpose to [6562, 768]
|
||||
02:55:27.772 TTS: head_code loaded, verifying first few values:
|
||||
02:55:27.772 head_code_weight[0] = 0.01361084 (expect ~0.01385498)
|
||||
02:55:27.772 head_code_weight[1] = -0.01660156 (expect ~-0.01647949)
|
||||
02:55:27.772 head_code_weight[768] = 0.04101562 (this is weight[1, 0], expect ~0.04150391)
|
||||
02:55:27.822 TTS: weights loaded successfully
|
||||
02:55:27.822 Projector: loading from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf
|
||||
02:55:27.922 Projector: loaded successfully
|
||||
02:55:27.922 === omni_init: loading APM model
|
||||
02:55:30.974 APM: init from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf
|
||||
02:55:36.460 Token2Wav: found models in \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf
|
||||
02:55:36.511 Token2Wav: all model files found, initializing session...
|
||||
02:55:36.511 Token2Wav: non-CUDA backend, vocoder using CPU for better performance
|
||||
02:55:36.511 Token2Wav: using prompt_cache from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf/prompt_cache.gguf
|
||||
02:55:41.024 Token2Wav: initialized successfully
|
||||
02:55:41.024 Python T2W: script_dir=./tools/omni/pyt2w, model_dir=./tools/omni/pyt2w/token2wav
|
||||
02:55:41.024 Token2Wav: 浣跨敤 C++ 瀹炵幇
|
||||
02:55:41.024 === omni_init success: ctx_llama = 0000020a8eedf830
|
||||
=== Running test case ===
|
||||
Audio prefix: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-19-8_qhill3\fixture_
|
||||
Count: 2
|
||||
02:55:41.025 stream_prefill: n_past = 0
|
||||
voice_clone_prompt = <|im_start|>system
|
||||
妯′豢闊抽�鏍锋湰鐨勯煶鑹插苟鐢熸垚鏂扮殑鍐呭�銆�
|
||||
<|audio_start|>
|
||||
assistant_prompt = <|audio_end|>浣犵殑浠诲姟鏄�敤杩欑�澹伴煶妯″紡鏉ュ綋涓€涓�姪鎵嬨€傝�璁ょ湡銆侀珮璐ㄩ噺鍦板洖澶嶇敤鎴风殑闂��銆傝�鐢ㄩ珮鑷�劧搴︾殑鏂瑰紡鍜岀敤鎴疯亰澶┿€�<|im_end|>
|
||||
<|im_start|>user
|
||||
|
||||
02:55:41.025 system prompt ref_audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-19-8_qhill3\fixture_0000.wav
|
||||
02:55:41.692 system prompt ref_audio embedding: n_pos=20
|
||||
02:55:42.194 馃敀 n_keep 璁剧疆涓� 72 (system prompt tokens)锛岃繖閮ㄥ垎姘歌繙涓嶄細琚�粦鍔ㄧ獥鍙ｅ垹闄�
|
||||
02:55:42.194 stream_prefill(index=0): system prompt 鍒濆�鍖栧畬鎴愶紝ref_audio 宸插湪鍏朵腑 prefill
|
||||
02:55:42.194 [SW] system_prompt registered: preserve_length=72 (will be protected from sliding)
|
||||
02:55:42.194 n_past = 72
|
||||
02:55:42.194
|
||||
|
||||
c++ finish stream_prefill(index=0). n_past=72, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 0 (audio) : 1.16932 s
|
||||
encode_image_with_vision_chunks: step 1 of 1 encoded in 617.48 ms
|
||||
encode_image_with_vision_chunks: all 1 chunks encoded in 635.95 ms (grid: 1x1)
|
||||
|
||||
encode_image_with_vision_chunks: image encoded in 635.96 ms by vision ( 635.96 ms per chunk, 64 total tokens)
|
||||
omni_image_embed_make_chunks_with_filename: created 1 vision chunks from C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-19-8_qhill3\fixture_0001.jpg
|
||||
stream_prefill: prefilled 1 vision chunks (64 tokens each)
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=49339
|
||||
audition_audio_preprocess: Audio not aligned to 100ms (49339 samples = 3083.7ms), padded with silence to 49600 samples (3100ms)
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=310, n_mel=80, total_size=24800
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=310, n_mel=80, total_elements=24800
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=310, n_tokens=155, current_iter=1, current_total=155, new_total=310, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 2 (total_tokens=310, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 31]
|
||||
audition_audio_batch_encode: Final output: 31 tokens x 4096 dims
|
||||
02:55:43.363 stream_prefill(index=1): processing user audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-19-8_qhill3\fixture_0001.wav
|
||||
02:55:43.665 stream_prefill(index=1): user audio embedding: n_pos=31
|
||||
02:55:43.978
|
||||
|
||||
c++ finish stream_prefill(index=1). n_past=172, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 1 (audio+vision) : 1.78203 s
|
||||
02:55:43.978 馃搷 stream_decode 寮€濮�: n_past=172, n_keep=72, n_ctx=2048, duplex_mode=0
|
||||
02:55:43.978 stream_decode: create tts thread (simplex mode)
|
||||
02:55:43.978 stream_decode: create t2w thread
|
||||
02:55:43.978 wait prefill done
|
||||
02:55:43.978 馃搷 [鍗曞伐TTS] 娣诲姞 assistant prompt: "<|im_end|>
|
||||
<|im_start|>assistant
|
||||
<think>
|
||||
|
||||
</think>
|
||||
|
||||
<|tts_bos|>", n_past=172
|
||||
02:55:43.978 T2W thread (C++) started
|
||||
<user>
|
||||
<assistant>02:55:43.982 TTS: 鍒涘缓鍗曞伐妯″紡杈撳嚭鐩�綍: ./tools/omni/output/round_000
|
||||
02:55:43.983 TTS thread started
|
||||
02:55:44.144 馃搷 [鍗曞伐TTS] assistant prompt 瀹屾垚, n_past=182
|
||||
02:55:44.144 LLM decode: max_tgt_len = 96, n_predict = 96, n_ctx = 2048
|
||||
02:55:44.664 LLM: detected end token, set llm_generation_done=true
|
||||
02:55:44.664 LLM->TTS: text='The square in the image is red.', n_tokens=8, hidden_size=32768, n_embd=4096, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:55:44.664 馃搷 Decode 缁撴潫: n_past=191, 鍓╀綑绌洪棿=1857, 鏃犻渶婊戠獥
|
||||
02:55:44.664 馃搷 杞�� 1 缁撴潫锛岃�褰曡竟鐣屼簬 n_past=191
|
||||
02:55:44.665 TTS<-LLM: chunk_idx=0, text='The square in the image is red.', n_tokens=8, hidden_size=32768, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:55:44.665 TTS: after queue pop - speek_done=0, llm_finish=1, llm_text.empty=0, token_ids.size=8
|
||||
02:55:44.665 TTS: before empty check - response.empty=0, response='The square in the image is red.', llm_finish=1
|
||||
Waiting for audio generation to complete...
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_0.wav does not exist or is empty
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_1.wav does not exist or is empty
|
||||
TTS: no valid WAV files to merge
|
||||
02:55:44.665 TTS: DEBUG before has_llm_data - token_ids.size=8, hidden_states.size=32768, n_embd=4096
|
||||
02:55:44.665 TTS: has_llm_data=1, token_ids=8, hidden_states=32768, n_embd=4096, llm_finish=1
|
||||
02:55:44.666 TTS: n_tokens_orig=8, n_tokens_filtered=8 (filtered 0 special tokens)
|
||||
02:55:44.695 TTS: saved merged embeddings (text) to ./tools/omni/output/round_000/llm_debug/chunk_0/merged_embeddings.txt
|
||||
02:55:44.695 TTS Simplex: generating audio tokens for chunk 0 (n_tokens=8, tts_n_embd=768)
|
||||
02:55:44.695 TTS Simplex: 鍦� prefill 鍓嶆坊鍔� audio_bos (chunk_idx=0, new_size=9)
|
||||
02:55:44.695 TTS Simplex: first chunk - cleared KV cache
|
||||
02:55:44.695 TTS Simplex: first chunk - cleared tts_all_generated_tokens and tts_token_buffer
|
||||
02:55:44.758 馃搷 涓轰笅涓€杞�噯澶�: eval <|im_end|>\n<|im_start|>user\n, n_past=196
|
||||
02:55:44.969 TTS Simplex: prefill completed, n_past_tts=9
|
||||
02:55:44.969 TTS Simplex: sampler created
|
||||
02:55:44.969 TTS simplex: is_audio_bos=true (first audio token)
|
||||
02:55:44.995 TTS Simplex Phase1: token 1/500: rel_id=1487
|
||||
02:55:45.005 TTS Simplex Phase1: token 2/500: rel_id=4218
|
||||
02:55:45.016 TTS Simplex Phase1: token 3/500: rel_id=4218
|
||||
02:55:45.027 TTS Simplex Phase1: token 4/500: rel_id=4218
|
||||
02:55:45.038 TTS Simplex Phase1: token 5/500: rel_id=4137
|
||||
02:55:45.256 TTS Simplex Phase1: yield 25 tokens 鍒� T2W (step 25, buffer=25)
|
||||
02:55:45.256 TTS Simplex Phase1: token 25/500: rel_id=2573
|
||||
02:55:45.256 T2W绾跨▼(C++): 杞��鍒囨崲 (-1 -> 0)锛屾洿鏂拌緭鍑虹洰褰�
|
||||
02:55:45.256 T2W绾跨▼(C++): 鏂拌緭鍑虹洰褰� ./tools/omni/output/round_000/tts_wav
|
||||
02:55:45.260 TTS Simplex Phase1: EOS token at step 26
|
||||
02:55:45.260 TTS Simplex: is_final_text_chunk=true, will enter Phase 2 for text_eos_embed
|
||||
02:55:45.260 TTS Simplex Phase2: injecting text_eos_embed + audio_bos at n_past=34
|
||||
02:55:45.267 TTS Simplex Phase2: text_eos_embed injected, n_past=35
|
||||
02:55:45.274 TTS Simplex Phase2: audio_bos injected, n_past=36, starting final generation
|
||||
02:55:45.554 TTS Simplex Phase2: yield 25 tokens 鍒� T2W
|
||||
02:55:45.642 TTS Simplex Phase2: EOS at step 33 (final end)
|
||||
02:55:45.642 TTS Simplex: chunk 缁撴潫锛宼ts_token_buffer 鍓╀綑 7 tokens (淇濈暀绛変笅涓€涓� chunk)
|
||||
02:55:45.642 TTS Simplex: is_final_text_chunk=true, flush 鍓╀綑 7 tokens 浠� tts_token_buffer
|
||||
02:55:45.642 TTS Simplex: generated 57 audio tokens for chunk 0
|
||||
02:55:45.642 TTS Simplex: chunk 0 first tokens: [1487 4218 4218 4218 4137 ], last tokens: [4218 4218 4137 4137 4137 ]
|
||||
02:55:45.642 TTS Simplex: updated tts_n_past_accumulated=69, total_generated_tokens=57
|
||||
02:55:45.643 TTS: finished processing all chunks
|
||||
02:55:45.644 TTS: 鍗曞伐妯″紡杞��缁撴潫锛屼笅涓€杞� round_idx=1
|
||||
02:55:45.644 TTS: sent is_final=true to T2W queue (turn end)
|
||||
02:55:47.594 馃帀 棣栧搷鏃堕棿 (First Audio Response): 3615ms
|
||||
02:55:47.594 T2W绾跨▼: wav_0.wav | 0.84s audio | 2336.4ms inference | RTF=2.78 | t=3615ms
|
||||
02:55:49.836 T2W绾跨▼: wav_1.wav | 1.00s audio | 2240.9ms inference | RTF=2.24 | t=5857ms
|
||||
02:55:51.925 T2W绾跨▼: wav_2.wav | 0.56s audio | 2088.6ms inference | RTF=3.73 | t=7946ms
|
||||
02:55:51.925 T2W绾跨▼: 鍐欏叆缁撴潫鏍囪� ./tools/omni/output/round_000/tts_wav/generation_done.flag (last_wav=2)
|
||||
02:55:51.925 T2W绾跨▼: 杞��缁撴潫鍚庢洿鏂拌緭鍑虹洰褰曚负 ./tools/omni/output/round_001/tts_wav
|
||||
02:55:02:55:51.956 omni_stop_threads: stop signals sent
|
||||
51.956 T2W(C++) 绾跨▼: 鍋滄�
|
||||
[timing] call=0 tokens=28 final=0 token2mel=2003.987ms vocoder=332.337ms total=2336.342ms audio=20160
|
||||
[timing] call=1 tokens=28 final=0 token2mel=1853.127ms vocoder=387.740ms total=2240.893ms audio=24000
|
||||
[timing] call=2 tokens=10 final=1 token2mel=1881.986ms vocoder=206.545ms total=2088.541ms audio=13440
|
||||
Audio generation completed.
|
||||
llama_perf_context_print: load time = 25425.01 ms
|
||||
llama_perf_context_print: prompt eval time = 3118.14 ms / 182 tokens ( 17.13 ms per token, 58.37 tokens per second)
|
||||
llama_perf_context_print: eval time = 517.87 ms / 9 runs ( 57.54 ms per token, 17.38 tokens per second)
|
||||
llama_perf_context_print: total time = 33237.41 ms / 191 tokens
|
||||
llama_perf_context_print: graphs reused = 10
|
||||
Token2Wav (C++): session released
|
||||
tts thread end
|
||||
t2w thread end
|
||||
+483
@@ -0,0 +1,483 @@
|
||||
build: 64 (5202b7b) with x86_64-w64-mingw32-gcc-posix (GCC) 13-posix for x86_64-w64-mingw32
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 30720 MiB free
|
||||
llama_model_loader: loaded meta data with 27 key-value pairs and 399 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = qwen3
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Llm
|
||||
llama_model_loader: - kv 3: general.size_label str = 8.2B
|
||||
llama_model_loader: - kv 4: qwen3.block_count u32 = 36
|
||||
llama_model_loader: - kv 5: qwen3.context_length u32 = 40960
|
||||
llama_model_loader: - kv 6: qwen3.embedding_length u32 = 4096
|
||||
llama_model_loader: - kv 7: qwen3.feed_forward_length u32 = 12288
|
||||
llama_model_loader: - kv 8: qwen3.attention.head_count u32 = 32
|
||||
llama_model_loader: - kv 9: qwen3.attention.head_count_kv u32 = 8
|
||||
llama_model_loader: - kv 10: qwen3.rope.freq_base f32 = 1000000.000000
|
||||
llama_model_loader: - kv 11: qwen3.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: qwen3.attention.key_length u32 = 128
|
||||
llama_model_loader: - kv 13: qwen3.attention.value_length u32 = 128
|
||||
llama_model_loader: - kv 14: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 15: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 16: tokenizer.ggml.tokens arr[str,151748] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 17: tokenizer.ggml.token_type arr[i32,151748] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 24: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 25: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 26: general.file_type u32 = 15
|
||||
llama_model_loader: - type f32: 145 tensors
|
||||
llama_model_loader: - type q4_K: 217 tensors
|
||||
llama_model_loader: - type q6_K: 37 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = Q4_K - Medium
|
||||
print_info: file size = 4.68 GiB (4.90 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = qwen3
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 40960
|
||||
print_info: n_embd = 4096
|
||||
print_info: n_layer = 36
|
||||
print_info: n_head = 32
|
||||
print_info: n_head_kv = 8
|
||||
print_info: n_rot = 128
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 128
|
||||
print_info: n_embd_head_v = 128
|
||||
print_info: n_gqa = 4
|
||||
print_info: n_embd_k_gqa = 1024
|
||||
print_info: n_embd_v_gqa = 1024
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 12288
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = -1
|
||||
print_info: rope type = 2
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 1000000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 40960
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = 8B
|
||||
print_info: model params = 8.19 B
|
||||
print_info: general.name = Llm
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 151748
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 36 repeating layers to GPU
|
||||
load_tensors: offloading output layer to GPU
|
||||
load_tensors: offloaded 37/37 layers to GPU
|
||||
load_tensors: Vulkan0 model buffer size = 4454.74 MiB
|
||||
load_tensors: CPU_Mapped model buffer size = 333.43 MiB
|
||||
.....................................................................................
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 1000000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (40960) -- the full capacity of the model will not be utilized
|
||||
llama_context: Vulkan_Host output buffer size = 0.58 MiB
|
||||
llama_kv_cache: Vulkan0 KV buffer size = 288.00 MiB
|
||||
llama_kv_cache: size = 288.00 MiB ( 2048 cells, 36 layers, 1/1 seqs), K (f16): 144.00 MiB, V (f16): 144.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 304.38 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 12.01 MiB
|
||||
llama_context: graph nodes = 1267
|
||||
llama_context: graph splits = 2
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 25673 MiB free
|
||||
llama_model_loader: loaded meta data with 30 key-value pairs and 193 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = llama
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Tts
|
||||
llama_model_loader: - kv 3: general.size_label str = 445M
|
||||
llama_model_loader: - kv 4: llama.block_count u32 = 20
|
||||
llama_model_loader: - kv 5: llama.context_length u32 = 4096
|
||||
llama_model_loader: - kv 6: llama.embedding_length u32 = 768
|
||||
llama_model_loader: - kv 7: llama.feed_forward_length u32 = 3072
|
||||
llama_model_loader: - kv 8: llama.attention.head_count u32 = 12
|
||||
llama_model_loader: - kv 9: llama.attention.head_count_kv u32 = 12
|
||||
llama_model_loader: - kv 10: llama.rope.freq_base f32 = 10000.000000
|
||||
llama_model_loader: - kv 11: llama.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: llama.attention.key_length u32 = 64
|
||||
llama_model_loader: - kv 13: llama.attention.value_length u32 = 64
|
||||
llama_model_loader: - kv 14: general.file_type u32 = 1
|
||||
llama_model_loader: - kv 15: llama.vocab_size u32 = 158626
|
||||
llama_model_loader: - kv 16: llama.rope.dimension_count u32 = 64
|
||||
llama_model_loader: - kv 17: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.tokens arr[str,158626] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.token_type arr[i32,158626] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 24: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 25: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 26: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 27: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 28: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 29: tokenizer.ggml.add_space_prefix bool = false
|
||||
llama_model_loader: - type f32: 50 tensors
|
||||
llama_model_loader: - type f16: 143 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = F16
|
||||
print_info: file size = 1.07 GiB (20.70 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = llama
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 4096
|
||||
print_info: n_embd = 768
|
||||
print_info: n_layer = 20
|
||||
print_info: n_head = 12
|
||||
print_info: n_head_kv = 12
|
||||
print_info: n_rot = 64
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 64
|
||||
print_info: n_embd_head_v = 64
|
||||
print_info: n_gqa = 1
|
||||
print_info: n_embd_k_gqa = 768
|
||||
print_info: n_embd_v_gqa = 768
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 3072
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = 0
|
||||
print_info: rope type = 0
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 10000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 4096
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = ?B
|
||||
print_info: model params = 444.94 M
|
||||
print_info: general.name = Tts
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 158626
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
load_tensors: skipping TTS tensor emb_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor emb_text.weight (size = 467140608 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.weight (size = 12582912 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.weight (size = 2359296 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.weight (size = 6291456 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.weight (size = 1179648 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor head_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 0 repeating layers to GPU
|
||||
load_tensors: offloaded 0/21 layers to GPU
|
||||
load_tensors: CPU_Mapped model buffer size = 592.48 MiB
|
||||
..............................................................
|
||||
llama_init_from_model: model default pooling_type is [0], but [-1] was specified
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 10000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (4096) -- the full capacity of the model will not be utilized
|
||||
llama_context: CPU output buffer size = 0.61 MiB
|
||||
llama_kv_cache: CPU KV buffer size = 120.00 MiB
|
||||
llama_kv_cache: size = 120.00 MiB ( 2048 cells, 20 layers, 1/1 seqs), K (f16): 60.00 MiB, V (f16): 60.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 30.75 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 7.01 MiB
|
||||
llama_context: graph nodes = 613
|
||||
llama_context: graph splits = 216 (with bs=512), 1 (with bs=1)
|
||||
vision_model_loader: model name:
|
||||
vision_model_loader: description: image encoder for MiniCPM-V
|
||||
vision_model_loader: GGUF version: 3
|
||||
vision_model_loader: alignment: 32
|
||||
vision_model_loader: n_tensors: 455
|
||||
vision_model_loader: n_kv: 20
|
||||
|
||||
vision_ctx: vision using Vulkan0 backend
|
||||
load_hparams: model_type: 0
|
||||
load_hparams: n_embd: 1152
|
||||
load_hparams: n_head: 16
|
||||
load_hparams: n_ff: 4304
|
||||
load_hparams: n_layer: 27
|
||||
load_hparams: ffn_op: gelu
|
||||
load_hparams: projection_dim: 0
|
||||
|
||||
--- vision hparams ---
|
||||
load_hparams: image_size: 448
|
||||
load_hparams: patch_size: 14
|
||||
load_hparams: minicpmv_version: 100045
|
||||
load_hparams: proj_scale_factor: 0
|
||||
load_hparams: n_wa_pattern: 0
|
||||
|
||||
load_hparams: model size: 1044.36 MiB
|
||||
load_hparams: metadata size: 0.16 MiB
|
||||
load_tensors: ffn up/down are swapped
|
||||
alloc_compute_meta: Vulkan0 compute buffer size = 100.30 MiB
|
||||
alloc_compute_meta: CPU compute buffer size = 18.30 MiB
|
||||
init tts....init t2w....voc_hg2_model: CPU backend using 8 threads
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=32000
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=200, n_mel=80, total_size=16000
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=200, n_mel=80, total_elements=16000
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=200, n_tokens=100, current_iter=0, current_total=0, new_total=100, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 1 (total_tokens=100, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 20]
|
||||
audition_audio_batch_encode: Final output: 20 tokens x 4096 dims
|
||||
=== Model Paths ===
|
||||
Base dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf
|
||||
LLM: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf [OK]
|
||||
Vision: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/vision/MiniCPM-o-4_5-vision-F16.gguf [OK]
|
||||
Audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf [OK]
|
||||
TTS: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf [OK]
|
||||
Projector: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf [OK]
|
||||
===================
|
||||
=== Initializing Omni Context ===
|
||||
Media type: 2 (omni: audio+vision)
|
||||
TTS enabled: yes
|
||||
Context size: 2048
|
||||
GPU layers: 99
|
||||
Token2Wav device: cpu
|
||||
Vision backend: metal
|
||||
TTS bin dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts
|
||||
Ref audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\vllm-omni-edge\benchmarks\edge_harness\results\e2e_expansion_20260923\evidence\minicpmo_cpp_radeon890m_prompted\fixture_0000.wav
|
||||
02:29:54.085 === omni_init start
|
||||
02:29:54.085 media_type = 2, duplex_mode = 0, base_output_dir = ./tools/omni/output
|
||||
02:29:54.085 === omni_init: loading new LLM model
|
||||
02:30:44.245 === omni_init: loading TTS model
|
||||
02:30:44.245 TTS model: loading with n_gpu_layers=0
|
||||
02:30:48.135 TTS sampler: temp=0.80, top_p=0.85, top_k=25, rep_penalty=1.05
|
||||
02:30:48.135 TTS: loading weights from GGUF (emb_code, emb_text, projector_semantic, head_code)...
|
||||
02:30:53.859 TTS: head_code shape: dim0=6562, dim1=768, will transpose to [6562, 768]
|
||||
02:30:53.875 TTS: head_code loaded, verifying first few values:
|
||||
02:30:53.875 head_code_weight[0] = 0.01361084 (expect ~0.01385498)
|
||||
02:30:53.875 head_code_weight[1] = -0.01660156 (expect ~-0.01647949)
|
||||
02:30:53.875 head_code_weight[768] = 0.04101562 (this is weight[1, 0], expect ~0.04150391)
|
||||
02:30:53.940 TTS: weights loaded successfully
|
||||
02:30:53.940 Projector: loading from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf
|
||||
02:30:54.047 Projector: loaded successfully
|
||||
02:30:54.048 === omni_init: loading APM model
|
||||
02:30:57.144 APM: init from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf
|
||||
02:31:03.140 Token2Wav: found models in \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf
|
||||
02:31:03.192 Token2Wav: all model files found, initializing session...
|
||||
02:31:03.193 Token2Wav: non-CUDA backend, vocoder using CPU for better performance
|
||||
02:31:03.193 Token2Wav: using prompt_cache from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf/prompt_cache.gguf
|
||||
02:31:07.919 Token2Wav: initialized successfully
|
||||
02:31:07.919 Python T2W: script_dir=./tools/omni/pyt2w, model_dir=./tools/omni/pyt2w/token2wav
|
||||
02:31:07.919 Token2Wav: 浣跨敤 C++ 瀹炵幇
|
||||
02:31:07.919 === omni_init success: ctx_llama = 0000011f354cfe80
|
||||
=== Running test case ===
|
||||
Audio prefix: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-2-4ont3qb6\fixture_
|
||||
Count: 2
|
||||
02:31:07.920 stream_prefill: n_past = 0
|
||||
voice_clone_prompt = <|im_start|>system
|
||||
妯′豢闊抽�鏍锋湰鐨勯煶鑹插苟鐢熸垚鏂扮殑鍐呭�銆�
|
||||
<|audio_start|>
|
||||
assistant_prompt = <|audio_end|>浣犵殑浠诲姟鏄�敤杩欑�澹伴煶妯″紡鏉ュ綋涓€涓�姪鎵嬨€傝�璁ょ湡銆侀珮璐ㄩ噺鍦板洖澶嶇敤鎴风殑闂��銆傝�鐢ㄩ珮鑷�劧搴︾殑鏂瑰紡鍜岀敤鎴疯亰澶┿€�<|im_end|>
|
||||
<|im_start|>user
|
||||
|
||||
02:31:07.920 system prompt ref_audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-2-4ont3qb6\fixture_0000.wav
|
||||
02:31:08.533 system prompt ref_audio embedding: n_pos=20
|
||||
02:31:09.097 馃敀 n_keep 璁剧疆涓� 72 (system prompt tokens)锛岃繖閮ㄥ垎姘歌繙涓嶄細琚�粦鍔ㄧ獥鍙ｅ垹闄�
|
||||
02:31:09.097 stream_prefill(index=0): system prompt 鍒濆�鍖栧畬鎴愶紝ref_audio 宸插湪鍏朵腑 prefill
|
||||
02:31:09.097 [SW] system_prompt registered: preserve_length=72 (will be protected from sliding)
|
||||
02:31:09.097 n_past = 72
|
||||
02:31:09.097
|
||||
|
||||
c++ finish stream_prefill(index=0). n_past=72, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 0 (audio) : 1.17765 s
|
||||
encode_image_with_vision_chunks: step 1 of 1 encoded in 1005.59 ms
|
||||
encode_image_with_vision_chunks: all 1 chunks encoded in 1023.42 ms (grid: 1x1)
|
||||
|
||||
encode_image_with_vision_chunks: image encoded in 1023.42 ms by vision ( 1023.42 ms per chunk, 64 total tokens)
|
||||
omni_image_embed_make_chunks_with_filename: created 1 vision chunks from C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-2-4ont3qb6\fixture_0001.jpg
|
||||
stream_prefill: prefilled 1 vision chunks (64 tokens each)
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=49339
|
||||
audition_audio_preprocess: Audio not aligned to 100ms (49339 samples = 3083.7ms), padded with silence to 49600 samples (3100ms)
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=310, n_mel=80, total_size=24800
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=310, n_mel=80, total_elements=24800
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=310, n_tokens=155, current_iter=1, current_total=155, new_total=310, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 2 (total_tokens=310, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 31]
|
||||
audition_audio_batch_encode: Final output: 31 tokens x 4096 dims
|
||||
02:31:10.698 stream_prefill(index=1): processing user audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-2-4ont3qb6\fixture_0001.wav
|
||||
02:31:11.008 stream_prefill(index=1): user audio embedding: n_pos=31
|
||||
02:31:11.371
|
||||
|
||||
c++ finish stream_prefill(index=1). n_past=172, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 1 (audio+vision) : 2.27174 s
|
||||
02:31:11.372 馃搷 stream_decode 寮€濮�: n_past=172, n_keep=72, n_ctx=2048, duplex_mode=0
|
||||
02:31:11.372 stream_decode: create tts thread (simplex mode)
|
||||
02:31:11.372 stream_decode: create t2w thread
|
||||
02:31:11.372 wait prefill done
|
||||
02:31:11.372 馃搷 [鍗曞伐TTS] 娣诲姞 assistant prompt: "<|im_end|>
|
||||
<|im_start|>assistant
|
||||
<think>
|
||||
|
||||
</think>
|
||||
|
||||
<|tts_bos|>", n_past=172
|
||||
02:31:11.372 T2W thread (C++) started
|
||||
<user>
|
||||
<assistant>02:31:11.378 TTS: 鍒涘缓鍗曞伐妯″紡杈撳嚭鐩�綍: ./tools/omni/output/round_000
|
||||
02:31:11.379 TTS thread started
|
||||
02:31:11.558 馃搷 [鍗曞伐TTS] assistant prompt 瀹屾垚, n_past=182
|
||||
02:31:11.558 LLM decode: max_tgt_len = 96, n_predict = 96, n_ctx = 2048
|
||||
02:31:12.115 LLM: detected end token, set llm_generation_done=true
|
||||
02:31:12.115 LLM->TTS: text='The square in the image is red.', n_tokens=8, hidden_size=32768, n_embd=4096, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:31:12.115 馃搷 Decode 缁撴潫: n_past=191, 鍓╀綑绌洪棿=1857, 鏃犻渶婊戠獥
|
||||
02:31:12.115 馃搷 杞�� 1 缁撴潫锛岃�褰曡竟鐣屼簬 n_past=191
|
||||
02:31:12.115 TTS<-LLM: chunk_idx=0, text='The square in the image is red.', n_tokens=8, hidden_size=32768, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:31:12.115 TTS: after queue pop - speek_done=0, llm_finish=1, llm_text.empty=0, token_ids.size=8
|
||||
02:31:12.115 TTS: before empty check - response.empty=0, response='The square in the image is red.', llm_finish=1
|
||||
Waiting for audio generation to complete...
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_0.wav does not exist or is empty
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_1.wav does not exist or is empty
|
||||
TTS: no valid WAV files to merge
|
||||
02:31:12.115 TTS: DEBUG before has_llm_data - token_ids.size=8, hidden_states.size=32768, n_embd=4096
|
||||
02:31:12.115 TTS: has_llm_data=1, token_ids=8, hidden_states=32768, n_embd=4096, llm_finish=1
|
||||
02:31:12.116 TTS: n_tokens_orig=8, n_tokens_filtered=8 (filtered 0 special tokens)
|
||||
02:31:12.150 TTS: saved merged embeddings (text) to ./tools/omni/output/round_000/llm_debug/chunk_0/merged_embeddings.txt
|
||||
02:31:12.150 TTS Simplex: generating audio tokens for chunk 0 (n_tokens=8, tts_n_embd=768)
|
||||
02:31:12.150 TTS Simplex: 鍦� prefill 鍓嶆坊鍔� audio_bos (chunk_idx=0, new_size=9)
|
||||
02:31:12.150 TTS Simplex: first chunk - cleared KV cache
|
||||
02:31:12.150 TTS Simplex: first chunk - cleared tts_all_generated_tokens and tts_token_buffer
|
||||
02:31:12.211 馃搷 涓轰笅涓€杞�噯澶�: eval <|im_end|>\n<|im_start|>user\n, n_past=196
|
||||
02:31:13.607 TTS Simplex: prefill completed, n_past_tts=9
|
||||
02:31:13.608 TTS Simplex: sampler created
|
||||
02:31:13.608 TTS simplex: is_audio_bos=true (first audio token)
|
||||
02:31:13.633 TTS Simplex Phase1: token 1/500: rel_id=767
|
||||
02:31:13.645 TTS Simplex Phase1: token 2/500: rel_id=4218
|
||||
02:31:13.655 TTS Simplex Phase1: token 3/500: rel_id=4218
|
||||
02:31:13.666 TTS Simplex Phase1: token 4/500: rel_id=4137
|
||||
02:31:13.677 TTS Simplex Phase1: token 5/500: rel_id=4137
|
||||
02:31:13.895 TTS Simplex Phase1: yield 25 tokens 鍒� T2W (step 25, buffer=25)
|
||||
02:31:13.895 TTS Simplex Phase1: token 25/500: rel_id=1685
|
||||
02:31:13.895 T2W绾跨▼(C++): 杞��鍒囨崲 (-1 -> 0)锛屾洿鏂拌緭鍑虹洰褰�
|
||||
02:31:13.895 T2W绾跨▼(C++): 鏂拌緭鍑虹洰褰� ./tools/omni/output/round_000/tts_wav
|
||||
02:31:14.080 TTS Simplex Phase1: EOS token at step 42
|
||||
02:31:14.080 TTS Simplex: is_final_text_chunk=true, will enter Phase 2 for text_eos_embed
|
||||
02:31:14.080 TTS Simplex Phase2: injecting text_eos_embed + audio_bos at n_past=50
|
||||
02:31:14.087 TTS Simplex Phase2: text_eos_embed injected, n_past=51
|
||||
02:31:14.095 TTS Simplex Phase2: audio_bos injected, n_past=52, starting final generation
|
||||
02:31:14.195 TTS Simplex Phase2: yield 25 tokens 鍒� T2W
|
||||
02:31:14.305 TTS Simplex Phase2: EOS at step 19 (final end)
|
||||
02:31:14.305 TTS Simplex: chunk 缁撴潫锛宼ts_token_buffer 鍓╀綑 9 tokens (淇濈暀绛変笅涓€涓� chunk)
|
||||
02:31:14.305 TTS Simplex: is_final_text_chunk=true, flush 鍓╀綑 9 tokens 浠� tts_token_buffer
|
||||
02:31:14.305 TTS Simplex: generated 59 audio tokens for chunk 0
|
||||
02:31:14.305 TTS Simplex: chunk 0 first tokens: [767 4218 4218 4137 4137 ], last tokens: [4218 4137 4218 4218 3895 ]
|
||||
02:31:14.305 TTS Simplex: updated tts_n_past_accumulated=71, total_generated_tokens=59
|
||||
02:31:14.306 TTS: finished processing all chunks
|
||||
02:31:14.306 TTS: 鍗曞伐妯″紡杞��缁撴潫锛屼笅涓€杞� round_idx=1
|
||||
02:31:14.306 TTS: sent is_final=true to T2W queue (turn end)
|
||||
02:31:16.353 馃帀 棣栧搷鏃堕棿 (First Audio Response): 4981ms
|
||||
02:31:16.353 T2W绾跨▼: wav_0.wav | 0.84s audio | 2456.4ms inference | RTF=2.92 | t=4981ms
|
||||
02:31:18.619 T2W绾跨▼: wav_1.wav | 1.00s audio | 2265.9ms inference | RTF=2.27 | t=7247ms
|
||||
02:31:20.748 T2W绾跨▼: wav_2.wav | 0.64s audio | 2128.3ms inference | RTF=3.33 | t=9376ms
|
||||
02:31:20.758 T2W绾跨▼: 鍐欏叆缁撴潫鏍囪� ./tools/omni/output/round_000/tts_wav/generation_done.flag (last_wav=2)
|
||||
02:31:20.758 T2W绾跨▼: 杞��缁撴潫鍚庢洿鏂拌緭鍑虹洰褰曚负 ./tools/omni/output/round_001/tts_wav
|
||||
02:31:20.843 omni_stop_threads: stop signals sent
|
||||
02:31:20.843 T2W(C++) 绾跨▼: 鍋滄�
|
||||
[timing] call=0 tokens=28 final=0 token2mel=2118.668ms vocoder=336.795ms total=2455.957ms audio=20160
|
||||
[timing] call=1 tokens=28 final=0 token2mel=1881.062ms vocoder=384.835ms total=2265.911ms audio=24000
|
||||
[timing] call=2 tokens=12 final=1 token2mel=1905.958ms vocoder=222.337ms total=2128.329ms audio=15360
|
||||
Audio generation completed.
|
||||
llama_perf_context_print: load time = 27180.52 ms
|
||||
llama_perf_context_print: prompt eval time = 3635.76 ms / 182 tokens ( 19.98 ms per token, 50.06 tokens per second)
|
||||
llama_perf_context_print: eval time = 553.65 ms / 9 runs ( 61.52 ms per token, 16.26 tokens per second)
|
||||
llama_perf_context_print: total time = 36465.48 ms / 191 tokens
|
||||
llama_perf_context_print: graphs reused = 10
|
||||
Token2Wav (C++): session released
|
||||
tts thread end
|
||||
t2w thread end
|
||||
+483
@@ -0,0 +1,483 @@
|
||||
build: 64 (5202b7b) with x86_64-w64-mingw32-gcc-posix (GCC) 13-posix for x86_64-w64-mingw32
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 30720 MiB free
|
||||
llama_model_loader: loaded meta data with 27 key-value pairs and 399 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = qwen3
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Llm
|
||||
llama_model_loader: - kv 3: general.size_label str = 8.2B
|
||||
llama_model_loader: - kv 4: qwen3.block_count u32 = 36
|
||||
llama_model_loader: - kv 5: qwen3.context_length u32 = 40960
|
||||
llama_model_loader: - kv 6: qwen3.embedding_length u32 = 4096
|
||||
llama_model_loader: - kv 7: qwen3.feed_forward_length u32 = 12288
|
||||
llama_model_loader: - kv 8: qwen3.attention.head_count u32 = 32
|
||||
llama_model_loader: - kv 9: qwen3.attention.head_count_kv u32 = 8
|
||||
llama_model_loader: - kv 10: qwen3.rope.freq_base f32 = 1000000.000000
|
||||
llama_model_loader: - kv 11: qwen3.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: qwen3.attention.key_length u32 = 128
|
||||
llama_model_loader: - kv 13: qwen3.attention.value_length u32 = 128
|
||||
llama_model_loader: - kv 14: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 15: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 16: tokenizer.ggml.tokens arr[str,151748] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 17: tokenizer.ggml.token_type arr[i32,151748] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 24: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 25: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 26: general.file_type u32 = 15
|
||||
llama_model_loader: - type f32: 145 tensors
|
||||
llama_model_loader: - type q4_K: 217 tensors
|
||||
llama_model_loader: - type q6_K: 37 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = Q4_K - Medium
|
||||
print_info: file size = 4.68 GiB (4.90 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = qwen3
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 40960
|
||||
print_info: n_embd = 4096
|
||||
print_info: n_layer = 36
|
||||
print_info: n_head = 32
|
||||
print_info: n_head_kv = 8
|
||||
print_info: n_rot = 128
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 128
|
||||
print_info: n_embd_head_v = 128
|
||||
print_info: n_gqa = 4
|
||||
print_info: n_embd_k_gqa = 1024
|
||||
print_info: n_embd_v_gqa = 1024
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 12288
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = -1
|
||||
print_info: rope type = 2
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 1000000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 40960
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = 8B
|
||||
print_info: model params = 8.19 B
|
||||
print_info: general.name = Llm
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 151748
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 36 repeating layers to GPU
|
||||
load_tensors: offloading output layer to GPU
|
||||
load_tensors: offloaded 37/37 layers to GPU
|
||||
load_tensors: Vulkan0 model buffer size = 4454.74 MiB
|
||||
load_tensors: CPU_Mapped model buffer size = 333.43 MiB
|
||||
.....................................................................................
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 1000000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (40960) -- the full capacity of the model will not be utilized
|
||||
llama_context: Vulkan_Host output buffer size = 0.58 MiB
|
||||
llama_kv_cache: Vulkan0 KV buffer size = 288.00 MiB
|
||||
llama_kv_cache: size = 288.00 MiB ( 2048 cells, 36 layers, 1/1 seqs), K (f16): 144.00 MiB, V (f16): 144.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 304.38 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 12.01 MiB
|
||||
llama_context: graph nodes = 1267
|
||||
llama_context: graph splits = 2
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 25673 MiB free
|
||||
llama_model_loader: loaded meta data with 30 key-value pairs and 193 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = llama
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Tts
|
||||
llama_model_loader: - kv 3: general.size_label str = 445M
|
||||
llama_model_loader: - kv 4: llama.block_count u32 = 20
|
||||
llama_model_loader: - kv 5: llama.context_length u32 = 4096
|
||||
llama_model_loader: - kv 6: llama.embedding_length u32 = 768
|
||||
llama_model_loader: - kv 7: llama.feed_forward_length u32 = 3072
|
||||
llama_model_loader: - kv 8: llama.attention.head_count u32 = 12
|
||||
llama_model_loader: - kv 9: llama.attention.head_count_kv u32 = 12
|
||||
llama_model_loader: - kv 10: llama.rope.freq_base f32 = 10000.000000
|
||||
llama_model_loader: - kv 11: llama.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: llama.attention.key_length u32 = 64
|
||||
llama_model_loader: - kv 13: llama.attention.value_length u32 = 64
|
||||
llama_model_loader: - kv 14: general.file_type u32 = 1
|
||||
llama_model_loader: - kv 15: llama.vocab_size u32 = 158626
|
||||
llama_model_loader: - kv 16: llama.rope.dimension_count u32 = 64
|
||||
llama_model_loader: - kv 17: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.tokens arr[str,158626] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.token_type arr[i32,158626] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 24: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 25: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 26: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 27: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 28: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 29: tokenizer.ggml.add_space_prefix bool = false
|
||||
llama_model_loader: - type f32: 50 tensors
|
||||
llama_model_loader: - type f16: 143 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = F16
|
||||
print_info: file size = 1.07 GiB (20.70 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = llama
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 4096
|
||||
print_info: n_embd = 768
|
||||
print_info: n_layer = 20
|
||||
print_info: n_head = 12
|
||||
print_info: n_head_kv = 12
|
||||
print_info: n_rot = 64
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 64
|
||||
print_info: n_embd_head_v = 64
|
||||
print_info: n_gqa = 1
|
||||
print_info: n_embd_k_gqa = 768
|
||||
print_info: n_embd_v_gqa = 768
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 3072
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = 0
|
||||
print_info: rope type = 0
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 10000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 4096
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = ?B
|
||||
print_info: model params = 444.94 M
|
||||
print_info: general.name = Tts
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 158626
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
load_tensors: skipping TTS tensor emb_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor emb_text.weight (size = 467140608 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.weight (size = 12582912 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.weight (size = 2359296 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.weight (size = 6291456 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.weight (size = 1179648 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor head_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 0 repeating layers to GPU
|
||||
load_tensors: offloaded 0/21 layers to GPU
|
||||
load_tensors: CPU_Mapped model buffer size = 592.48 MiB
|
||||
..............................................................
|
||||
llama_init_from_model: model default pooling_type is [0], but [-1] was specified
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 10000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (4096) -- the full capacity of the model will not be utilized
|
||||
llama_context: CPU output buffer size = 0.61 MiB
|
||||
llama_kv_cache: CPU KV buffer size = 120.00 MiB
|
||||
llama_kv_cache: size = 120.00 MiB ( 2048 cells, 20 layers, 1/1 seqs), K (f16): 60.00 MiB, V (f16): 60.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 30.75 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 7.01 MiB
|
||||
llama_context: graph nodes = 613
|
||||
llama_context: graph splits = 216 (with bs=512), 1 (with bs=1)
|
||||
vision_model_loader: model name:
|
||||
vision_model_loader: description: image encoder for MiniCPM-V
|
||||
vision_model_loader: GGUF version: 3
|
||||
vision_model_loader: alignment: 32
|
||||
vision_model_loader: n_tensors: 455
|
||||
vision_model_loader: n_kv: 20
|
||||
|
||||
vision_ctx: vision using Vulkan0 backend
|
||||
load_hparams: model_type: 0
|
||||
load_hparams: n_embd: 1152
|
||||
load_hparams: n_head: 16
|
||||
load_hparams: n_ff: 4304
|
||||
load_hparams: n_layer: 27
|
||||
load_hparams: ffn_op: gelu
|
||||
load_hparams: projection_dim: 0
|
||||
|
||||
--- vision hparams ---
|
||||
load_hparams: image_size: 448
|
||||
load_hparams: patch_size: 14
|
||||
load_hparams: minicpmv_version: 100045
|
||||
load_hparams: proj_scale_factor: 0
|
||||
load_hparams: n_wa_pattern: 0
|
||||
|
||||
load_hparams: model size: 1044.36 MiB
|
||||
load_hparams: metadata size: 0.16 MiB
|
||||
load_tensors: ffn up/down are swapped
|
||||
alloc_compute_meta: Vulkan0 compute buffer size = 100.30 MiB
|
||||
alloc_compute_meta: CPU compute buffer size = 18.30 MiB
|
||||
init tts....init t2w....voc_hg2_model: CPU backend using 8 threads
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=32000
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=200, n_mel=80, total_size=16000
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=200, n_mel=80, total_elements=16000
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=200, n_tokens=100, current_iter=0, current_total=0, new_total=100, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 1 (total_tokens=100, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 20]
|
||||
audition_audio_batch_encode: Final output: 20 tokens x 4096 dims
|
||||
=== Model Paths ===
|
||||
Base dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf
|
||||
LLM: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf [OK]
|
||||
Vision: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/vision/MiniCPM-o-4_5-vision-F16.gguf [OK]
|
||||
Audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf [OK]
|
||||
TTS: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf [OK]
|
||||
Projector: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf [OK]
|
||||
===================
|
||||
=== Initializing Omni Context ===
|
||||
Media type: 2 (omni: audio+vision)
|
||||
TTS enabled: yes
|
||||
Context size: 2048
|
||||
GPU layers: 99
|
||||
Token2Wav device: cpu
|
||||
Vision backend: metal
|
||||
TTS bin dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts
|
||||
Ref audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\vllm-omni-edge\benchmarks\edge_harness\results\e2e_expansion_20260923\evidence\minicpmo_cpp_radeon890m_prompted\fixture_0000.wav
|
||||
02:55:53.695 === omni_init start
|
||||
02:55:53.695 media_type = 2, duplex_mode = 0, base_output_dir = ./tools/omni/output
|
||||
02:55:53.695 === omni_init: loading new LLM model
|
||||
02:56:44.599 === omni_init: loading TTS model
|
||||
02:56:44.599 TTS model: loading with n_gpu_layers=0
|
||||
02:56:48.272 TTS sampler: temp=0.80, top_p=0.85, top_k=25, rep_penalty=1.05
|
||||
02:56:48.272 TTS: loading weights from GGUF (emb_code, emb_text, projector_semantic, head_code)...
|
||||
02:56:53.782 TTS: head_code shape: dim0=6562, dim1=768, will transpose to [6562, 768]
|
||||
02:56:53.787 TTS: head_code loaded, verifying first few values:
|
||||
02:56:53.787 head_code_weight[0] = 0.01361084 (expect ~0.01385498)
|
||||
02:56:53.787 head_code_weight[1] = -0.01660156 (expect ~-0.01647949)
|
||||
02:56:53.787 head_code_weight[768] = 0.04101562 (this is weight[1, 0], expect ~0.04150391)
|
||||
02:56:53.836 TTS: weights loaded successfully
|
||||
02:56:53.836 Projector: loading from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf
|
||||
02:56:53.938 Projector: loaded successfully
|
||||
02:56:53.939 === omni_init: loading APM model
|
||||
02:56:56.997 APM: init from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf
|
||||
02:57:02.691 Token2Wav: found models in \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf
|
||||
02:57:02.745 Token2Wav: all model files found, initializing session...
|
||||
02:57:02.745 Token2Wav: non-CUDA backend, vocoder using CPU for better performance
|
||||
02:57:02.745 Token2Wav: using prompt_cache from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf/prompt_cache.gguf
|
||||
02:57:07.258 Token2Wav: initialized successfully
|
||||
02:57:07.258 Python T2W: script_dir=./tools/omni/pyt2w, model_dir=./tools/omni/pyt2w/token2wav
|
||||
02:57:07.258 Token2Wav: 浣跨敤 C++ 瀹炵幇
|
||||
02:57:07.258 === omni_init success: ctx_llama = 000002181303f830
|
||||
=== Running test case ===
|
||||
Audio prefix: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-20-zns5l6d7\fixture_
|
||||
Count: 2
|
||||
02:57:07.258 stream_prefill: n_past = 0
|
||||
voice_clone_prompt = <|im_start|>system
|
||||
妯′豢闊抽�鏍锋湰鐨勯煶鑹插苟鐢熸垚鏂扮殑鍐呭�銆�
|
||||
<|audio_start|>
|
||||
assistant_prompt = <|audio_end|>浣犵殑浠诲姟鏄�敤杩欑�澹伴煶妯″紡鏉ュ綋涓€涓�姪鎵嬨€傝�璁ょ湡銆侀珮璐ㄩ噺鍦板洖澶嶇敤鎴风殑闂��銆傝�鐢ㄩ珮鑷�劧搴︾殑鏂瑰紡鍜岀敤鎴疯亰澶┿€�<|im_end|>
|
||||
<|im_start|>user
|
||||
|
||||
02:57:07.258 system prompt ref_audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-20-zns5l6d7\fixture_0000.wav
|
||||
02:57:07.899 system prompt ref_audio embedding: n_pos=20
|
||||
02:57:08.408 馃敀 n_keep 璁剧疆涓� 72 (system prompt tokens)锛岃繖閮ㄥ垎姘歌繙涓嶄細琚�粦鍔ㄧ獥鍙ｅ垹闄�
|
||||
02:57:08.408 stream_prefill(index=0): system prompt 鍒濆�鍖栧畬鎴愶紝ref_audio 宸插湪鍏朵腑 prefill
|
||||
02:57:08.408 [SW] system_prompt registered: preserve_length=72 (will be protected from sliding)
|
||||
02:57:08.408 n_past = 72
|
||||
02:57:08.408
|
||||
|
||||
c++ finish stream_prefill(index=0). n_past=72, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 0 (audio) : 1.14979 s
|
||||
encode_image_with_vision_chunks: step 1 of 1 encoded in 649.98 ms
|
||||
encode_image_with_vision_chunks: all 1 chunks encoded in 670.06 ms (grid: 1x1)
|
||||
|
||||
encode_image_with_vision_chunks: image encoded in 670.07 ms by vision ( 670.07 ms per chunk, 64 total tokens)
|
||||
omni_image_embed_make_chunks_with_filename: created 1 vision chunks from C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-20-zns5l6d7\fixture_0001.jpg
|
||||
stream_prefill: prefilled 1 vision chunks (64 tokens each)
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=49339
|
||||
audition_audio_preprocess: Audio not aligned to 100ms (49339 samples = 3083.7ms), padded with silence to 49600 samples (3100ms)
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=310, n_mel=80, total_size=24800
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=310, n_mel=80, total_elements=24800
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=310, n_tokens=155, current_iter=1, current_total=155, new_total=310, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 2 (total_tokens=310, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 31]
|
||||
audition_audio_batch_encode: Final output: 31 tokens x 4096 dims
|
||||
02:57:09.600 stream_prefill(index=1): processing user audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-20-zns5l6d7\fixture_0001.wav
|
||||
02:57:09.906 stream_prefill(index=1): user audio embedding: n_pos=31
|
||||
02:57:10.221
|
||||
|
||||
c++ finish stream_prefill(index=1). n_past=172, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 1 (audio+vision) : 1.81084 s
|
||||
02:57:10.221 馃搷 stream_decode 寮€濮�: n_past=172, n_keep=72, n_ctx=2048, duplex_mode=0
|
||||
02:57:10.221 stream_decode: create tts thread (simplex mode)
|
||||
02:57:10.221 stream_decode: create t2w thread
|
||||
02:57:10.221 wait prefill done
|
||||
02:57:10.221 馃搷 [鍗曞伐TTS] 娣诲姞 assistant prompt: "<|im_end|>
|
||||
<|im_start|>assistant
|
||||
<think>
|
||||
|
||||
</think>
|
||||
|
||||
<|tts_bos|>", n_past=172
|
||||
02:57:10.222 T2W thread (C++) started
|
||||
<user>
|
||||
<assistant>02:57:10.226 TTS: 鍒涘缓鍗曞伐妯″紡杈撳嚭鐩�綍: ./tools/omni/output/round_000
|
||||
02:57:10.227 TTS thread started
|
||||
02:57:10.383 馃搷 [鍗曞伐TTS] assistant prompt 瀹屾垚, n_past=182
|
||||
02:57:10.383 LLM decode: max_tgt_len = 96, n_predict = 96, n_ctx = 2048
|
||||
02:57:10.902 LLM: detected end token, set llm_generation_done=true
|
||||
02:57:10.902 LLM->TTS: text='The square in the image is red.', n_tokens=8, hidden_size=32768, n_embd=4096, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:57:10.902 馃搷 Decode 缁撴潫: n_past=191, 鍓╀綑绌洪棿=1857, 鏃犻渶婊戠獥
|
||||
02:57:10.902 馃搷 杞�� 1 缁撴潫锛岃�褰曡竟鐣屼簬 n_past=191
|
||||
02:57:10.902 TTS<-LLM: chunk_idx=0, text='The square in the image is red.', n_tokens=8, hidden_size=32768, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:57:10.902 TTS: after queue pop - speek_done=0, llm_finish=1, llm_text.empty=0, token_ids.size=8
|
||||
02:57:10.902 TTS: before empty check - response.empty=0, response='The square in the image is red.', llm_finish=1
|
||||
Waiting for audio generation to complete...
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_0.wav does not exist or is empty
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_1.wav does not exist or is empty
|
||||
TTS: no valid WAV files to merge
|
||||
02:57:10.902 TTS: DEBUG before has_llm_data - token_ids.size=8, hidden_states.size=32768, n_embd=4096
|
||||
02:57:10.902 TTS: has_llm_data=1, token_ids=8, hidden_states=32768, n_embd=4096, llm_finish=1
|
||||
02:57:10.903 TTS: n_tokens_orig=8, n_tokens_filtered=8 (filtered 0 special tokens)
|
||||
02:57:10.945 TTS: saved merged embeddings (text) to ./tools/omni/output/round_000/llm_debug/chunk_0/merged_embeddings.txt
|
||||
02:57:10.946 TTS Simplex: generating audio tokens for chunk 0 (n_tokens=8, tts_n_embd=768)
|
||||
02:57:10.946 TTS Simplex: 鍦� prefill 鍓嶆坊鍔� audio_bos (chunk_idx=0, new_size=9)
|
||||
02:57:10.946 TTS Simplex: first chunk - cleared KV cache
|
||||
02:57:10.946 TTS Simplex: first chunk - cleared tts_all_generated_tokens and tts_token_buffer
|
||||
02:57:10.994 馃搷 涓轰笅涓€杞�噯澶�: eval <|im_end|>\n<|im_start|>user\n, n_past=196
|
||||
02:57:11.220 TTS Simplex: prefill completed, n_past_tts=9
|
||||
02:57:11.220 TTS Simplex: sampler created
|
||||
02:57:11.220 TTS simplex: is_audio_bos=true (first audio token)
|
||||
02:57:11.247 TTS Simplex Phase1: token 1/500: rel_id=770
|
||||
02:57:11.258 TTS Simplex Phase1: token 2/500: rel_id=4137
|
||||
02:57:11.268 TTS Simplex Phase1: token 3/500: rel_id=4218
|
||||
02:57:11.279 TTS Simplex Phase1: token 4/500: rel_id=4218
|
||||
02:57:11.290 TTS Simplex Phase1: token 5/500: rel_id=1479
|
||||
02:57:11.513 TTS Simplex Phase1: yield 25 tokens 鍒� T2W (step 25, buffer=25)
|
||||
02:57:11.513 TTS Simplex Phase1: token 25/500: rel_id=638
|
||||
02:57:11.513 T2W绾跨▼(C++): 杞��鍒囨崲 (-1 -> 0)锛屾洿鏂拌緭鍑虹洰褰�
|
||||
02:57:11.513 T2W绾跨▼(C++): 鏂拌緭鍑虹洰褰� ./tools/omni/output/round_000/tts_wav
|
||||
02:57:11.592 TTS Simplex Phase1: EOS token at step 33
|
||||
02:57:11.592 TTS Simplex: is_final_text_chunk=true, will enter Phase 2 for text_eos_embed
|
||||
02:57:11.592 TTS Simplex Phase2: injecting text_eos_embed + audio_bos at n_past=41
|
||||
02:57:11.600 TTS Simplex Phase2: text_eos_embed injected, n_past=42
|
||||
02:57:11.609 TTS Simplex Phase2: audio_bos injected, n_past=43, starting final generation
|
||||
02:57:11.808 TTS Simplex Phase2: yield 25 tokens 鍒� T2W
|
||||
02:57:11.830 TTS Simplex Phase2: EOS at step 20 (final end)
|
||||
02:57:11.830 TTS Simplex: chunk 缁撴潫锛宼ts_token_buffer 鍓╀綑 1 tokens (淇濈暀绛変笅涓€涓� chunk)
|
||||
02:57:11.830 TTS Simplex: is_final_text_chunk=true, flush 鍓╀綑 1 tokens 浠� tts_token_buffer
|
||||
02:57:11.830 TTS Simplex: generated 51 audio tokens for chunk 0
|
||||
02:57:11.830 TTS Simplex: chunk 0 first tokens: [770 4137 4218 4218 1479 ], last tokens: [1536 40 736 4218 4137 ]
|
||||
02:57:11.830 TTS Simplex: updated tts_n_past_accumulated=63, total_generated_tokens=51
|
||||
02:57:11.840 TTS: finished processing all chunks
|
||||
02:57:11.841 TTS: 鍗曞伐妯″紡杞��缁撴潫锛屼笅涓€杞� round_idx=1
|
||||
02:57:11.841 TTS: sent is_final=true to T2W queue (turn end)
|
||||
02:57:13.843 馃帀 棣栧搷鏃堕棿 (First Audio Response): 3621ms
|
||||
02:57:13.843 T2W绾跨▼: wav_0.wav | 0.84s audio | 2328.7ms inference | RTF=2.77 | t=3621ms
|
||||
02:57:16.094 T2W绾跨▼: wav_1.wav | 1.00s audio | 2250.2ms inference | RTF=2.25 | t=5872ms
|
||||
02:57:18.093 T2W绾跨▼: wav_2.wav | 0.32s audio | 1998.2ms inference | RTF=6.24 | t=7871ms
|
||||
02:57:18.093 T2W绾跨▼: 鍐欏叆缁撴潫鏍囪� ./tools/omni/output/round_000/tts_wav/generation_done.flag (last_wav=2)
|
||||
02:57:18.093 T2W绾跨▼: 杞��缁撴潫鍚庢洿鏂拌緭鍑虹洰褰曚负 ./tools/omni/output/round_001/tts_wav
|
||||
02:02:57:57:18.153 T2W(C++) 绾跨▼: 鍋滄�
|
||||
[timing] call=0 tokens=28 final=0 token2mel=2000.972ms vocoder=327.689ms total=2328.679ms audio=20160
|
||||
[timing] call=1 tokens=28 final=0 token2mel=1858.415ms vocoder=391.779ms total=2250.218ms audio=24000
|
||||
[timing] call=2 tokens=4 final=1 token2mel=1864.264ms vocoder=133.880ms total=1998.153ms audio=7680
|
||||
Audio generation completed.
|
||||
llama_perf_context_print: load time = 25393.23 ms
|
||||
llama_perf_context_print: prompt eval time = 3124.73 ms / 182 tokens ( 17.17 ms per token, 58.24 tokens per second)
|
||||
llama_perf_context_print: eval time = 515.78 ms / 9 runs ( 57.31 ms per token, 17.45 tokens per second)
|
||||
llama_perf_context_print: total time = 33162.38 ms / 191 tokens
|
||||
llama_perf_context_print: graphs reused = 10
|
||||
Token2Wav (C++): session released
|
||||
18.153 omni_stop_threads: stop signals sent
|
||||
tts thread end
|
||||
t2w thread end
|
||||
+483
@@ -0,0 +1,483 @@
|
||||
build: 64 (5202b7b) with x86_64-w64-mingw32-gcc-posix (GCC) 13-posix for x86_64-w64-mingw32
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 30720 MiB free
|
||||
llama_model_loader: loaded meta data with 27 key-value pairs and 399 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = qwen3
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Llm
|
||||
llama_model_loader: - kv 3: general.size_label str = 8.2B
|
||||
llama_model_loader: - kv 4: qwen3.block_count u32 = 36
|
||||
llama_model_loader: - kv 5: qwen3.context_length u32 = 40960
|
||||
llama_model_loader: - kv 6: qwen3.embedding_length u32 = 4096
|
||||
llama_model_loader: - kv 7: qwen3.feed_forward_length u32 = 12288
|
||||
llama_model_loader: - kv 8: qwen3.attention.head_count u32 = 32
|
||||
llama_model_loader: - kv 9: qwen3.attention.head_count_kv u32 = 8
|
||||
llama_model_loader: - kv 10: qwen3.rope.freq_base f32 = 1000000.000000
|
||||
llama_model_loader: - kv 11: qwen3.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: qwen3.attention.key_length u32 = 128
|
||||
llama_model_loader: - kv 13: qwen3.attention.value_length u32 = 128
|
||||
llama_model_loader: - kv 14: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 15: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 16: tokenizer.ggml.tokens arr[str,151748] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 17: tokenizer.ggml.token_type arr[i32,151748] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 24: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 25: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 26: general.file_type u32 = 15
|
||||
llama_model_loader: - type f32: 145 tensors
|
||||
llama_model_loader: - type q4_K: 217 tensors
|
||||
llama_model_loader: - type q6_K: 37 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = Q4_K - Medium
|
||||
print_info: file size = 4.68 GiB (4.90 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = qwen3
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 40960
|
||||
print_info: n_embd = 4096
|
||||
print_info: n_layer = 36
|
||||
print_info: n_head = 32
|
||||
print_info: n_head_kv = 8
|
||||
print_info: n_rot = 128
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 128
|
||||
print_info: n_embd_head_v = 128
|
||||
print_info: n_gqa = 4
|
||||
print_info: n_embd_k_gqa = 1024
|
||||
print_info: n_embd_v_gqa = 1024
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 12288
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = -1
|
||||
print_info: rope type = 2
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 1000000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 40960
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = 8B
|
||||
print_info: model params = 8.19 B
|
||||
print_info: general.name = Llm
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 151748
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 36 repeating layers to GPU
|
||||
load_tensors: offloading output layer to GPU
|
||||
load_tensors: offloaded 37/37 layers to GPU
|
||||
load_tensors: Vulkan0 model buffer size = 4454.74 MiB
|
||||
load_tensors: CPU_Mapped model buffer size = 333.43 MiB
|
||||
.....................................................................................
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 1000000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (40960) -- the full capacity of the model will not be utilized
|
||||
llama_context: Vulkan_Host output buffer size = 0.58 MiB
|
||||
llama_kv_cache: Vulkan0 KV buffer size = 288.00 MiB
|
||||
llama_kv_cache: size = 288.00 MiB ( 2048 cells, 36 layers, 1/1 seqs), K (f16): 144.00 MiB, V (f16): 144.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 304.38 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 12.01 MiB
|
||||
llama_context: graph nodes = 1267
|
||||
llama_context: graph splits = 2
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 25673 MiB free
|
||||
llama_model_loader: loaded meta data with 30 key-value pairs and 193 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = llama
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Tts
|
||||
llama_model_loader: - kv 3: general.size_label str = 445M
|
||||
llama_model_loader: - kv 4: llama.block_count u32 = 20
|
||||
llama_model_loader: - kv 5: llama.context_length u32 = 4096
|
||||
llama_model_loader: - kv 6: llama.embedding_length u32 = 768
|
||||
llama_model_loader: - kv 7: llama.feed_forward_length u32 = 3072
|
||||
llama_model_loader: - kv 8: llama.attention.head_count u32 = 12
|
||||
llama_model_loader: - kv 9: llama.attention.head_count_kv u32 = 12
|
||||
llama_model_loader: - kv 10: llama.rope.freq_base f32 = 10000.000000
|
||||
llama_model_loader: - kv 11: llama.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: llama.attention.key_length u32 = 64
|
||||
llama_model_loader: - kv 13: llama.attention.value_length u32 = 64
|
||||
llama_model_loader: - kv 14: general.file_type u32 = 1
|
||||
llama_model_loader: - kv 15: llama.vocab_size u32 = 158626
|
||||
llama_model_loader: - kv 16: llama.rope.dimension_count u32 = 64
|
||||
llama_model_loader: - kv 17: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.tokens arr[str,158626] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.token_type arr[i32,158626] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 24: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 25: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 26: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 27: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 28: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 29: tokenizer.ggml.add_space_prefix bool = false
|
||||
llama_model_loader: - type f32: 50 tensors
|
||||
llama_model_loader: - type f16: 143 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = F16
|
||||
print_info: file size = 1.07 GiB (20.70 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = llama
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 4096
|
||||
print_info: n_embd = 768
|
||||
print_info: n_layer = 20
|
||||
print_info: n_head = 12
|
||||
print_info: n_head_kv = 12
|
||||
print_info: n_rot = 64
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 64
|
||||
print_info: n_embd_head_v = 64
|
||||
print_info: n_gqa = 1
|
||||
print_info: n_embd_k_gqa = 768
|
||||
print_info: n_embd_v_gqa = 768
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 3072
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = 0
|
||||
print_info: rope type = 0
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 10000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 4096
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = ?B
|
||||
print_info: model params = 444.94 M
|
||||
print_info: general.name = Tts
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 158626
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
load_tensors: skipping TTS tensor emb_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor emb_text.weight (size = 467140608 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.weight (size = 12582912 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.weight (size = 2359296 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.weight (size = 6291456 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.weight (size = 1179648 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor head_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 0 repeating layers to GPU
|
||||
load_tensors: offloaded 0/21 layers to GPU
|
||||
load_tensors: CPU_Mapped model buffer size = 592.48 MiB
|
||||
..............................................................
|
||||
llama_init_from_model: model default pooling_type is [0], but [-1] was specified
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 10000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (4096) -- the full capacity of the model will not be utilized
|
||||
llama_context: CPU output buffer size = 0.61 MiB
|
||||
llama_kv_cache: CPU KV buffer size = 120.00 MiB
|
||||
llama_kv_cache: size = 120.00 MiB ( 2048 cells, 20 layers, 1/1 seqs), K (f16): 60.00 MiB, V (f16): 60.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 30.75 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 7.01 MiB
|
||||
llama_context: graph nodes = 613
|
||||
llama_context: graph splits = 216 (with bs=512), 1 (with bs=1)
|
||||
vision_model_loader: model name:
|
||||
vision_model_loader: description: image encoder for MiniCPM-V
|
||||
vision_model_loader: GGUF version: 3
|
||||
vision_model_loader: alignment: 32
|
||||
vision_model_loader: n_tensors: 455
|
||||
vision_model_loader: n_kv: 20
|
||||
|
||||
vision_ctx: vision using Vulkan0 backend
|
||||
load_hparams: model_type: 0
|
||||
load_hparams: n_embd: 1152
|
||||
load_hparams: n_head: 16
|
||||
load_hparams: n_ff: 4304
|
||||
load_hparams: n_layer: 27
|
||||
load_hparams: ffn_op: gelu
|
||||
load_hparams: projection_dim: 0
|
||||
|
||||
--- vision hparams ---
|
||||
load_hparams: image_size: 448
|
||||
load_hparams: patch_size: 14
|
||||
load_hparams: minicpmv_version: 100045
|
||||
load_hparams: proj_scale_factor: 0
|
||||
load_hparams: n_wa_pattern: 0
|
||||
|
||||
load_hparams: model size: 1044.36 MiB
|
||||
load_hparams: metadata size: 0.16 MiB
|
||||
load_tensors: ffn up/down are swapped
|
||||
alloc_compute_meta: Vulkan0 compute buffer size = 100.30 MiB
|
||||
alloc_compute_meta: CPU compute buffer size = 18.30 MiB
|
||||
init tts....init t2w....voc_hg2_model: CPU backend using 8 threads
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=32000
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=200, n_mel=80, total_size=16000
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=200, n_mel=80, total_elements=16000
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=200, n_tokens=100, current_iter=0, current_total=0, new_total=100, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 1 (total_tokens=100, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 20]
|
||||
audition_audio_batch_encode: Final output: 20 tokens x 4096 dims
|
||||
=== Model Paths ===
|
||||
Base dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf
|
||||
LLM: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf [OK]
|
||||
Vision: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/vision/MiniCPM-o-4_5-vision-F16.gguf [OK]
|
||||
Audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf [OK]
|
||||
TTS: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf [OK]
|
||||
Projector: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf [OK]
|
||||
===================
|
||||
=== Initializing Omni Context ===
|
||||
Media type: 2 (omni: audio+vision)
|
||||
TTS enabled: yes
|
||||
Context size: 2048
|
||||
GPU layers: 99
|
||||
Token2Wav device: cpu
|
||||
Vision backend: metal
|
||||
TTS bin dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts
|
||||
Ref audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\vllm-omni-edge\benchmarks\edge_harness\results\e2e_expansion_20260923\evidence\minicpmo_cpp_radeon890m_prompted\fixture_0000.wav
|
||||
02:57:19.826 === omni_init start
|
||||
02:57:19.826 media_type = 2, duplex_mode = 0, base_output_dir = ./tools/omni/output
|
||||
02:57:19.826 === omni_init: loading new LLM model
|
||||
02:58:15.498 === omni_init: loading TTS model
|
||||
02:58:15.498 TTS model: loading with n_gpu_layers=0
|
||||
02:58:19.149 TTS sampler: temp=0.80, top_p=0.85, top_k=25, rep_penalty=1.05
|
||||
02:58:19.149 TTS: loading weights from GGUF (emb_code, emb_text, projector_semantic, head_code)...
|
||||
02:58:24.603 TTS: head_code shape: dim0=6562, dim1=768, will transpose to [6562, 768]
|
||||
02:58:24.610 TTS: head_code loaded, verifying first few values:
|
||||
02:58:24.610 head_code_weight[0] = 0.01361084 (expect ~0.01385498)
|
||||
02:58:24.610 head_code_weight[1] = -0.01660156 (expect ~-0.01647949)
|
||||
02:58:24.610 head_code_weight[768] = 0.04101562 (this is weight[1, 0], expect ~0.04150391)
|
||||
02:58:24.662 TTS: weights loaded successfully
|
||||
02:58:24.662 Projector: loading from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf
|
||||
02:58:24.764 Projector: loaded successfully
|
||||
02:58:24.765 === omni_init: loading APM model
|
||||
02:58:27.797 APM: init from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf
|
||||
02:58:33.423 Token2Wav: found models in \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf
|
||||
02:58:33.481 Token2Wav: all model files found, initializing session...
|
||||
02:58:33.481 Token2Wav: non-CUDA backend, vocoder using CPU for better performance
|
||||
02:58:33.481 Token2Wav: using prompt_cache from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf/prompt_cache.gguf
|
||||
02:58:38.019 Token2Wav: initialized successfully
|
||||
02:58:38.019 Python T2W: script_dir=./tools/omni/pyt2w, model_dir=./tools/omni/pyt2w/token2wav
|
||||
02:58:38.019 Token2Wav: 浣跨敤 C++ 瀹炵幇
|
||||
02:58:38.020 === omni_init success: ctx_llama = 00000176fcc9f830
|
||||
=== Running test case ===
|
||||
Audio prefix: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-21-2jf0zhae\fixture_
|
||||
Count: 2
|
||||
02:58:38.020 stream_prefill: n_past = 0
|
||||
voice_clone_prompt = <|im_start|>system
|
||||
妯′豢闊抽�鏍锋湰鐨勯煶鑹插苟鐢熸垚鏂扮殑鍐呭�銆�
|
||||
<|audio_start|>
|
||||
assistant_prompt = <|audio_end|>浣犵殑浠诲姟鏄�敤杩欑�澹伴煶妯″紡鏉ュ綋涓€涓�姪鎵嬨€傝�璁ょ湡銆侀珮璐ㄩ噺鍦板洖澶嶇敤鎴风殑闂��銆傝�鐢ㄩ珮鑷�劧搴︾殑鏂瑰紡鍜岀敤鎴疯亰澶┿€�<|im_end|>
|
||||
<|im_start|>user
|
||||
|
||||
02:58:38.020 system prompt ref_audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-21-2jf0zhae\fixture_0000.wav
|
||||
02:58:38.674 system prompt ref_audio embedding: n_pos=20
|
||||
02:58:39.166 馃敀 n_keep 璁剧疆涓� 72 (system prompt tokens)锛岃繖閮ㄥ垎姘歌繙涓嶄細琚�粦鍔ㄧ獥鍙ｅ垹闄�
|
||||
02:58:39.166 stream_prefill(index=0): system prompt 鍒濆�鍖栧畬鎴愶紝ref_audio 宸插湪鍏朵腑 prefill
|
||||
02:58:39.166 [SW] system_prompt registered: preserve_length=72 (will be protected from sliding)
|
||||
02:58:39.166 n_past = 72
|
||||
02:58:39.166
|
||||
|
||||
c++ finish stream_prefill(index=0). n_past=72, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 0 (audio) : 1.14662 s
|
||||
encode_image_with_vision_chunks: step 1 of 1 encoded in 624.48 ms
|
||||
encode_image_with_vision_chunks: all 1 chunks encoded in 642.83 ms (grid: 1x1)
|
||||
|
||||
encode_image_with_vision_chunks: image encoded in 642.84 ms by vision ( 642.84 ms per chunk, 64 total tokens)
|
||||
omni_image_embed_make_chunks_with_filename: created 1 vision chunks from C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-21-2jf0zhae\fixture_0001.jpg
|
||||
stream_prefill: prefilled 1 vision chunks (64 tokens each)
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=49339
|
||||
audition_audio_preprocess: Audio not aligned to 100ms (49339 samples = 3083.7ms), padded with silence to 49600 samples (3100ms)
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=310, n_mel=80, total_size=24800
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=310, n_mel=80, total_elements=24800
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=310, n_tokens=155, current_iter=1, current_total=155, new_total=310, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 2 (total_tokens=310, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 31]
|
||||
audition_audio_batch_encode: Final output: 31 tokens x 4096 dims
|
||||
02:58:40.324 stream_prefill(index=1): processing user audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-21-2jf0zhae\fixture_0001.wav
|
||||
02:58:40.642 stream_prefill(index=1): user audio embedding: n_pos=31
|
||||
02:58:40.952
|
||||
|
||||
c++ finish stream_prefill(index=1). n_past=172, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 1 (audio+vision) : 1.78447 s
|
||||
02:58:40.952 馃搷 stream_decode 寮€濮�: n_past=172, n_keep=72, n_ctx=2048, duplex_mode=0
|
||||
02:58:40.952 stream_decode: create tts thread (simplex mode)
|
||||
02:58:40.952 stream_decode: create t2w thread
|
||||
02:58:40.952 wait prefill done
|
||||
02:58:40.952 馃搷 [鍗曞伐TTS] 娣诲姞 assistant prompt: "<|im_end|>
|
||||
<|im_start|>assistant
|
||||
<think>
|
||||
|
||||
</think>
|
||||
|
||||
<|tts_bos|>", n_past=172
|
||||
02:58:40.953 T2W thread (C++) started
|
||||
<user>
|
||||
<assistant>02:58:40.956 TTS: 鍒涘缓鍗曞伐妯″紡杈撳嚭鐩�綍: ./tools/omni/output/round_000
|
||||
02:58:40.957 TTS thread started
|
||||
02:58:41.114 馃搷 [鍗曞伐TTS] assistant prompt 瀹屾垚, n_past=182
|
||||
02:58:41.114 LLM decode: max_tgt_len = 96, n_predict = 96, n_ctx = 2048
|
||||
02:58:41.628 LLM: detected end token, set llm_generation_done=true
|
||||
02:58:41.628 LLM->TTS: text='The square in the image is red.', n_tokens=8, hidden_size=32768, n_embd=4096, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:58:41.628 馃搷 Decode 缁撴潫: n_past=191, 鍓╀綑绌洪棿=1857, 鏃犻渶婊戠獥
|
||||
02:58:41.628 馃搷 杞�� 1 缁撴潫锛岃�褰曡竟鐣屼簬 n_past=191
|
||||
02:58:41.628 TTS<-LLM: chunk_idx=0, text='The square in the image is red.', n_tokens=8, hidden_size=32768, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:58:41.628 TTS: after queue pop - speek_done=0, llm_finish=1, llm_text.empty=0, token_ids.size=8
|
||||
02:58:41.628 TTS: before empty check - response.empty=0, response='The square in the image is red.', llm_finish=1
|
||||
Waiting for audio generation to complete...
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_0.wav does not exist or is empty
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_1.wav does not exist or is empty
|
||||
TTS: no valid WAV files to merge
|
||||
02:58:41.628 TTS: DEBUG before has_llm_data - token_ids.size=8, hidden_states.size=32768, n_embd=4096
|
||||
02:58:41.628 TTS: has_llm_data=1, token_ids=8, hidden_states=32768, n_embd=4096, llm_finish=1
|
||||
02:58:41.629 TTS: n_tokens_orig=8, n_tokens_filtered=8 (filtered 0 special tokens)
|
||||
02:58:41.659 TTS: saved merged embeddings (text) to ./tools/omni/output/round_000/llm_debug/chunk_0/merged_embeddings.txt
|
||||
02:58:41.659 TTS Simplex: generating audio tokens for chunk 0 (n_tokens=8, tts_n_embd=768)
|
||||
02:58:41.659 TTS Simplex: 鍦� prefill 鍓嶆坊鍔� audio_bos (chunk_idx=0, new_size=9)
|
||||
02:58:41.659 TTS Simplex: first chunk - cleared KV cache
|
||||
02:58:41.659 TTS Simplex: first chunk - cleared tts_all_generated_tokens and tts_token_buffer
|
||||
02:58:41.721 馃搷 涓轰笅涓€杞�噯澶�: eval <|im_end|>\n<|im_start|>user\n, n_past=196
|
||||
02:58:41.930 TTS Simplex: prefill completed, n_past_tts=9
|
||||
02:58:41.930 TTS Simplex: sampler created
|
||||
02:58:41.930 TTS simplex: is_audio_bos=true (first audio token)
|
||||
02:58:41.955 TTS Simplex Phase1: token 1/500: rel_id=1487
|
||||
02:58:41.966 TTS Simplex Phase1: token 2/500: rel_id=1959
|
||||
02:58:41.978 TTS Simplex Phase1: token 3/500: rel_id=4137
|
||||
02:58:41.989 TTS Simplex Phase1: token 4/500: rel_id=4218
|
||||
02:58:42.000 TTS Simplex Phase1: token 5/500: rel_id=4137
|
||||
02:58:42.216 TTS Simplex Phase1: yield 25 tokens 鍒� T2W (step 25, buffer=25)
|
||||
02:58:42.216 TTS Simplex Phase1: token 25/500: rel_id=2501
|
||||
02:58:42.216 T2W绾跨▼(C++): 杞��鍒囨崲 (-1 -> 0)锛屾洿鏂拌緭鍑虹洰褰�
|
||||
02:58:42.216 T2W绾跨▼(C++): 鏂拌緭鍑虹洰褰� ./tools/omni/output/round_000/tts_wav
|
||||
02:58:42.429 TTS Simplex Phase1: EOS token at step 45
|
||||
02:58:42.429 TTS Simplex: is_final_text_chunk=true, will enter Phase 2 for text_eos_embed
|
||||
02:58:42.429 TTS Simplex Phase2: injecting text_eos_embed + audio_bos at n_past=53
|
||||
02:58:42.436 TTS Simplex Phase2: text_eos_embed injected, n_past=54
|
||||
02:58:42.444 TTS Simplex Phase2: audio_bos injected, n_past=55, starting final generation
|
||||
02:58:42.510 TTS Simplex Phase2: yield 25 tokens 鍒� T2W
|
||||
02:58:42.685 TTS Simplex Phase2: EOS at step 22 (final end)
|
||||
02:58:42.685 TTS Simplex: chunk 缁撴潫锛宼ts_token_buffer 鍓╀綑 15 tokens (淇濈暀绛変笅涓€涓� chunk)
|
||||
02:58:42.685 TTS Simplex: is_final_text_chunk=true, flush 鍓╀綑 15 tokens 浠� tts_token_buffer
|
||||
02:58:42.685 TTS Simplex: generated 65 audio tokens for chunk 0
|
||||
02:58:42.685 TTS Simplex: chunk 0 first tokens: [1487 1959 4137 4218 4137 ], last tokens: [1488 4218 4218 4218 4137 ]
|
||||
02:58:42.685 TTS Simplex: updated tts_n_past_accumulated=77, total_generated_tokens=65
|
||||
02:58:42.696 TTS: finished processing all chunks
|
||||
02:58:42.696 TTS: 鍗曞伐妯″紡杞��缁撴潫锛屼笅涓€杞� round_idx=1
|
||||
02:58:42.696 TTS: sent is_final=true to T2W queue (turn end)
|
||||
02:58:44.569 馃帀 棣栧搷鏃堕棿 (First Audio Response): 3616ms
|
||||
02:58:44.569 T2W绾跨▼: wav_0.wav | 0.84s audio | 2351.6ms inference | RTF=2.80 | t=3616ms
|
||||
02:58:46.811 T2W绾跨▼: wav_1.wav | 1.00s audio | 2241.3ms inference | RTF=2.24 | t=5858ms
|
||||
02:58:48.986 T2W绾跨▼: wav_2.wav | 0.88s audio | 2174.1ms inference | RTF=2.47 | t=8033ms
|
||||
02:58:48.986 T2W绾跨▼: 鍐欏叆缁撴潫鏍囪� ./tools/omni/output/round_000/tts_wav/generation_done.flag (last_wav=2)
|
||||
02:58:48.986 T2W绾跨▼: 杞��缁撴潫鍚庢洿鏂拌緭鍑虹洰褰曚负 ./tools/omni/output/round_001/tts_wav
|
||||
02:58:48.997 omni_stop_threads: stop signals sent
|
||||
02:58:48.997 T2W(C++) 绾跨▼: 鍋滄�
|
||||
[timing] call=0 tokens=28 final=0 token2mel=2019.335ms vocoder=332.188ms total=2351.536ms audio=20160
|
||||
[timing] call=1 tokens=28 final=0 token2mel=1858.889ms vocoder=382.342ms total=2241.243ms audio=24000
|
||||
[timing] call=2 tokens=18 final=1 token2mel=1877.159ms vocoder=296.909ms total=2174.077ms audio=21120
|
||||
Audio generation completed.
|
||||
llama_perf_context_print: load time = 25483.27 ms
|
||||
llama_perf_context_print: prompt eval time = 3093.56 ms / 182 tokens ( 17.00 ms per token, 58.83 tokens per second)
|
||||
llama_perf_context_print: eval time = 511.53 ms / 9 runs ( 56.84 ms per token, 17.59 tokens per second)
|
||||
llama_perf_context_print: total time = 33365.85 ms / 191 tokens
|
||||
llama_perf_context_print: graphs reused = 10
|
||||
Token2Wav (C++): session released
|
||||
tts thread end
|
||||
t2w thread end
|
||||
Whitespace-only changes.
+484
@@ -0,0 +1,484 @@
|
||||
build: 64 (5202b7b) with x86_64-w64-mingw32-gcc-posix (GCC) 13-posix for x86_64-w64-mingw32
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 30720 MiB free
|
||||
llama_model_loader: loaded meta data with 27 key-value pairs and 399 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = qwen3
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Llm
|
||||
llama_model_loader: - kv 3: general.size_label str = 8.2B
|
||||
llama_model_loader: - kv 4: qwen3.block_count u32 = 36
|
||||
llama_model_loader: - kv 5: qwen3.context_length u32 = 40960
|
||||
llama_model_loader: - kv 6: qwen3.embedding_length u32 = 4096
|
||||
llama_model_loader: - kv 7: qwen3.feed_forward_length u32 = 12288
|
||||
llama_model_loader: - kv 8: qwen3.attention.head_count u32 = 32
|
||||
llama_model_loader: - kv 9: qwen3.attention.head_count_kv u32 = 8
|
||||
llama_model_loader: - kv 10: qwen3.rope.freq_base f32 = 1000000.000000
|
||||
llama_model_loader: - kv 11: qwen3.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: qwen3.attention.key_length u32 = 128
|
||||
llama_model_loader: - kv 13: qwen3.attention.value_length u32 = 128
|
||||
llama_model_loader: - kv 14: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 15: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 16: tokenizer.ggml.tokens arr[str,151748] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 17: tokenizer.ggml.token_type arr[i32,151748] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 24: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 25: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 26: general.file_type u32 = 15
|
||||
llama_model_loader: - type f32: 145 tensors
|
||||
llama_model_loader: - type q4_K: 217 tensors
|
||||
llama_model_loader: - type q6_K: 37 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = Q4_K - Medium
|
||||
print_info: file size = 4.68 GiB (4.90 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = qwen3
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 40960
|
||||
print_info: n_embd = 4096
|
||||
print_info: n_layer = 36
|
||||
print_info: n_head = 32
|
||||
print_info: n_head_kv = 8
|
||||
print_info: n_rot = 128
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 128
|
||||
print_info: n_embd_head_v = 128
|
||||
print_info: n_gqa = 4
|
||||
print_info: n_embd_k_gqa = 1024
|
||||
print_info: n_embd_v_gqa = 1024
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 12288
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = -1
|
||||
print_info: rope type = 2
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 1000000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 40960
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = 8B
|
||||
print_info: model params = 8.19 B
|
||||
print_info: general.name = Llm
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 151748
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 36 repeating layers to GPU
|
||||
load_tensors: offloading output layer to GPU
|
||||
load_tensors: offloaded 37/37 layers to GPU
|
||||
load_tensors: Vulkan0 model buffer size = 4454.74 MiB
|
||||
load_tensors: CPU_Mapped model buffer size = 333.43 MiB
|
||||
.....................................................................................
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 1000000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (40960) -- the full capacity of the model will not be utilized
|
||||
llama_context: Vulkan_Host output buffer size = 0.58 MiB
|
||||
llama_kv_cache: Vulkan0 KV buffer size = 288.00 MiB
|
||||
llama_kv_cache: size = 288.00 MiB ( 2048 cells, 36 layers, 1/1 seqs), K (f16): 144.00 MiB, V (f16): 144.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 304.38 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 12.01 MiB
|
||||
llama_context: graph nodes = 1267
|
||||
llama_context: graph splits = 2
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 25673 MiB free
|
||||
llama_model_loader: loaded meta data with 30 key-value pairs and 193 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = llama
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Tts
|
||||
llama_model_loader: - kv 3: general.size_label str = 445M
|
||||
llama_model_loader: - kv 4: llama.block_count u32 = 20
|
||||
llama_model_loader: - kv 5: llama.context_length u32 = 4096
|
||||
llama_model_loader: - kv 6: llama.embedding_length u32 = 768
|
||||
llama_model_loader: - kv 7: llama.feed_forward_length u32 = 3072
|
||||
llama_model_loader: - kv 8: llama.attention.head_count u32 = 12
|
||||
llama_model_loader: - kv 9: llama.attention.head_count_kv u32 = 12
|
||||
llama_model_loader: - kv 10: llama.rope.freq_base f32 = 10000.000000
|
||||
llama_model_loader: - kv 11: llama.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: llama.attention.key_length u32 = 64
|
||||
llama_model_loader: - kv 13: llama.attention.value_length u32 = 64
|
||||
llama_model_loader: - kv 14: general.file_type u32 = 1
|
||||
llama_model_loader: - kv 15: llama.vocab_size u32 = 158626
|
||||
llama_model_loader: - kv 16: llama.rope.dimension_count u32 = 64
|
||||
llama_model_loader: - kv 17: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.tokens arr[str,158626] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.token_type arr[i32,158626] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 24: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 25: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 26: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 27: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 28: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 29: tokenizer.ggml.add_space_prefix bool = false
|
||||
llama_model_loader: - type f32: 50 tensors
|
||||
llama_model_loader: - type f16: 143 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = F16
|
||||
print_info: file size = 1.07 GiB (20.70 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = llama
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 4096
|
||||
print_info: n_embd = 768
|
||||
print_info: n_layer = 20
|
||||
print_info: n_head = 12
|
||||
print_info: n_head_kv = 12
|
||||
print_info: n_rot = 64
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 64
|
||||
print_info: n_embd_head_v = 64
|
||||
print_info: n_gqa = 1
|
||||
print_info: n_embd_k_gqa = 768
|
||||
print_info: n_embd_v_gqa = 768
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 3072
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = 0
|
||||
print_info: rope type = 0
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 10000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 4096
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = ?B
|
||||
print_info: model params = 444.94 M
|
||||
print_info: general.name = Tts
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 158626
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
load_tensors: skipping TTS tensor emb_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor emb_text.weight (size = 467140608 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.weight (size = 12582912 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.weight (size = 2359296 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.weight (size = 6291456 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.weight (size = 1179648 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor head_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 0 repeating layers to GPU
|
||||
load_tensors: offloaded 0/21 layers to GPU
|
||||
load_tensors: CPU_Mapped model buffer size = 592.48 MiB
|
||||
..............................................................
|
||||
llama_init_from_model: model default pooling_type is [0], but [-1] was specified
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 10000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (4096) -- the full capacity of the model will not be utilized
|
||||
llama_context: CPU output buffer size = 0.61 MiB
|
||||
llama_kv_cache: CPU KV buffer size = 120.00 MiB
|
||||
llama_kv_cache: size = 120.00 MiB ( 2048 cells, 20 layers, 1/1 seqs), K (f16): 60.00 MiB, V (f16): 60.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 30.75 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 7.01 MiB
|
||||
llama_context: graph nodes = 613
|
||||
llama_context: graph splits = 216 (with bs=512), 1 (with bs=1)
|
||||
vision_model_loader: model name:
|
||||
vision_model_loader: description: image encoder for MiniCPM-V
|
||||
vision_model_loader: GGUF version: 3
|
||||
vision_model_loader: alignment: 32
|
||||
vision_model_loader: n_tensors: 455
|
||||
vision_model_loader: n_kv: 20
|
||||
|
||||
vision_ctx: vision using Vulkan0 backend
|
||||
load_hparams: model_type: 0
|
||||
load_hparams: n_embd: 1152
|
||||
load_hparams: n_head: 16
|
||||
load_hparams: n_ff: 4304
|
||||
load_hparams: n_layer: 27
|
||||
load_hparams: ffn_op: gelu
|
||||
load_hparams: projection_dim: 0
|
||||
|
||||
--- vision hparams ---
|
||||
load_hparams: image_size: 448
|
||||
load_hparams: patch_size: 14
|
||||
load_hparams: minicpmv_version: 100045
|
||||
load_hparams: proj_scale_factor: 0
|
||||
load_hparams: n_wa_pattern: 0
|
||||
|
||||
load_hparams: model size: 1044.36 MiB
|
||||
load_hparams: metadata size: 0.16 MiB
|
||||
load_tensors: ffn up/down are swapped
|
||||
alloc_compute_meta: Vulkan0 compute buffer size = 100.30 MiB
|
||||
alloc_compute_meta: CPU compute buffer size = 18.30 MiB
|
||||
init tts....init t2w....voc_hg2_model: CPU backend using 8 threads
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=32000
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=200, n_mel=80, total_size=16000
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=200, n_mel=80, total_elements=16000
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=200, n_tokens=100, current_iter=0, current_total=0, new_total=100, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 1 (total_tokens=100, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 20]
|
||||
audition_audio_batch_encode: Final output: 20 tokens x 4096 dims
|
||||
=== Model Paths ===
|
||||
Base dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf
|
||||
LLM: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf [OK]
|
||||
Vision: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/vision/MiniCPM-o-4_5-vision-F16.gguf [OK]
|
||||
Audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf [OK]
|
||||
TTS: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf [OK]
|
||||
Projector: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf [OK]
|
||||
===================
|
||||
=== Initializing Omni Context ===
|
||||
Media type: 2 (omni: audio+vision)
|
||||
TTS enabled: yes
|
||||
Context size: 2048
|
||||
GPU layers: 99
|
||||
Token2Wav device: cpu
|
||||
Vision backend: metal
|
||||
TTS bin dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts
|
||||
Ref audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\vllm-omni-edge\benchmarks\edge_harness\results\e2e_expansion_20260923\evidence\minicpmo_cpp_radeon890m_prompted\fixture_0000.wav
|
||||
02:31:23.750 === omni_init start
|
||||
02:31:23.750 media_type = 2, duplex_mode = 0, base_output_dir = ./tools/omni/output
|
||||
02:31:23.750 === omni_init: loading new LLM model
|
||||
02:32:15.536 === omni_init: loading TTS model
|
||||
02:32:15.536 TTS model: loading with n_gpu_layers=0
|
||||
02:32:19.053 TTS sampler: temp=0.80, top_p=0.85, top_k=25, rep_penalty=1.05
|
||||
02:32:19.053 TTS: loading weights from GGUF (emb_code, emb_text, projector_semantic, head_code)...
|
||||
02:32:24.501 TTS: head_code shape: dim0=6562, dim1=768, will transpose to [6562, 768]
|
||||
02:32:24.507 TTS: head_code loaded, verifying first few values:
|
||||
02:32:24.507 head_code_weight[0] = 0.01361084 (expect ~0.01385498)
|
||||
02:32:24.507 head_code_weight[1] = -0.01660156 (expect ~-0.01647949)
|
||||
02:32:24.507 head_code_weight[768] = 0.04101562 (this is weight[1, 0], expect ~0.04150391)
|
||||
02:32:24.555 TTS: weights loaded successfully
|
||||
02:32:24.555 Projector: loading from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf
|
||||
02:32:24.663 Projector: loaded successfully
|
||||
02:32:24.663 === omni_init: loading APM model
|
||||
02:32:27.730 APM: init from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf
|
||||
02:32:33.482 Token2Wav: found models in \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf
|
||||
02:32:33.545 Token2Wav: all model files found, initializing session...
|
||||
02:32:33.545 Token2Wav: non-CUDA backend, vocoder using CPU for better performance
|
||||
02:32:33.545 Token2Wav: using prompt_cache from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf/prompt_cache.gguf
|
||||
02:32:38.397 Token2Wav: initialized successfully
|
||||
02:32:38.397 Python T2W: script_dir=./tools/omni/pyt2w, model_dir=./tools/omni/pyt2w/token2wav
|
||||
02:32:38.397 Token2Wav: 浣跨敤 C++ 瀹炵幇
|
||||
02:32:38.397 === omni_init success: ctx_llama = 00000203d98cfe80
|
||||
=== Running test case ===
|
||||
Audio prefix: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-3-tuhz94cx\fixture_
|
||||
Count: 2
|
||||
02:32:38.397 stream_prefill: n_past = 0
|
||||
voice_clone_prompt = <|im_start|>system
|
||||
妯′豢闊抽�鏍锋湰鐨勯煶鑹插苟鐢熸垚鏂扮殑鍐呭�銆�
|
||||
<|audio_start|>
|
||||
assistant_prompt = <|audio_end|>浣犵殑浠诲姟鏄�敤杩欑�澹伴煶妯″紡鏉ュ綋涓€涓�姪鎵嬨€傝�璁ょ湡銆侀珮璐ㄩ噺鍦板洖澶嶇敤鎴风殑闂��銆傝�鐢ㄩ珮鑷�劧搴︾殑鏂瑰紡鍜岀敤鎴疯亰澶┿€�<|im_end|>
|
||||
<|im_start|>user
|
||||
|
||||
02:32:38.397 system prompt ref_audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-3-tuhz94cx\fixture_0000.wav
|
||||
02:32:38.798 system prompt ref_audio embedding: n_pos=20
|
||||
02:32:39.357 馃敀 n_keep 璁剧疆涓� 72 (system prompt tokens)锛岃繖閮ㄥ垎姘歌繙涓嶄細琚�粦鍔ㄧ獥鍙ｅ垹闄�
|
||||
02:32:39.357 stream_prefill(index=0): system prompt 鍒濆�鍖栧畬鎴愶紝ref_audio 宸插湪鍏朵腑 prefill
|
||||
02:32:39.357 [SW] system_prompt registered: preserve_length=72 (will be protected from sliding)
|
||||
02:32:39.357 n_past = 72
|
||||
02:32:39.357
|
||||
|
||||
c++ finish stream_prefill(index=0). n_past=72, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 0 (audio) : 0.960375 s
|
||||
encode_image_with_vision_chunks: step 1 of 1 encoded in 863.16 ms
|
||||
encode_image_with_vision_chunks: all 1 chunks encoded in 880.76 ms (grid: 1x1)
|
||||
|
||||
encode_image_with_vision_chunks: image encoded in 880.77 ms by vision ( 880.77 ms per chunk, 64 total tokens)
|
||||
omni_image_embed_make_chunks_with_filename: created 1 vision chunks from C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-3-tuhz94cx\fixture_0001.jpg
|
||||
stream_prefill: prefilled 1 vision chunks (64 tokens each)
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=49339
|
||||
audition_audio_preprocess: Audio not aligned to 100ms (49339 samples = 3083.7ms), padded with silence to 49600 samples (3100ms)
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=310, n_mel=80, total_size=24800
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=310, n_mel=80, total_elements=24800
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=310, n_tokens=155, current_iter=1, current_total=155, new_total=310, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 2 (total_tokens=310, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 31]
|
||||
audition_audio_batch_encode: Final output: 31 tokens x 4096 dims
|
||||
02:32:40.792 stream_prefill(index=1): processing user audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-3-tuhz94cx\fixture_0001.wav
|
||||
02:32:41.104 stream_prefill(index=1): user audio embedding: n_pos=31
|
||||
02:32:41.441
|
||||
|
||||
c++ finish stream_prefill(index=1). n_past=172, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 1 (audio+vision) : 2.08277 s
|
||||
02:32:41.452 馃搷 stream_decode 寮€濮�: n_past=172, n_keep=72, n_ctx=2048, duplex_mode=0
|
||||
02:32:41.452 stream_decode: create tts thread (simplex mode)
|
||||
02:32:41.452 stream_decode: create t2w thread
|
||||
02:32:41.452 wait prefill done
|
||||
02:32:41.452 馃搷 [鍗曞伐TTS] 娣诲姞 assistant prompt: "<|im_end|>
|
||||
<|im_start|>assistant
|
||||
<think>
|
||||
|
||||
</think>
|
||||
|
||||
<|tts_bos|>", n_past=172
|
||||
02:32:41.452 T2W thread (C++) started
|
||||
<user>
|
||||
<assistant>02:32:41.455 TTS: 鍒涘缓鍗曞伐妯″紡杈撳嚭鐩�綍: ./tools/omni/output/round_000
|
||||
02:32:41.455 TTS thread started
|
||||
02:32:41.632 馃搷 [鍗曞伐TTS] assistant prompt 瀹屾垚, n_past=182
|
||||
02:32:41.632 LLM decode: max_tgt_len = 96, n_predict = 96, n_ctx = 2048
|
||||
02:32:42.156 LLM: detected end token, set llm_generation_done=true
|
||||
02:32:42.157 LLM->TTS: text='The square in the image is red.', n_tokens=8, hidden_size=32768, n_embd=4096, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:32:42.157 馃搷 Decode 缁撴潫: n_past=191, 鍓╀綑绌洪棿=1857, 鏃犻渶婊戠獥
|
||||
02:32:42.157 馃搷 杞�� 1 缁撴潫锛岃�褰曡竟鐣屼簬 n_past=191
|
||||
02:32:42.157 TTS<-LLM: chunk_idx=0, text='The square in the image is red.', n_tokens=8, hidden_size=32768, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:32:42.157 TTS: after queue pop - speek_done=0, llm_finish=1, llm_text.empty=0, token_ids.size=8
|
||||
02:32:42.157 TTS: before empty check - response.empty=0, response='The square in the image is red.', llm_finish=1
|
||||
Waiting for audio generation to complete...
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_0.wav does not exist or is empty
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_1.wav does not exist or is empty
|
||||
TTS: no valid WAV files to merge
|
||||
02:32:42.157 TTS: DEBUG before has_llm_data - token_ids.size=8, hidden_states.size=32768, n_embd=4096
|
||||
02:32:42.157 TTS: has_llm_data=1, token_ids=8, hidden_states=32768, n_embd=4096, llm_finish=1
|
||||
02:32:42.158 TTS: n_tokens_orig=8, n_tokens_filtered=8 (filtered 0 special tokens)
|
||||
02:32:42.191 TTS: saved merged embeddings (text) to ./tools/omni/output/round_000/llm_debug/chunk_0/merged_embeddings.txt
|
||||
02:32:42.191 TTS Simplex: generating audio tokens for chunk 0 (n_tokens=8, tts_n_embd=768)
|
||||
02:32:42.191 TTS Simplex: 鍦� prefill 鍓嶆坊鍔� audio_bos (chunk_idx=0, new_size=9)
|
||||
02:32:42.191 TTS Simplex: first chunk - cleared KV cache
|
||||
02:32:42.191 TTS Simplex: first chunk - cleared tts_all_generated_tokens and tts_token_buffer
|
||||
02:32:42.250 馃搷 涓轰笅涓€杞�噯澶�: eval <|im_end|>\n<|im_start|>user\n, n_past=196
|
||||
02:32:42.486 TTS Simplex: prefill completed, n_past_tts=9
|
||||
02:32:42.486 TTS Simplex: sampler created
|
||||
02:32:42.486 TTS simplex: is_audio_bos=true (first audio token)
|
||||
02:32:42.511 TTS Simplex Phase1: token 1/500: rel_id=2963
|
||||
02:32:42.523 TTS Simplex Phase1: token 2/500: rel_id=4218
|
||||
02:32:42.535 TTS Simplex Phase1: token 3/500: rel_id=4137
|
||||
02:32:42.546 TTS Simplex Phase1: token 4/500: rel_id=4137
|
||||
02:32:42.556 TTS Simplex Phase1: token 5/500: rel_id=4137
|
||||
02:32:42.779 TTS Simplex Phase1: yield 25 tokens 鍒� T2W (step 25, buffer=25)
|
||||
02:32:42.779 TTS Simplex Phase1: token 25/500: rel_id=6302
|
||||
02:32:42.779 T2W绾跨▼(C++): 杞��鍒囨崲 (-1 -> 0)锛屾洿鏂拌緭鍑虹洰褰�
|
||||
02:32:42.779 T2W绾跨▼(C++): 鏂拌緭鍑虹洰褰� ./tools/omni/output/round_000/tts_wav
|
||||
02:32:43.049 TTS Simplex Phase1: EOS token at step 50
|
||||
02:32:43.049 TTS Simplex: is_final_text_chunk=true, will enter Phase 2 for text_eos_embed
|
||||
02:32:43.049 TTS Simplex Phase1: token 50/500: rel_id=6561
|
||||
02:32:43.049 TTS Simplex Phase2: injecting text_eos_embed + audio_bos at n_past=58
|
||||
02:32:43.057 TTS Simplex Phase2: text_eos_embed injected, n_past=59
|
||||
02:32:43.065 TTS Simplex Phase2: audio_bos injected, n_past=60, starting final generation
|
||||
02:32:43.076 TTS Simplex Phase2: yield 25 tokens 鍒� T2W
|
||||
02:32:43.264 TTS Simplex Phase2: EOS at step 18 (final end)
|
||||
02:32:43.264 TTS Simplex: chunk 缁撴潫锛宼ts_token_buffer 鍓╀綑 16 tokens (淇濈暀绛変笅涓€涓� chunk)
|
||||
02:32:43.264 TTS Simplex: is_final_text_chunk=true, flush 鍓╀綑 16 tokens 浠� tts_token_buffer
|
||||
02:32:43.264 TTS Simplex: generated 66 audio tokens for chunk 0
|
||||
02:32:43.264 TTS Simplex: chunk 0 first tokens: [2963 4218 4137 4137 4137 ], last tokens: [36 1486 1707 4218 4137 ]
|
||||
02:32:43.264 TTS Simplex: updated tts_n_past_accumulated=78, total_generated_tokens=66
|
||||
02:32:43.266 TTS: finished processing all chunks
|
||||
02:32:43.266 TTS: 鍗曞伐妯″紡杞��缁撴潫锛屼笅涓€杞� round_idx=1
|
||||
02:32:43.266 TTS: sent is_final=true to T2W queue (turn end)
|
||||
02:32:45.205 馃帀 棣栧搷鏃堕棿 (First Audio Response): 3752ms
|
||||
02:32:45.205 T2W绾跨▼: wav_0.wav | 0.84s audio | 2424.0ms inference | RTF=2.89 | t=3752ms
|
||||
02:32:47.486 T2W绾跨▼: wav_1.wav | 1.00s audio | 2280.5ms inference | RTF=2.28 | t=6034ms
|
||||
02:32:49.700 T2W绾跨▼: wav_2.wav | 0.92s audio | 2213.2ms inference | RTF=2.41 | t=8247ms
|
||||
02:32:49.700 T2W绾跨▼: 鍐欏叆缁撴潫鏍囪� ./tools/omni/output/round_000/tts_wav/generation_done.flag (last_wav=2)
|
||||
02:32:49.700 T2W绾跨▼: 杞��缁撴潫鍚庢洿鏂拌緭鍑虹洰褰曚负 ./tools/omni/output/round_001/tts_wav
|
||||
02:32:02:49.32:738 omni_stop_threads: stop signals sent
|
||||
49.738 T2W(C++) 绾跨▼: 鍋滄�
|
||||
[timing] call=0 tokens=28 final=0 token2mel=2080.289ms vocoder=342.817ms total=2423.505ms audio=20160
|
||||
[timing] call=1 tokens=28 final=0 token2mel=1882.980ms vocoder=397.498ms total=2280.489ms audio=24000
|
||||
[timing] call=2 tokens=19 final=1 token2mel=1904.339ms vocoder=308.800ms total=2213.151ms audio=22080
|
||||
Audio generation completed.
|
||||
llama_perf_context_print: load time = 25709.10 ms
|
||||
llama_perf_context_print: prompt eval time = 3233.37 ms / 182 tokens ( 17.77 ms per token, 56.29 tokens per second)
|
||||
llama_perf_context_print: eval time = 520.35 ms / 9 runs ( 57.82 ms per token, 17.30 tokens per second)
|
||||
llama_perf_context_print: total time = 33814.38 ms / 191 tokens
|
||||
llama_perf_context_print: graphs reused = 10
|
||||
Token2Wav (C++): session released
|
||||
tts thread end
|
||||
t2w thread end
|
||||
+479
@@ -0,0 +1,479 @@
|
||||
build: 64 (5202b7b) with x86_64-w64-mingw32-gcc-posix (GCC) 13-posix for x86_64-w64-mingw32
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 30720 MiB free
|
||||
llama_model_loader: loaded meta data with 27 key-value pairs and 399 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = qwen3
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Llm
|
||||
llama_model_loader: - kv 3: general.size_label str = 8.2B
|
||||
llama_model_loader: - kv 4: qwen3.block_count u32 = 36
|
||||
llama_model_loader: - kv 5: qwen3.context_length u32 = 40960
|
||||
llama_model_loader: - kv 6: qwen3.embedding_length u32 = 4096
|
||||
llama_model_loader: - kv 7: qwen3.feed_forward_length u32 = 12288
|
||||
llama_model_loader: - kv 8: qwen3.attention.head_count u32 = 32
|
||||
llama_model_loader: - kv 9: qwen3.attention.head_count_kv u32 = 8
|
||||
llama_model_loader: - kv 10: qwen3.rope.freq_base f32 = 1000000.000000
|
||||
llama_model_loader: - kv 11: qwen3.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: qwen3.attention.key_length u32 = 128
|
||||
llama_model_loader: - kv 13: qwen3.attention.value_length u32 = 128
|
||||
llama_model_loader: - kv 14: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 15: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 16: tokenizer.ggml.tokens arr[str,151748] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 17: tokenizer.ggml.token_type arr[i32,151748] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 24: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 25: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 26: general.file_type u32 = 15
|
||||
llama_model_loader: - type f32: 145 tensors
|
||||
llama_model_loader: - type q4_K: 217 tensors
|
||||
llama_model_loader: - type q6_K: 37 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = Q4_K - Medium
|
||||
print_info: file size = 4.68 GiB (4.90 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = qwen3
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 40960
|
||||
print_info: n_embd = 4096
|
||||
print_info: n_layer = 36
|
||||
print_info: n_head = 32
|
||||
print_info: n_head_kv = 8
|
||||
print_info: n_rot = 128
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 128
|
||||
print_info: n_embd_head_v = 128
|
||||
print_info: n_gqa = 4
|
||||
print_info: n_embd_k_gqa = 1024
|
||||
print_info: n_embd_v_gqa = 1024
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 12288
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = -1
|
||||
print_info: rope type = 2
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 1000000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 40960
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = 8B
|
||||
print_info: model params = 8.19 B
|
||||
print_info: general.name = Llm
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 151748
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 36 repeating layers to GPU
|
||||
load_tensors: offloading output layer to GPU
|
||||
load_tensors: offloaded 37/37 layers to GPU
|
||||
load_tensors: Vulkan0 model buffer size = 4454.74 MiB
|
||||
load_tensors: CPU_Mapped model buffer size = 333.43 MiB
|
||||
.....................................................................................
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 1000000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (40960) -- the full capacity of the model will not be utilized
|
||||
llama_context: Vulkan_Host output buffer size = 0.58 MiB
|
||||
llama_kv_cache: Vulkan0 KV buffer size = 288.00 MiB
|
||||
llama_kv_cache: size = 288.00 MiB ( 2048 cells, 36 layers, 1/1 seqs), K (f16): 144.00 MiB, V (f16): 144.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 304.38 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 12.01 MiB
|
||||
llama_context: graph nodes = 1267
|
||||
llama_context: graph splits = 2
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 25673 MiB free
|
||||
llama_model_loader: loaded meta data with 30 key-value pairs and 193 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = llama
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Tts
|
||||
llama_model_loader: - kv 3: general.size_label str = 445M
|
||||
llama_model_loader: - kv 4: llama.block_count u32 = 20
|
||||
llama_model_loader: - kv 5: llama.context_length u32 = 4096
|
||||
llama_model_loader: - kv 6: llama.embedding_length u32 = 768
|
||||
llama_model_loader: - kv 7: llama.feed_forward_length u32 = 3072
|
||||
llama_model_loader: - kv 8: llama.attention.head_count u32 = 12
|
||||
llama_model_loader: - kv 9: llama.attention.head_count_kv u32 = 12
|
||||
llama_model_loader: - kv 10: llama.rope.freq_base f32 = 10000.000000
|
||||
llama_model_loader: - kv 11: llama.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: llama.attention.key_length u32 = 64
|
||||
llama_model_loader: - kv 13: llama.attention.value_length u32 = 64
|
||||
llama_model_loader: - kv 14: general.file_type u32 = 1
|
||||
llama_model_loader: - kv 15: llama.vocab_size u32 = 158626
|
||||
llama_model_loader: - kv 16: llama.rope.dimension_count u32 = 64
|
||||
llama_model_loader: - kv 17: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.tokens arr[str,158626] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.token_type arr[i32,158626] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 24: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 25: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 26: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 27: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 28: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 29: tokenizer.ggml.add_space_prefix bool = false
|
||||
llama_model_loader: - type f32: 50 tensors
|
||||
llama_model_loader: - type f16: 143 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = F16
|
||||
print_info: file size = 1.07 GiB (20.70 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = llama
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 4096
|
||||
print_info: n_embd = 768
|
||||
print_info: n_layer = 20
|
||||
print_info: n_head = 12
|
||||
print_info: n_head_kv = 12
|
||||
print_info: n_rot = 64
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 64
|
||||
print_info: n_embd_head_v = 64
|
||||
print_info: n_gqa = 1
|
||||
print_info: n_embd_k_gqa = 768
|
||||
print_info: n_embd_v_gqa = 768
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 3072
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = 0
|
||||
print_info: rope type = 0
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 10000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 4096
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = ?B
|
||||
print_info: model params = 444.94 M
|
||||
print_info: general.name = Tts
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 158626
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
load_tensors: skipping TTS tensor emb_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor emb_text.weight (size = 467140608 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.weight (size = 12582912 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.weight (size = 2359296 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.weight (size = 6291456 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.weight (size = 1179648 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor head_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 0 repeating layers to GPU
|
||||
load_tensors: offloaded 0/21 layers to GPU
|
||||
load_tensors: CPU_Mapped model buffer size = 592.48 MiB
|
||||
..............................................................
|
||||
llama_init_from_model: model default pooling_type is [0], but [-1] was specified
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 10000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (4096) -- the full capacity of the model will not be utilized
|
||||
llama_context: CPU output buffer size = 0.61 MiB
|
||||
llama_kv_cache: CPU KV buffer size = 120.00 MiB
|
||||
llama_kv_cache: size = 120.00 MiB ( 2048 cells, 20 layers, 1/1 seqs), K (f16): 60.00 MiB, V (f16): 60.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 30.75 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 7.01 MiB
|
||||
llama_context: graph nodes = 613
|
||||
llama_context: graph splits = 216 (with bs=512), 1 (with bs=1)
|
||||
vision_model_loader: model name:
|
||||
vision_model_loader: description: image encoder for MiniCPM-V
|
||||
vision_model_loader: GGUF version: 3
|
||||
vision_model_loader: alignment: 32
|
||||
vision_model_loader: n_tensors: 455
|
||||
vision_model_loader: n_kv: 20
|
||||
|
||||
vision_ctx: vision using Vulkan0 backend
|
||||
load_hparams: model_type: 0
|
||||
load_hparams: n_embd: 1152
|
||||
load_hparams: n_head: 16
|
||||
load_hparams: n_ff: 4304
|
||||
load_hparams: n_layer: 27
|
||||
load_hparams: ffn_op: gelu
|
||||
load_hparams: projection_dim: 0
|
||||
|
||||
--- vision hparams ---
|
||||
load_hparams: image_size: 448
|
||||
load_hparams: patch_size: 14
|
||||
load_hparams: minicpmv_version: 100045
|
||||
load_hparams: proj_scale_factor: 0
|
||||
load_hparams: n_wa_pattern: 0
|
||||
|
||||
load_hparams: model size: 1044.36 MiB
|
||||
load_hparams: metadata size: 0.16 MiB
|
||||
load_tensors: ffn up/down are swapped
|
||||
alloc_compute_meta: Vulkan0 compute buffer size = 100.30 MiB
|
||||
alloc_compute_meta: CPU compute buffer size = 18.30 MiB
|
||||
init tts....init t2w....voc_hg2_model: CPU backend using 8 threads
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=32000
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=200, n_mel=80, total_size=16000
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=200, n_mel=80, total_elements=16000
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=200, n_tokens=100, current_iter=0, current_total=0, new_total=100, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 1 (total_tokens=100, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 20]
|
||||
audition_audio_batch_encode: Final output: 20 tokens x 4096 dims
|
||||
=== Model Paths ===
|
||||
Base dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf
|
||||
LLM: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf [OK]
|
||||
Vision: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/vision/MiniCPM-o-4_5-vision-F16.gguf [OK]
|
||||
Audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf [OK]
|
||||
TTS: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf [OK]
|
||||
Projector: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf [OK]
|
||||
===================
|
||||
=== Initializing Omni Context ===
|
||||
Media type: 2 (omni: audio+vision)
|
||||
TTS enabled: yes
|
||||
Context size: 2048
|
||||
GPU layers: 99
|
||||
Token2Wav device: cpu
|
||||
Vision backend: metal
|
||||
TTS bin dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts
|
||||
Ref audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\vllm-omni-edge\benchmarks\edge_harness\results\e2e_expansion_20260923\evidence\minicpmo_cpp_radeon890m_prompted\fixture_0000.wav
|
||||
02:32:53.004 === omni_init start
|
||||
02:32:53.004 media_type = 2, duplex_mode = 0, base_output_dir = ./tools/omni/output
|
||||
02:32:53.004 === omni_init: loading new LLM model
|
||||
02:33:42.567 === omni_init: loading TTS model
|
||||
02:33:42.567 TTS model: loading with n_gpu_layers=0
|
||||
02:33:45.341 TTS sampler: temp=0.80, top_p=0.85, top_k=25, rep_penalty=1.05
|
||||
02:33:45.341 TTS: loading weights from GGUF (emb_code, emb_text, projector_semantic, head_code)...
|
||||
02:33:50.973 TTS: head_code shape: dim0=6562, dim1=768, will transpose to [6562, 768]
|
||||
02:33:50.979 TTS: head_code loaded, verifying first few values:
|
||||
02:33:50.979 head_code_weight[0] = 0.01361084 (expect ~0.01385498)
|
||||
02:33:50.979 head_code_weight[1] = -0.01660156 (expect ~-0.01647949)
|
||||
02:33:50.979 head_code_weight[768] = 0.04101562 (this is weight[1, 0], expect ~0.04150391)
|
||||
02:33:51.034 TTS: weights loaded successfully
|
||||
02:33:51.034 Projector: loading from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf
|
||||
02:33:51.146 Projector: loaded successfully
|
||||
02:33:51.147 === omni_init: loading APM model
|
||||
02:33:54.119 APM: init from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf
|
||||
02:33:59.353 Token2Wav: found models in \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf
|
||||
02:33:59.403 Token2Wav: all model files found, initializing session...
|
||||
02:33:59.403 Token2Wav: non-CUDA backend, vocoder using CPU for better performance
|
||||
02:33:59.403 Token2Wav: using prompt_cache from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf/prompt_cache.gguf
|
||||
02:34:03.892 Token2Wav: initialized successfully
|
||||
02:34:03.892 Python T2W: script_dir=./tools/omni/pyt2w, model_dir=./tools/omni/pyt2w/token2wav
|
||||
02:34:03.892 Token2Wav: 浣跨敤 C++ 瀹炵幇
|
||||
02:34:03.892 === omni_init success: ctx_llama = 0000018ebc7dfe80
|
||||
=== Running test case ===
|
||||
Audio prefix: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-4-_xol6uv7\fixture_
|
||||
Count: 2
|
||||
02:34:03.892 stream_prefill: n_past = 0
|
||||
voice_clone_prompt = <|im_start|>system
|
||||
妯′豢闊抽�鏍锋湰鐨勯煶鑹插苟鐢熸垚鏂扮殑鍐呭�銆�
|
||||
<|audio_start|>
|
||||
assistant_prompt = <|audio_end|>浣犵殑浠诲姟鏄�敤杩欑�澹伴煶妯″紡鏉ュ綋涓€涓�姪鎵嬨€傝�璁ょ湡銆侀珮璐ㄩ噺鍦板洖澶嶇敤鎴风殑闂��銆傝�鐢ㄩ珮鑷�劧搴︾殑鏂瑰紡鍜岀敤鎴疯亰澶┿€�<|im_end|>
|
||||
<|im_start|>user
|
||||
|
||||
02:34:03.892 system prompt ref_audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-4-_xol6uv7\fixture_0000.wav
|
||||
02:34:04.574 system prompt ref_audio embedding: n_pos=20
|
||||
02:34:05.084 馃敀 n_keep 璁剧疆涓� 72 (system prompt tokens)锛岃繖閮ㄥ垎姘歌繙涓嶄細琚�粦鍔ㄧ獥鍙ｅ垹闄�
|
||||
02:34:05.084 stream_prefill(index=0): system prompt 鍒濆�鍖栧畬鎴愶紝ref_audio 宸插湪鍏朵腑 prefill
|
||||
02:34:05.084 [SW] system_prompt registered: preserve_length=72 (will be protected from sliding)
|
||||
02:34:05.084 n_past = 72
|
||||
02:34:05.084
|
||||
|
||||
c++ finish stream_prefill(index=0). n_past=72, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 0 (audio) : 1.19259 s
|
||||
encode_image_with_vision_chunks: step 1 of 1 encoded in 566.22 ms
|
||||
encode_image_with_vision_chunks: all 1 chunks encoded in 583.82 ms (grid: 1x1)
|
||||
|
||||
encode_image_with_vision_chunks: image encoded in 583.83 ms by vision ( 583.83 ms per chunk, 64 total tokens)
|
||||
omni_image_embed_make_chunks_with_filename: created 1 vision chunks from C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-4-_xol6uv7\fixture_0001.jpg
|
||||
stream_prefill: prefilled 1 vision chunks (64 tokens each)
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=49339
|
||||
audition_audio_preprocess: Audio not aligned to 100ms (49339 samples = 3083.7ms), padded with silence to 49600 samples (3100ms)
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=310, n_mel=80, total_size=24800
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=310, n_mel=80, total_elements=24800
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=310, n_tokens=155, current_iter=1, current_total=155, new_total=310, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 2 (total_tokens=310, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 31]
|
||||
audition_audio_batch_encode: Final output: 31 tokens x 4096 dims
|
||||
02:34:06.196 stream_prefill(index=1): processing user audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-4-_xol6uv7\fixture_0001.wav
|
||||
02:34:06.507 stream_prefill(index=1): user audio embedding: n_pos=31
|
||||
02:34:06.821
|
||||
|
||||
c++ finish stream_prefill(index=1). n_past=172, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 1 (audio+vision) : 1.73603 s
|
||||
02:34:06.821 馃搷 stream_decode 寮€濮�: n_past=172, n_keep=72, n_ctx=2048, duplex_mode=0
|
||||
02:34:06.821 stream_decode: create tts thread (simplex mode)
|
||||
02:34:06.821 stream_decode: create t2w thread
|
||||
02:34:06.821 wait prefill done
|
||||
02:34:06.821 馃搷 [鍗曞伐TTS] 娣诲姞 assistant prompt: "<|im_end|>
|
||||
<|im_start|>assistant
|
||||
<think>
|
||||
|
||||
</think>
|
||||
|
||||
<|tts_bos|>", n_past=172
|
||||
02:34:06.822 T2W thread (C++) started
|
||||
<user>
|
||||
<assistant>02:34:06.826 TTS: 鍒涘缓鍗曞伐妯″紡杈撳嚭鐩�綍: ./tools/omni/output/round_000
|
||||
02:34:06.826 TTS thread started
|
||||
02:34:06.986 馃搷 [鍗曞伐TTS] assistant prompt 瀹屾垚, n_past=182
|
||||
02:34:06.986 LLM decode: max_tgt_len = 96, n_predict = 96, n_ctx = 2048
|
||||
02:34:07.524 LLM: detected end token, set llm_generation_done=true
|
||||
02:34:07.524 LLM->TTS: text='The square in the image is red.', n_tokens=8, hidden_size=32768, n_embd=4096, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:34:07.524 馃搷 Decode 缁撴潫: n_past=191, 鍓╀綑绌洪棿=1857, 鏃犻渶婊戠獥
|
||||
02:34:07.524 馃搷 杞�� 1 缁撴潫锛岃�褰曡竟鐣屼簬 n_past=191
|
||||
02:34:07.524 TTS<-LLM: chunk_idx=0, text='The square in the image is red.', n_tokens=8, hidden_size=32768, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:34:07.524 TTS: after queue pop - speek_done=0, llm_finish=1, llm_text.empty=0, token_ids.size=8
|
||||
02:34:07.524 TTS: before empty check - response.empty=0, response='The square in the image is red.', llm_finish=1
|
||||
Waiting for audio generation to complete...
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_0.wav does not exist or is empty
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_1.wav does not exist or is empty
|
||||
TTS: no valid WAV files to merge
|
||||
02:34:07.524 TTS: DEBUG before has_llm_data - token_ids.size=8, hidden_states.size=32768, n_embd=4096
|
||||
02:34:07.524 TTS: has_llm_data=1, token_ids=8, hidden_states=32768, n_embd=4096, llm_finish=1
|
||||
02:34:07.525 TTS: n_tokens_orig=8, n_tokens_filtered=8 (filtered 0 special tokens)
|
||||
02:34:07.555 TTS: saved merged embeddings (text) to ./tools/omni/output/round_000/llm_debug/chunk_0/merged_embeddings.txt
|
||||
02:34:07.555 TTS Simplex: generating audio tokens for chunk 0 (n_tokens=8, tts_n_embd=768)
|
||||
02:34:07.555 TTS Simplex: 鍦� prefill 鍓嶆坊鍔� audio_bos (chunk_idx=0, new_size=9)
|
||||
02:34:07.555 TTS Simplex: first chunk - cleared KV cache
|
||||
02:34:07.555 TTS Simplex: first chunk - cleared tts_all_generated_tokens and tts_token_buffer
|
||||
02:34:07.617 馃搷 涓轰笅涓€杞�噯澶�: eval <|im_end|>\n<|im_start|>user\n, n_past=196
|
||||
02:34:09.819 TTS Simplex: prefill completed, n_past_tts=9
|
||||
02:34:09.819 TTS Simplex: sampler created
|
||||
02:34:09.819 TTS simplex: is_audio_bos=true (first audio token)
|
||||
02:34:09.846 TTS Simplex Phase1: token 1/500: rel_id=3674
|
||||
02:34:09.857 TTS Simplex Phase1: token 2/500: rel_id=2031
|
||||
02:34:09.868 TTS Simplex Phase1: token 3/500: rel_id=1950
|
||||
02:34:09.879 TTS Simplex Phase1: token 4/500: rel_id=3891
|
||||
02:34:09.890 TTS Simplex Phase1: token 5/500: rel_id=1704
|
||||
02:34:10.091 TTS Simplex Phase1: EOS token at step 24
|
||||
02:34:10.091 TTS Simplex: is_final_text_chunk=true, will enter Phase 2 for text_eos_embed
|
||||
02:34:10.091 TTS Simplex Phase2: injecting text_eos_embed + audio_bos at n_past=32
|
||||
02:34:10.099 TTS Simplex Phase2: text_eos_embed injected, n_past=33
|
||||
02:34:10.107 TTS Simplex Phase2: audio_bos injected, n_past=34, starting final generation
|
||||
02:34:10.130 TTS Simplex Phase2: yield 25 tokens 鍒� T2W
|
||||
02:34:10.130 T2W绾跨▼(C++): 杞��鍒囨崲 (-1 -> 0)锛屾洿鏂拌緭鍑虹洰褰�
|
||||
02:34:10.130 T2W绾跨▼(C++): 鏂拌緭鍑虹洰褰� ./tools/omni/output/round_000/tts_wav
|
||||
02:34:10.407 TTS Simplex Phase2: EOS at step 27 (final end)
|
||||
02:34:10.407 TTS Simplex: chunk 缁撴潫锛宼ts_token_buffer 鍓╀綑 24 tokens (淇濈暀绛変笅涓€涓� chunk)
|
||||
02:34:10.407 TTS Simplex: is_final_text_chunk=true, flush 鍓╀綑 24 tokens 浠� tts_token_buffer
|
||||
02:34:10.407 TTS Simplex: generated 49 audio tokens for chunk 0
|
||||
02:34:10.407 TTS Simplex: chunk 0 first tokens: [3674 2031 1950 3891 1704 ], last tokens: [66 1464 4218 4218 4137 ]
|
||||
02:34:10.407 TTS Simplex: updated tts_n_past_accumulated=61, total_generated_tokens=49
|
||||
02:34:10.408 TTS: finished processing all chunks
|
||||
02:34:10.409 TTS: 鍗曞伐妯″紡杞��缁撴潫锛屼笅涓€杞� round_idx=1
|
||||
02:34:10.409 TTS: sent is_final=true to T2W queue (turn end)
|
||||
02:34:12.541 馃帀 棣栧搷鏃堕棿 (First Audio Response): 5719ms
|
||||
02:34:12.541 T2W绾跨▼: wav_0.wav | 0.84s audio | 2410.3ms inference | RTF=2.87 | t=5719ms
|
||||
02:34:14.862 T2W绾跨▼: wav_1.wav | 1.24s audio | 2320.2ms inference | RTF=1.87 | t=8040ms
|
||||
02:34:14.862 T2W绾跨▼: 鍐欏叆缁撴潫鏍囪� ./tools/omni/output/round_000/tts_wav/generation_done.flag (last_wav=1)
|
||||
02:34:14.862 T2W绾跨▼: 杞��缁撴潫鍚庢洿鏂拌緭鍑虹洰褰曚负 ./tools/omni/output/round_001/tts_wav
|
||||
02:34:14.912 T2W(C++) 绾跨▼: 鍋滄�
|
||||
[timing] call=0 tokens=28 final=0 token2mel=2069.921ms vocoder=340.286ms total=2410.226ms audio=20160
|
||||
[timing] call=1 tokens=27 final=1 token2mel=1902.984ms vocoder=417.183ms total=2320.190ms audio=29760
|
||||
Audio generation completed.
|
||||
llama_perf_context_print: load time = 24042.82 ms
|
||||
llama_perf_context_print: prompt eval time = 3093.20 ms / 182 tokens ( 17.00 ms per token, 58.84 tokens per second)
|
||||
llama_perf_context_print: eval time = 535.27 ms / 9 runs ( 59.47 ms per token, 16.81 tokens per second)
|
||||
llama_perf_context_print: total time = 31968.52 ms / 191 tokens
|
||||
llama_perf_context_print: graphs reused = 10
|
||||
Token2Wav (C++): session released
|
||||
02:34:14.912 omni_stop_threads: stop signals sent
|
||||
tts thread end
|
||||
t2w thread end
|
||||
+483
@@ -0,0 +1,483 @@
|
||||
build: 64 (5202b7b) with x86_64-w64-mingw32-gcc-posix (GCC) 13-posix for x86_64-w64-mingw32
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 30720 MiB free
|
||||
llama_model_loader: loaded meta data with 27 key-value pairs and 399 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = qwen3
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Llm
|
||||
llama_model_loader: - kv 3: general.size_label str = 8.2B
|
||||
llama_model_loader: - kv 4: qwen3.block_count u32 = 36
|
||||
llama_model_loader: - kv 5: qwen3.context_length u32 = 40960
|
||||
llama_model_loader: - kv 6: qwen3.embedding_length u32 = 4096
|
||||
llama_model_loader: - kv 7: qwen3.feed_forward_length u32 = 12288
|
||||
llama_model_loader: - kv 8: qwen3.attention.head_count u32 = 32
|
||||
llama_model_loader: - kv 9: qwen3.attention.head_count_kv u32 = 8
|
||||
llama_model_loader: - kv 10: qwen3.rope.freq_base f32 = 1000000.000000
|
||||
llama_model_loader: - kv 11: qwen3.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: qwen3.attention.key_length u32 = 128
|
||||
llama_model_loader: - kv 13: qwen3.attention.value_length u32 = 128
|
||||
llama_model_loader: - kv 14: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 15: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 16: tokenizer.ggml.tokens arr[str,151748] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 17: tokenizer.ggml.token_type arr[i32,151748] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 24: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 25: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 26: general.file_type u32 = 15
|
||||
llama_model_loader: - type f32: 145 tensors
|
||||
llama_model_loader: - type q4_K: 217 tensors
|
||||
llama_model_loader: - type q6_K: 37 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = Q4_K - Medium
|
||||
print_info: file size = 4.68 GiB (4.90 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = qwen3
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 40960
|
||||
print_info: n_embd = 4096
|
||||
print_info: n_layer = 36
|
||||
print_info: n_head = 32
|
||||
print_info: n_head_kv = 8
|
||||
print_info: n_rot = 128
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 128
|
||||
print_info: n_embd_head_v = 128
|
||||
print_info: n_gqa = 4
|
||||
print_info: n_embd_k_gqa = 1024
|
||||
print_info: n_embd_v_gqa = 1024
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 12288
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = -1
|
||||
print_info: rope type = 2
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 1000000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 40960
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = 8B
|
||||
print_info: model params = 8.19 B
|
||||
print_info: general.name = Llm
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 151748
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 36 repeating layers to GPU
|
||||
load_tensors: offloading output layer to GPU
|
||||
load_tensors: offloaded 37/37 layers to GPU
|
||||
load_tensors: Vulkan0 model buffer size = 4454.74 MiB
|
||||
load_tensors: CPU_Mapped model buffer size = 333.43 MiB
|
||||
.....................................................................................
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 1000000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (40960) -- the full capacity of the model will not be utilized
|
||||
llama_context: Vulkan_Host output buffer size = 0.58 MiB
|
||||
llama_kv_cache: Vulkan0 KV buffer size = 288.00 MiB
|
||||
llama_kv_cache: size = 288.00 MiB ( 2048 cells, 36 layers, 1/1 seqs), K (f16): 144.00 MiB, V (f16): 144.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 304.38 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 12.01 MiB
|
||||
llama_context: graph nodes = 1267
|
||||
llama_context: graph splits = 2
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 25673 MiB free
|
||||
llama_model_loader: loaded meta data with 30 key-value pairs and 193 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = llama
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Tts
|
||||
llama_model_loader: - kv 3: general.size_label str = 445M
|
||||
llama_model_loader: - kv 4: llama.block_count u32 = 20
|
||||
llama_model_loader: - kv 5: llama.context_length u32 = 4096
|
||||
llama_model_loader: - kv 6: llama.embedding_length u32 = 768
|
||||
llama_model_loader: - kv 7: llama.feed_forward_length u32 = 3072
|
||||
llama_model_loader: - kv 8: llama.attention.head_count u32 = 12
|
||||
llama_model_loader: - kv 9: llama.attention.head_count_kv u32 = 12
|
||||
llama_model_loader: - kv 10: llama.rope.freq_base f32 = 10000.000000
|
||||
llama_model_loader: - kv 11: llama.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: llama.attention.key_length u32 = 64
|
||||
llama_model_loader: - kv 13: llama.attention.value_length u32 = 64
|
||||
llama_model_loader: - kv 14: general.file_type u32 = 1
|
||||
llama_model_loader: - kv 15: llama.vocab_size u32 = 158626
|
||||
llama_model_loader: - kv 16: llama.rope.dimension_count u32 = 64
|
||||
llama_model_loader: - kv 17: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.tokens arr[str,158626] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.token_type arr[i32,158626] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 24: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 25: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 26: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 27: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 28: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 29: tokenizer.ggml.add_space_prefix bool = false
|
||||
llama_model_loader: - type f32: 50 tensors
|
||||
llama_model_loader: - type f16: 143 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = F16
|
||||
print_info: file size = 1.07 GiB (20.70 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = llama
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 4096
|
||||
print_info: n_embd = 768
|
||||
print_info: n_layer = 20
|
||||
print_info: n_head = 12
|
||||
print_info: n_head_kv = 12
|
||||
print_info: n_rot = 64
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 64
|
||||
print_info: n_embd_head_v = 64
|
||||
print_info: n_gqa = 1
|
||||
print_info: n_embd_k_gqa = 768
|
||||
print_info: n_embd_v_gqa = 768
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 3072
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = 0
|
||||
print_info: rope type = 0
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 10000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 4096
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = ?B
|
||||
print_info: model params = 444.94 M
|
||||
print_info: general.name = Tts
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 158626
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
load_tensors: skipping TTS tensor emb_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor emb_text.weight (size = 467140608 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.weight (size = 12582912 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.weight (size = 2359296 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.weight (size = 6291456 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.weight (size = 1179648 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor head_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 0 repeating layers to GPU
|
||||
load_tensors: offloaded 0/21 layers to GPU
|
||||
load_tensors: CPU_Mapped model buffer size = 592.48 MiB
|
||||
..............................................................
|
||||
llama_init_from_model: model default pooling_type is [0], but [-1] was specified
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 10000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (4096) -- the full capacity of the model will not be utilized
|
||||
llama_context: CPU output buffer size = 0.61 MiB
|
||||
llama_kv_cache: CPU KV buffer size = 120.00 MiB
|
||||
llama_kv_cache: size = 120.00 MiB ( 2048 cells, 20 layers, 1/1 seqs), K (f16): 60.00 MiB, V (f16): 60.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 30.75 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 7.01 MiB
|
||||
llama_context: graph nodes = 613
|
||||
llama_context: graph splits = 216 (with bs=512), 1 (with bs=1)
|
||||
vision_model_loader: model name:
|
||||
vision_model_loader: description: image encoder for MiniCPM-V
|
||||
vision_model_loader: GGUF version: 3
|
||||
vision_model_loader: alignment: 32
|
||||
vision_model_loader: n_tensors: 455
|
||||
vision_model_loader: n_kv: 20
|
||||
|
||||
vision_ctx: vision using Vulkan0 backend
|
||||
load_hparams: model_type: 0
|
||||
load_hparams: n_embd: 1152
|
||||
load_hparams: n_head: 16
|
||||
load_hparams: n_ff: 4304
|
||||
load_hparams: n_layer: 27
|
||||
load_hparams: ffn_op: gelu
|
||||
load_hparams: projection_dim: 0
|
||||
|
||||
--- vision hparams ---
|
||||
load_hparams: image_size: 448
|
||||
load_hparams: patch_size: 14
|
||||
load_hparams: minicpmv_version: 100045
|
||||
load_hparams: proj_scale_factor: 0
|
||||
load_hparams: n_wa_pattern: 0
|
||||
|
||||
load_hparams: model size: 1044.36 MiB
|
||||
load_hparams: metadata size: 0.16 MiB
|
||||
load_tensors: ffn up/down are swapped
|
||||
alloc_compute_meta: Vulkan0 compute buffer size = 100.30 MiB
|
||||
alloc_compute_meta: CPU compute buffer size = 18.30 MiB
|
||||
init tts....init t2w....voc_hg2_model: CPU backend using 8 threads
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=32000
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=200, n_mel=80, total_size=16000
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=200, n_mel=80, total_elements=16000
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=200, n_tokens=100, current_iter=0, current_total=0, new_total=100, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 1 (total_tokens=100, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 20]
|
||||
audition_audio_batch_encode: Final output: 20 tokens x 4096 dims
|
||||
=== Model Paths ===
|
||||
Base dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf
|
||||
LLM: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf [OK]
|
||||
Vision: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/vision/MiniCPM-o-4_5-vision-F16.gguf [OK]
|
||||
Audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf [OK]
|
||||
TTS: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf [OK]
|
||||
Projector: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf [OK]
|
||||
===================
|
||||
=== Initializing Omni Context ===
|
||||
Media type: 2 (omni: audio+vision)
|
||||
TTS enabled: yes
|
||||
Context size: 2048
|
||||
GPU layers: 99
|
||||
Token2Wav device: cpu
|
||||
Vision backend: metal
|
||||
TTS bin dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts
|
||||
Ref audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\vllm-omni-edge\benchmarks\edge_harness\results\e2e_expansion_20260923\evidence\minicpmo_cpp_radeon890m_prompted\fixture_0000.wav
|
||||
02:34:17.437 === omni_init start
|
||||
02:34:17.437 media_type = 2, duplex_mode = 0, base_output_dir = ./tools/omni/output
|
||||
02:34:17.437 === omni_init: loading new LLM model
|
||||
02:35:11.530 === omni_init: loading TTS model
|
||||
02:35:11.530 TTS model: loading with n_gpu_layers=0
|
||||
02:35:14.637 TTS sampler: temp=0.80, top_p=0.85, top_k=25, rep_penalty=1.05
|
||||
02:35:14.637 TTS: loading weights from GGUF (emb_code, emb_text, projector_semantic, head_code)...
|
||||
02:35:20.270 TTS: head_code shape: dim0=6562, dim1=768, will transpose to [6562, 768]
|
||||
02:35:20.276 TTS: head_code loaded, verifying first few values:
|
||||
02:35:20.276 head_code_weight[0] = 0.01361084 (expect ~0.01385498)
|
||||
02:35:20.276 head_code_weight[1] = -0.01660156 (expect ~-0.01647949)
|
||||
02:35:20.276 head_code_weight[768] = 0.04101562 (this is weight[1, 0], expect ~0.04150391)
|
||||
02:35:20.325 TTS: weights loaded successfully
|
||||
02:35:20.325 Projector: loading from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf
|
||||
02:35:20.424 Projector: loaded successfully
|
||||
02:35:20.425 === omni_init: loading APM model
|
||||
02:35:23.456 APM: init from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf
|
||||
02:35:28.810 Token2Wav: found models in \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf
|
||||
02:35:28.861 Token2Wav: all model files found, initializing session...
|
||||
02:35:28.861 Token2Wav: non-CUDA backend, vocoder using CPU for better performance
|
||||
02:35:28.861 Token2Wav: using prompt_cache from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf/prompt_cache.gguf
|
||||
02:35:33.372 Token2Wav: initialized successfully
|
||||
02:35:33.372 Python T2W: script_dir=./tools/omni/pyt2w, model_dir=./tools/omni/pyt2w/token2wav
|
||||
02:35:33.372 Token2Wav: 浣跨敤 C++ 瀹炵幇
|
||||
02:35:33.372 === omni_init success: ctx_llama = 000002753e28fe80
|
||||
=== Running test case ===
|
||||
Audio prefix: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-5-5nm6m829\fixture_
|
||||
Count: 2
|
||||
02:35:33.372 stream_prefill: n_past = 0
|
||||
voice_clone_prompt = <|im_start|>system
|
||||
妯′豢闊抽�鏍锋湰鐨勯煶鑹插苟鐢熸垚鏂扮殑鍐呭�銆�
|
||||
<|audio_start|>
|
||||
assistant_prompt = <|audio_end|>浣犵殑浠诲姟鏄�敤杩欑�澹伴煶妯″紡鏉ュ綋涓€涓�姪鎵嬨€傝�璁ょ湡銆侀珮璐ㄩ噺鍦板洖澶嶇敤鎴风殑闂��銆傝�鐢ㄩ珮鑷�劧搴︾殑鏂瑰紡鍜岀敤鎴疯亰澶┿€�<|im_end|>
|
||||
<|im_start|>user
|
||||
|
||||
02:35:33.372 system prompt ref_audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-5-5nm6m829\fixture_0000.wav
|
||||
02:35:34.093 system prompt ref_audio embedding: n_pos=20
|
||||
02:35:34.580 馃敀 n_keep 璁剧疆涓� 72 (system prompt tokens)锛岃繖閮ㄥ垎姘歌繙涓嶄細琚�粦鍔ㄧ獥鍙ｅ垹闄�
|
||||
02:35:34.580 stream_prefill(index=0): system prompt 鍒濆�鍖栧畬鎴愶紝ref_audio 宸插湪鍏朵腑 prefill
|
||||
02:35:34.581 [SW] system_prompt registered: preserve_length=72 (will be protected from sliding)
|
||||
02:35:34.581 n_past = 72
|
||||
02:35:34.581
|
||||
|
||||
c++ finish stream_prefill(index=0). n_past=72, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 0 (audio) : 1.20814 s
|
||||
encode_image_with_vision_chunks: step 1 of 1 encoded in 571.64 ms
|
||||
encode_image_with_vision_chunks: all 1 chunks encoded in 589.86 ms (grid: 1x1)
|
||||
|
||||
encode_image_with_vision_chunks: image encoded in 589.86 ms by vision ( 589.86 ms per chunk, 64 total tokens)
|
||||
omni_image_embed_make_chunks_with_filename: created 1 vision chunks from C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-5-5nm6m829\fixture_0001.jpg
|
||||
stream_prefill: prefilled 1 vision chunks (64 tokens each)
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=49339
|
||||
audition_audio_preprocess: Audio not aligned to 100ms (49339 samples = 3083.7ms), padded with silence to 49600 samples (3100ms)
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=310, n_mel=80, total_size=24800
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=310, n_mel=80, total_elements=24800
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=310, n_tokens=155, current_iter=1, current_total=155, new_total=310, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 2 (total_tokens=310, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 31]
|
||||
audition_audio_batch_encode: Final output: 31 tokens x 4096 dims
|
||||
02:35:35.686 stream_prefill(index=1): processing user audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-5-5nm6m829\fixture_0001.wav
|
||||
02:35:35.969 stream_prefill(index=1): user audio embedding: n_pos=31
|
||||
02:35:36.277
|
||||
|
||||
c++ finish stream_prefill(index=1). n_past=172, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 1 (audio+vision) : 1.69494 s
|
||||
02:35:36.277 馃搷 stream_decode 寮€濮�: n_past=172, n_keep=72, n_ctx=2048, duplex_mode=0
|
||||
02:35:36.277 stream_decode: create tts thread (simplex mode)
|
||||
02:35:36.277 stream_decode: create t2w thread
|
||||
02:35:36.277 wait prefill done
|
||||
02:35:36.277 馃搷 [鍗曞伐TTS] 娣诲姞 assistant prompt: "<|im_end|>
|
||||
<|im_start|>assistant
|
||||
<think>
|
||||
|
||||
</think>
|
||||
|
||||
<|tts_bos|>", n_past=172
|
||||
02:35:36.278 T2W thread (C++) started
|
||||
<user>
|
||||
<assistant>02:35:36.281 TTS: 鍒涘缓鍗曞伐妯″紡杈撳嚭鐩�綍: ./tools/omni/output/round_000
|
||||
02:35:36.281 TTS thread started
|
||||
02:35:36.437 馃搷 [鍗曞伐TTS] assistant prompt 瀹屾垚, n_past=182
|
||||
02:35:36.437 LLM decode: max_tgt_len = 96, n_predict = 96, n_ctx = 2048
|
||||
02:35:36.954 LLM: detected end token, set llm_generation_done=true
|
||||
02:35:36.955 LLM->TTS: text='The square in the image is red.', n_tokens=8, hidden_size=32768, n_embd=4096, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:35:36.955 馃搷 Decode 缁撴潫: n_past=191, 鍓╀綑绌洪棿=1857, 鏃犻渶婊戠獥
|
||||
02:35:36.955 馃搷 杞�� 1 缁撴潫锛岃�褰曡竟鐣屼簬 n_past=191
|
||||
02:35:36.955 TTS<-LLM: chunk_idx=0, text='The square in the image is red.', n_tokens=8, hidden_size=32768, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:35:36.955 TTS: after queue pop - speek_done=0, llm_finish=1, llm_text.empty=0, token_ids.size=8
|
||||
02:35:36.955 TTS: before empty check - response.empty=0, response='The square in the image is red.', llm_finish=1
|
||||
Waiting for audio generation to complete...
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_0.wav does not exist or is empty
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_1.wav does not exist or is empty
|
||||
TTS: no valid WAV files to merge
|
||||
02:35:36.955 TTS: DEBUG before has_llm_data - token_ids.size=8, hidden_states.size=32768, n_embd=4096
|
||||
02:35:36.955 TTS: has_llm_data=1, token_ids=8, hidden_states=32768, n_embd=4096, llm_finish=1
|
||||
02:35:36.956 TTS: n_tokens_orig=8, n_tokens_filtered=8 (filtered 0 special tokens)
|
||||
02:35:36.996 TTS: saved merged embeddings (text) to ./tools/omni/output/round_000/llm_debug/chunk_0/merged_embeddings.txt
|
||||
02:35:36.996 TTS Simplex: generating audio tokens for chunk 0 (n_tokens=8, tts_n_embd=768)
|
||||
02:35:36.996 TTS Simplex: 鍦� prefill 鍓嶆坊鍔� audio_bos (chunk_idx=0, new_size=9)
|
||||
02:35:36.996 TTS Simplex: first chunk - cleared KV cache
|
||||
02:35:36.996 TTS Simplex: first chunk - cleared tts_all_generated_tokens and tts_token_buffer
|
||||
02:35:37.051 馃搷 涓轰笅涓€杞�噯澶�: eval <|im_end|>\n<|im_start|>user\n, n_past=196
|
||||
02:35:38.873 TTS Simplex: prefill completed, n_past_tts=9
|
||||
02:35:38.873 TTS Simplex: sampler created
|
||||
02:35:38.873 TTS simplex: is_audio_bos=true (first audio token)
|
||||
02:35:38.900 TTS Simplex Phase1: token 1/500: rel_id=740
|
||||
02:35:38.911 TTS Simplex Phase1: token 2/500: rel_id=4137
|
||||
02:35:38.922 TTS Simplex Phase1: token 3/500: rel_id=4218
|
||||
02:35:38.934 TTS Simplex Phase1: token 4/500: rel_id=4137
|
||||
02:35:38.944 TTS Simplex Phase1: token 5/500: rel_id=1950
|
||||
02:35:39.163 TTS Simplex Phase1: yield 25 tokens 鍒� T2W (step 25, buffer=25)
|
||||
02:35:39.163 TTS Simplex Phase1: token 25/500: rel_id=4519
|
||||
02:35:39.163 T2W绾跨▼(C++): 杞��鍒囨崲 (-1 -> 0)锛屾洿鏂拌緭鍑虹洰褰�
|
||||
02:35:39.163 T2W绾跨▼(C++): 鏂拌緭鍑虹洰褰� ./tools/omni/output/round_000/tts_wav
|
||||
02:35:39.355 TTS Simplex Phase1: EOS token at step 43
|
||||
02:35:39.355 TTS Simplex: is_final_text_chunk=true, will enter Phase 2 for text_eos_embed
|
||||
02:35:39.355 TTS Simplex Phase2: injecting text_eos_embed + audio_bos at n_past=51
|
||||
02:35:39.363 TTS Simplex Phase2: text_eos_embed injected, n_past=52
|
||||
02:35:39.370 TTS Simplex Phase2: audio_bos injected, n_past=53, starting final generation
|
||||
02:35:39.458 TTS Simplex Phase2: yield 25 tokens 鍒� T2W
|
||||
02:35:39.568 TTS Simplex Phase2: EOS at step 18 (final end)
|
||||
02:35:39.568 TTS Simplex: chunk 缁撴潫锛宼ts_token_buffer 鍓╀綑 9 tokens (淇濈暀绛変笅涓€涓� chunk)
|
||||
02:35:39.568 TTS Simplex: is_final_text_chunk=true, flush 鍓╀綑 9 tokens 浠� tts_token_buffer
|
||||
02:35:39.568 TTS Simplex: generated 59 audio tokens for chunk 0
|
||||
02:35:39.568 TTS Simplex: chunk 0 first tokens: [740 4137 4218 4137 1950 ], last tokens: [2919 1461 4218 4218 3894 ]
|
||||
02:35:39.568 TTS Simplex: updated tts_n_past_accumulated=71, total_generated_tokens=59
|
||||
02:35:39.569 TTS: finished processing all chunks
|
||||
02:35:39.569 TTS: 鍗曞伐妯″紡杞��缁撴潫锛屼笅涓€杞� round_idx=1
|
||||
02:35:39.569 TTS: sent is_final=true to T2W queue (turn end)
|
||||
02:35:41.543 馃帀 棣栧搷鏃堕棿 (First Audio Response): 5266ms
|
||||
02:35:41.543 T2W绾跨▼: wav_0.wav | 0.84s audio | 2378.7ms inference | RTF=2.83 | t=5266ms
|
||||
02:35:43.790 T2W绾跨▼: wav_1.wav | 1.00s audio | 2245.6ms inference | RTF=2.25 | t=7512ms
|
||||
02:35:45.902 T2W绾跨▼: wav_2.wav | 0.64s audio | 2111.8ms inference | RTF=3.30 | t=9624ms
|
||||
02:35:45.902 T2W绾跨▼: 鍐欏叆缁撴潫鏍囪� ./tools/omni/output/round_000/tts_wav/generation_done.flag (last_wav=2)
|
||||
02:35:45.902 T2W绾跨▼: 杞��缁撴潫鍚庢洿鏂拌緭鍑虹洰褰曚负 ./tools/omni/output/round_001/tts_wav
|
||||
02:35:45.903 omni_stop_threads: stop signals sent
|
||||
02:35:45.903 T2W(C++) 绾跨▼: 鍋滄�
|
||||
[timing] call=0 tokens=28 final=0 token2mel=2039.361ms vocoder=339.277ms total=2378.651ms audio=20160
|
||||
[timing] call=1 tokens=28 final=0 token2mel=1864.287ms vocoder=381.301ms total=2245.599ms audio=24000
|
||||
[timing] call=2 tokens=12 final=1 token2mel=1886.133ms vocoder=225.612ms total=2111.757ms audio=15360
|
||||
Audio generation completed.
|
||||
llama_perf_context_print: load time = 24779.65 ms
|
||||
llama_perf_context_print: prompt eval time = 3063.44 ms / 182 tokens ( 16.83 ms per token, 59.41 tokens per second)
|
||||
llama_perf_context_print: eval time = 515.09 ms / 9 runs ( 57.23 ms per token, 17.47 tokens per second)
|
||||
llama_perf_context_print: total time = 34245.34 ms / 191 tokens
|
||||
llama_perf_context_print: graphs reused = 10
|
||||
Token2Wav (C++): session released
|
||||
tts thread end
|
||||
t2w thread end
|
||||
+483
@@ -0,0 +1,483 @@
|
||||
build: 64 (5202b7b) with x86_64-w64-mingw32-gcc-posix (GCC) 13-posix for x86_64-w64-mingw32
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 30720 MiB free
|
||||
llama_model_loader: loaded meta data with 27 key-value pairs and 399 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = qwen3
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Llm
|
||||
llama_model_loader: - kv 3: general.size_label str = 8.2B
|
||||
llama_model_loader: - kv 4: qwen3.block_count u32 = 36
|
||||
llama_model_loader: - kv 5: qwen3.context_length u32 = 40960
|
||||
llama_model_loader: - kv 6: qwen3.embedding_length u32 = 4096
|
||||
llama_model_loader: - kv 7: qwen3.feed_forward_length u32 = 12288
|
||||
llama_model_loader: - kv 8: qwen3.attention.head_count u32 = 32
|
||||
llama_model_loader: - kv 9: qwen3.attention.head_count_kv u32 = 8
|
||||
llama_model_loader: - kv 10: qwen3.rope.freq_base f32 = 1000000.000000
|
||||
llama_model_loader: - kv 11: qwen3.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: qwen3.attention.key_length u32 = 128
|
||||
llama_model_loader: - kv 13: qwen3.attention.value_length u32 = 128
|
||||
llama_model_loader: - kv 14: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 15: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 16: tokenizer.ggml.tokens arr[str,151748] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 17: tokenizer.ggml.token_type arr[i32,151748] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 24: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 25: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 26: general.file_type u32 = 15
|
||||
llama_model_loader: - type f32: 145 tensors
|
||||
llama_model_loader: - type q4_K: 217 tensors
|
||||
llama_model_loader: - type q6_K: 37 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = Q4_K - Medium
|
||||
print_info: file size = 4.68 GiB (4.90 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = qwen3
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 40960
|
||||
print_info: n_embd = 4096
|
||||
print_info: n_layer = 36
|
||||
print_info: n_head = 32
|
||||
print_info: n_head_kv = 8
|
||||
print_info: n_rot = 128
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 128
|
||||
print_info: n_embd_head_v = 128
|
||||
print_info: n_gqa = 4
|
||||
print_info: n_embd_k_gqa = 1024
|
||||
print_info: n_embd_v_gqa = 1024
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 12288
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = -1
|
||||
print_info: rope type = 2
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 1000000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 40960
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = 8B
|
||||
print_info: model params = 8.19 B
|
||||
print_info: general.name = Llm
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 151748
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 36 repeating layers to GPU
|
||||
load_tensors: offloading output layer to GPU
|
||||
load_tensors: offloaded 37/37 layers to GPU
|
||||
load_tensors: Vulkan0 model buffer size = 4454.74 MiB
|
||||
load_tensors: CPU_Mapped model buffer size = 333.43 MiB
|
||||
.....................................................................................
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 1000000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (40960) -- the full capacity of the model will not be utilized
|
||||
llama_context: Vulkan_Host output buffer size = 0.58 MiB
|
||||
llama_kv_cache: Vulkan0 KV buffer size = 288.00 MiB
|
||||
llama_kv_cache: size = 288.00 MiB ( 2048 cells, 36 layers, 1/1 seqs), K (f16): 144.00 MiB, V (f16): 144.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 304.38 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 12.01 MiB
|
||||
llama_context: graph nodes = 1267
|
||||
llama_context: graph splits = 2
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 25673 MiB free
|
||||
llama_model_loader: loaded meta data with 30 key-value pairs and 193 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = llama
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Tts
|
||||
llama_model_loader: - kv 3: general.size_label str = 445M
|
||||
llama_model_loader: - kv 4: llama.block_count u32 = 20
|
||||
llama_model_loader: - kv 5: llama.context_length u32 = 4096
|
||||
llama_model_loader: - kv 6: llama.embedding_length u32 = 768
|
||||
llama_model_loader: - kv 7: llama.feed_forward_length u32 = 3072
|
||||
llama_model_loader: - kv 8: llama.attention.head_count u32 = 12
|
||||
llama_model_loader: - kv 9: llama.attention.head_count_kv u32 = 12
|
||||
llama_model_loader: - kv 10: llama.rope.freq_base f32 = 10000.000000
|
||||
llama_model_loader: - kv 11: llama.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: llama.attention.key_length u32 = 64
|
||||
llama_model_loader: - kv 13: llama.attention.value_length u32 = 64
|
||||
llama_model_loader: - kv 14: general.file_type u32 = 1
|
||||
llama_model_loader: - kv 15: llama.vocab_size u32 = 158626
|
||||
llama_model_loader: - kv 16: llama.rope.dimension_count u32 = 64
|
||||
llama_model_loader: - kv 17: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.tokens arr[str,158626] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.token_type arr[i32,158626] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 24: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 25: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 26: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 27: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 28: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 29: tokenizer.ggml.add_space_prefix bool = false
|
||||
llama_model_loader: - type f32: 50 tensors
|
||||
llama_model_loader: - type f16: 143 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = F16
|
||||
print_info: file size = 1.07 GiB (20.70 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = llama
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 4096
|
||||
print_info: n_embd = 768
|
||||
print_info: n_layer = 20
|
||||
print_info: n_head = 12
|
||||
print_info: n_head_kv = 12
|
||||
print_info: n_rot = 64
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 64
|
||||
print_info: n_embd_head_v = 64
|
||||
print_info: n_gqa = 1
|
||||
print_info: n_embd_k_gqa = 768
|
||||
print_info: n_embd_v_gqa = 768
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 3072
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = 0
|
||||
print_info: rope type = 0
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 10000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 4096
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = ?B
|
||||
print_info: model params = 444.94 M
|
||||
print_info: general.name = Tts
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 158626
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
load_tensors: skipping TTS tensor emb_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor emb_text.weight (size = 467140608 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.weight (size = 12582912 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.weight (size = 2359296 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.weight (size = 6291456 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.weight (size = 1179648 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor head_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 0 repeating layers to GPU
|
||||
load_tensors: offloaded 0/21 layers to GPU
|
||||
load_tensors: CPU_Mapped model buffer size = 592.48 MiB
|
||||
..............................................................
|
||||
llama_init_from_model: model default pooling_type is [0], but [-1] was specified
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 10000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (4096) -- the full capacity of the model will not be utilized
|
||||
llama_context: CPU output buffer size = 0.61 MiB
|
||||
llama_kv_cache: CPU KV buffer size = 120.00 MiB
|
||||
llama_kv_cache: size = 120.00 MiB ( 2048 cells, 20 layers, 1/1 seqs), K (f16): 60.00 MiB, V (f16): 60.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 30.75 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 7.01 MiB
|
||||
llama_context: graph nodes = 613
|
||||
llama_context: graph splits = 216 (with bs=512), 1 (with bs=1)
|
||||
vision_model_loader: model name:
|
||||
vision_model_loader: description: image encoder for MiniCPM-V
|
||||
vision_model_loader: GGUF version: 3
|
||||
vision_model_loader: alignment: 32
|
||||
vision_model_loader: n_tensors: 455
|
||||
vision_model_loader: n_kv: 20
|
||||
|
||||
vision_ctx: vision using Vulkan0 backend
|
||||
load_hparams: model_type: 0
|
||||
load_hparams: n_embd: 1152
|
||||
load_hparams: n_head: 16
|
||||
load_hparams: n_ff: 4304
|
||||
load_hparams: n_layer: 27
|
||||
load_hparams: ffn_op: gelu
|
||||
load_hparams: projection_dim: 0
|
||||
|
||||
--- vision hparams ---
|
||||
load_hparams: image_size: 448
|
||||
load_hparams: patch_size: 14
|
||||
load_hparams: minicpmv_version: 100045
|
||||
load_hparams: proj_scale_factor: 0
|
||||
load_hparams: n_wa_pattern: 0
|
||||
|
||||
load_hparams: model size: 1044.36 MiB
|
||||
load_hparams: metadata size: 0.16 MiB
|
||||
load_tensors: ffn up/down are swapped
|
||||
alloc_compute_meta: Vulkan0 compute buffer size = 100.30 MiB
|
||||
alloc_compute_meta: CPU compute buffer size = 18.30 MiB
|
||||
init tts....init t2w....voc_hg2_model: CPU backend using 8 threads
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=32000
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=200, n_mel=80, total_size=16000
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=200, n_mel=80, total_elements=16000
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=200, n_tokens=100, current_iter=0, current_total=0, new_total=100, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 1 (total_tokens=100, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 20]
|
||||
audition_audio_batch_encode: Final output: 20 tokens x 4096 dims
|
||||
=== Model Paths ===
|
||||
Base dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf
|
||||
LLM: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf [OK]
|
||||
Vision: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/vision/MiniCPM-o-4_5-vision-F16.gguf [OK]
|
||||
Audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf [OK]
|
||||
TTS: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf [OK]
|
||||
Projector: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf [OK]
|
||||
===================
|
||||
=== Initializing Omni Context ===
|
||||
Media type: 2 (omni: audio+vision)
|
||||
TTS enabled: yes
|
||||
Context size: 2048
|
||||
GPU layers: 99
|
||||
Token2Wav device: cpu
|
||||
Vision backend: metal
|
||||
TTS bin dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts
|
||||
Ref audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\vllm-omni-edge\benchmarks\edge_harness\results\e2e_expansion_20260923\evidence\minicpmo_cpp_radeon890m_prompted\fixture_0000.wav
|
||||
02:35:48.961 === omni_init start
|
||||
02:35:48.962 media_type = 2, duplex_mode = 0, base_output_dir = ./tools/omni/output
|
||||
02:35:48.962 === omni_init: loading new LLM model
|
||||
02:36:48.879 === omni_init: loading TTS model
|
||||
02:36:48.879 TTS model: loading with n_gpu_layers=0
|
||||
02:36:52.539 TTS sampler: temp=0.80, top_p=0.85, top_k=25, rep_penalty=1.05
|
||||
02:36:52.539 TTS: loading weights from GGUF (emb_code, emb_text, projector_semantic, head_code)...
|
||||
02:36:58.443 TTS: head_code shape: dim0=6562, dim1=768, will transpose to [6562, 768]
|
||||
02:36:58.448 TTS: head_code loaded, verifying first few values:
|
||||
02:36:58.448 head_code_weight[0] = 0.01361084 (expect ~0.01385498)
|
||||
02:36:58.448 head_code_weight[1] = -0.01660156 (expect ~-0.01647949)
|
||||
02:36:58.448 head_code_weight[768] = 0.04101562 (this is weight[1, 0], expect ~0.04150391)
|
||||
02:36:58.494 TTS: weights loaded successfully
|
||||
02:36:58.494 Projector: loading from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf
|
||||
02:36:58.606 Projector: loaded successfully
|
||||
02:36:58.606 === omni_init: loading APM model
|
||||
02:37:02.258 APM: init from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf
|
||||
02:37:08.457 Token2Wav: found models in \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf
|
||||
02:37:08.511 Token2Wav: all model files found, initializing session...
|
||||
02:37:08.511 Token2Wav: non-CUDA backend, vocoder using CPU for better performance
|
||||
02:37:08.511 Token2Wav: using prompt_cache from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf/prompt_cache.gguf
|
||||
02:37:13.567 Token2Wav: initialized successfully
|
||||
02:37:13.567 Python T2W: script_dir=./tools/omni/pyt2w, model_dir=./tools/omni/pyt2w/token2wav
|
||||
02:37:13.567 Token2Wav: 浣跨敤 C++ 瀹炵幇
|
||||
02:37:13.567 === omni_init success: ctx_llama = 0000024aa791fe80
|
||||
=== Running test case ===
|
||||
Audio prefix: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-6-97uvprio\fixture_
|
||||
Count: 2
|
||||
02:37:13.567 stream_prefill: n_past = 0
|
||||
voice_clone_prompt = <|im_start|>system
|
||||
妯′豢闊抽�鏍锋湰鐨勯煶鑹插苟鐢熸垚鏂扮殑鍐呭�銆�
|
||||
<|audio_start|>
|
||||
assistant_prompt = <|audio_end|>浣犵殑浠诲姟鏄�敤杩欑�澹伴煶妯″紡鏉ュ綋涓€涓�姪鎵嬨€傝�璁ょ湡銆侀珮璐ㄩ噺鍦板洖澶嶇敤鎴风殑闂��銆傝�鐢ㄩ珮鑷�劧搴︾殑鏂瑰紡鍜岀敤鎴疯亰澶┿€�<|im_end|>
|
||||
<|im_start|>user
|
||||
|
||||
02:37:13.567 system prompt ref_audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-6-97uvprio\fixture_0000.wav
|
||||
02:37:14.378 system prompt ref_audio embedding: n_pos=20
|
||||
02:37:14.918 馃敀 n_keep 璁剧疆涓� 72 (system prompt tokens)锛岃繖閮ㄥ垎姘歌繙涓嶄細琚�粦鍔ㄧ獥鍙ｅ垹闄�
|
||||
02:37:14.918 stream_prefill(index=0): system prompt 鍒濆�鍖栧畬鎴愶紝ref_audio 宸插湪鍏朵腑 prefill
|
||||
02:37:14.918 [SW] system_prompt registered: preserve_length=72 (will be protected from sliding)
|
||||
02:37:14.918 n_past = 72
|
||||
02:37:14.918
|
||||
|
||||
c++ finish stream_prefill(index=0). n_past=72, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 0 (audio) : 1.35118 s
|
||||
encode_image_with_vision_chunks: step 1 of 1 encoded in 584.36 ms
|
||||
encode_image_with_vision_chunks: all 1 chunks encoded in 602.34 ms (grid: 1x1)
|
||||
|
||||
encode_image_with_vision_chunks: image encoded in 602.34 ms by vision ( 602.34 ms per chunk, 64 total tokens)
|
||||
omni_image_embed_make_chunks_with_filename: created 1 vision chunks from C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-6-97uvprio\fixture_0001.jpg
|
||||
stream_prefill: prefilled 1 vision chunks (64 tokens each)
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=49339
|
||||
audition_audio_preprocess: Audio not aligned to 100ms (49339 samples = 3083.7ms), padded with silence to 49600 samples (3100ms)
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=310, n_mel=80, total_size=24800
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=310, n_mel=80, total_elements=24800
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=310, n_tokens=155, current_iter=1, current_total=155, new_total=310, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 2 (total_tokens=310, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 31]
|
||||
audition_audio_batch_encode: Final output: 31 tokens x 4096 dims
|
||||
02:37:16.065 stream_prefill(index=1): processing user audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-6-97uvprio\fixture_0001.wav
|
||||
02:37:16.362 stream_prefill(index=1): user audio embedding: n_pos=31
|
||||
02:37:16.688
|
||||
|
||||
c++ finish stream_prefill(index=1). n_past=172, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 1 (audio+vision) : 1.76831 s
|
||||
02:37:16.688 馃搷 stream_decode 寮€濮�: n_past=172, n_keep=72, n_ctx=2048, duplex_mode=0
|
||||
02:37:16.688 stream_decode: create tts thread (simplex mode)
|
||||
02:37:16.688 stream_decode: create t2w thread
|
||||
02:37:16.688 wait prefill done
|
||||
02:37:16.688 馃搷 [鍗曞伐TTS] 娣诲姞 assistant prompt: "<|im_end|>
|
||||
<|im_start|>assistant
|
||||
<think>
|
||||
|
||||
</think>
|
||||
|
||||
<|tts_bos|>", n_past=172
|
||||
02:37:16.689 T2W thread (C++) started
|
||||
<user>
|
||||
<assistant>02:37:16.692 TTS: 鍒涘缓鍗曞伐妯″紡杈撳嚭鐩�綍: ./tools/omni/output/round_000
|
||||
02:37:16.693 TTS thread started
|
||||
02:37:16.868 馃搷 [鍗曞伐TTS] assistant prompt 瀹屾垚, n_past=182
|
||||
02:37:16.868 LLM decode: max_tgt_len = 96, n_predict = 96, n_ctx = 2048
|
||||
02:37:17.385 LLM: detected end token, set llm_generation_done=true
|
||||
02:37:17.385 LLM->TTS: text='The square in the image is red.', n_tokens=8, hidden_size=32768, n_embd=4096, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:37:17.385 馃搷 Decode 缁撴潫: n_past=191, 鍓╀綑绌洪棿=1857, 鏃犻渶婊戠獥
|
||||
02:37:17.385 馃搷 杞�� 1 缁撴潫锛岃�褰曡竟鐣屼簬 n_past=191
|
||||
02:37:17.385 TTS<-LLM: chunk_idx=0, text='The square in the image is red.', n_tokens=8, hidden_size=32768, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:37:17.385 TTS: after queue pop - speek_done=0, llm_finish=1, llm_text.empty=0, token_ids.size=8
|
||||
02:37:17.385 TTS: before empty check - response.empty=0, response='The square in the image is red.', llm_finish=1
|
||||
Waiting for audio generation to complete...
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_0.wav does not exist or is empty
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_1.wav does not exist or is empty
|
||||
TTS: no valid WAV files to merge
|
||||
02:37:17.385 TTS: DEBUG before has_llm_data - token_ids.size=8, hidden_states.size=32768, n_embd=4096
|
||||
02:37:17.385 TTS: has_llm_data=1, token_ids=8, hidden_states=32768, n_embd=4096, llm_finish=1
|
||||
02:37:17.386 TTS: n_tokens_orig=8, n_tokens_filtered=8 (filtered 0 special tokens)
|
||||
02:37:17.415 TTS: saved merged embeddings (text) to ./tools/omni/output/round_000/llm_debug/chunk_0/merged_embeddings.txt
|
||||
02:37:17.415 TTS Simplex: generating audio tokens for chunk 0 (n_tokens=8, tts_n_embd=768)
|
||||
02:37:17.415 TTS Simplex: 鍦� prefill 鍓嶆坊鍔� audio_bos (chunk_idx=0, new_size=9)
|
||||
02:37:17.415 TTS Simplex: first chunk - cleared KV cache
|
||||
02:37:17.415 TTS Simplex: first chunk - cleared tts_all_generated_tokens and tts_token_buffer
|
||||
02:37:17.478 馃搷 涓轰笅涓€杞�噯澶�: eval <|im_end|>\n<|im_start|>user\n, n_past=196
|
||||
02:37:17.699 TTS Simplex: prefill completed, n_past_tts=9
|
||||
02:37:17.699 TTS Simplex: sampler created
|
||||
02:37:17.699 TTS simplex: is_audio_bos=true (first audio token)
|
||||
02:37:17.726 TTS Simplex Phase1: token 1/500: rel_id=1487
|
||||
02:37:17.737 TTS Simplex Phase1: token 2/500: rel_id=4137
|
||||
02:37:17.747 TTS Simplex Phase1: token 3/500: rel_id=6405
|
||||
02:37:17.758 TTS Simplex Phase1: token 4/500: rel_id=4218
|
||||
02:37:17.769 TTS Simplex Phase1: token 5/500: rel_id=4218
|
||||
02:37:17.989 TTS Simplex Phase1: yield 25 tokens 鍒� T2W (step 25, buffer=25)
|
||||
02:37:17.989 TTS Simplex Phase1: token 25/500: rel_id=5853
|
||||
02:37:17.989 T2W绾跨▼(C++): 杞��鍒囨崲 (-1 -> 0)锛屾洿鏂拌緭鍑虹洰褰�
|
||||
02:37:17.989 T2W绾跨▼(C++): 鏂拌緭鍑虹洰褰� ./tools/omni/output/round_000/tts_wav
|
||||
02:37:18.201 TTS Simplex Phase1: EOS token at step 45
|
||||
02:37:18.201 TTS Simplex: is_final_text_chunk=true, will enter Phase 2 for text_eos_embed
|
||||
02:37:18.201 TTS Simplex Phase2: injecting text_eos_embed + audio_bos at n_past=53
|
||||
02:37:18.209 TTS Simplex Phase2: text_eos_embed injected, n_past=54
|
||||
02:37:18.217 TTS Simplex Phase2: audio_bos injected, n_past=55, starting final generation
|
||||
02:37:18.281 TTS Simplex Phase2: yield 25 tokens 鍒� T2W
|
||||
02:37:18.476 TTS Simplex Phase2: EOS at step 24 (final end)
|
||||
02:37:18.476 TTS Simplex: chunk 缁撴潫锛宼ts_token_buffer 鍓╀綑 17 tokens (淇濈暀绛変笅涓€涓� chunk)
|
||||
02:37:18.476 TTS Simplex: is_final_text_chunk=true, flush 鍓╀綑 17 tokens 浠� tts_token_buffer
|
||||
02:37:18.476 TTS Simplex: generated 67 audio tokens for chunk 0
|
||||
02:37:18.476 TTS Simplex: chunk 0 first tokens: [1487 4137 6405 4218 4218 ], last tokens: [4218 4218 6405 2031 6088 ]
|
||||
02:37:18.476 TTS Simplex: updated tts_n_past_accumulated=79, total_generated_tokens=67
|
||||
02:37:18.477 TTS: finished processing all chunks
|
||||
02:37:18.477 TTS: 鍗曞伐妯″紡杞��缁撴潫锛屼笅涓€杞� round_idx=1
|
||||
02:37:18.477 TTS: sent is_final=true to T2W queue (turn end)
|
||||
02:37:20.345 馃帀 棣栧搷鏃堕棿 (First Audio Response): 3657ms
|
||||
02:37:20.345 T2W绾跨▼: wav_0.wav | 0.84s audio | 2355.0ms inference | RTF=2.80 | t=3657ms
|
||||
02:37:22.572 T2W绾跨▼: wav_1.wav | 1.00s audio | 2226.3ms inference | RTF=2.23 | t=5884ms
|
||||
02:37:24.760 T2W绾跨▼: wav_2.wav | 0.96s audio | 2187.1ms inference | RTF=2.28 | t=8071ms
|
||||
02:37:24.761 T2W绾跨▼: 鍐欏叆缁撴潫鏍囪� ./tools/omni/output/round_000/tts_wav/generation_done.flag (last_wav=2)
|
||||
02:37:24.761 T2W绾跨▼: 杞��缁撴潫鍚庢洿鏂拌緭鍑虹洰褰曚负 ./tools/omni/output/round_001/tts_wav
|
||||
02:37:02:24.800 omni_stop_threads: stop signals sent
|
||||
37:24.800 T2W(C++) 绾跨▼: 鍋滄�
|
||||
[timing] call=0 tokens=28 final=0 token2mel=2022.706ms vocoder=332.287ms total=2355.008ms audio=20160
|
||||
[timing] call=1 tokens=28 final=0 token2mel=1848.701ms vocoder=377.532ms total=2226.244ms audio=24000
|
||||
[timing] call=2 tokens=20 final=1 token2mel=1873.463ms vocoder=313.589ms total=2187.061ms audio=23040
|
||||
Audio generation completed.
|
||||
llama_perf_context_print: load time = 27691.90 ms
|
||||
llama_perf_context_print: prompt eval time = 3299.37 ms / 182 tokens ( 18.13 ms per token, 55.16 tokens per second)
|
||||
llama_perf_context_print: eval time = 514.87 ms / 9 runs ( 57.21 ms per token, 17.48 tokens per second)
|
||||
llama_perf_context_print: total time = 35624.36 ms / 191 tokens
|
||||
llama_perf_context_print: graphs reused = 10
|
||||
Token2Wav (C++): session released
|
||||
tts thread end
|
||||
t2w thread end
|
||||
+484
@@ -0,0 +1,484 @@
|
||||
build: 64 (5202b7b) with x86_64-w64-mingw32-gcc-posix (GCC) 13-posix for x86_64-w64-mingw32
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 30720 MiB free
|
||||
llama_model_loader: loaded meta data with 27 key-value pairs and 399 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = qwen3
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Llm
|
||||
llama_model_loader: - kv 3: general.size_label str = 8.2B
|
||||
llama_model_loader: - kv 4: qwen3.block_count u32 = 36
|
||||
llama_model_loader: - kv 5: qwen3.context_length u32 = 40960
|
||||
llama_model_loader: - kv 6: qwen3.embedding_length u32 = 4096
|
||||
llama_model_loader: - kv 7: qwen3.feed_forward_length u32 = 12288
|
||||
llama_model_loader: - kv 8: qwen3.attention.head_count u32 = 32
|
||||
llama_model_loader: - kv 9: qwen3.attention.head_count_kv u32 = 8
|
||||
llama_model_loader: - kv 10: qwen3.rope.freq_base f32 = 1000000.000000
|
||||
llama_model_loader: - kv 11: qwen3.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: qwen3.attention.key_length u32 = 128
|
||||
llama_model_loader: - kv 13: qwen3.attention.value_length u32 = 128
|
||||
llama_model_loader: - kv 14: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 15: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 16: tokenizer.ggml.tokens arr[str,151748] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 17: tokenizer.ggml.token_type arr[i32,151748] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 24: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 25: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 26: general.file_type u32 = 15
|
||||
llama_model_loader: - type f32: 145 tensors
|
||||
llama_model_loader: - type q4_K: 217 tensors
|
||||
llama_model_loader: - type q6_K: 37 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = Q4_K - Medium
|
||||
print_info: file size = 4.68 GiB (4.90 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = qwen3
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 40960
|
||||
print_info: n_embd = 4096
|
||||
print_info: n_layer = 36
|
||||
print_info: n_head = 32
|
||||
print_info: n_head_kv = 8
|
||||
print_info: n_rot = 128
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 128
|
||||
print_info: n_embd_head_v = 128
|
||||
print_info: n_gqa = 4
|
||||
print_info: n_embd_k_gqa = 1024
|
||||
print_info: n_embd_v_gqa = 1024
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 12288
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = -1
|
||||
print_info: rope type = 2
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 1000000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 40960
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = 8B
|
||||
print_info: model params = 8.19 B
|
||||
print_info: general.name = Llm
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 151748
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 36 repeating layers to GPU
|
||||
load_tensors: offloading output layer to GPU
|
||||
load_tensors: offloaded 37/37 layers to GPU
|
||||
load_tensors: Vulkan0 model buffer size = 4454.74 MiB
|
||||
load_tensors: CPU_Mapped model buffer size = 333.43 MiB
|
||||
.....................................................................................
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 1000000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (40960) -- the full capacity of the model will not be utilized
|
||||
llama_context: Vulkan_Host output buffer size = 0.58 MiB
|
||||
llama_kv_cache: Vulkan0 KV buffer size = 288.00 MiB
|
||||
llama_kv_cache: size = 288.00 MiB ( 2048 cells, 36 layers, 1/1 seqs), K (f16): 144.00 MiB, V (f16): 144.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 304.38 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 12.01 MiB
|
||||
llama_context: graph nodes = 1267
|
||||
llama_context: graph splits = 2
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 25673 MiB free
|
||||
llama_model_loader: loaded meta data with 30 key-value pairs and 193 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = llama
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Tts
|
||||
llama_model_loader: - kv 3: general.size_label str = 445M
|
||||
llama_model_loader: - kv 4: llama.block_count u32 = 20
|
||||
llama_model_loader: - kv 5: llama.context_length u32 = 4096
|
||||
llama_model_loader: - kv 6: llama.embedding_length u32 = 768
|
||||
llama_model_loader: - kv 7: llama.feed_forward_length u32 = 3072
|
||||
llama_model_loader: - kv 8: llama.attention.head_count u32 = 12
|
||||
llama_model_loader: - kv 9: llama.attention.head_count_kv u32 = 12
|
||||
llama_model_loader: - kv 10: llama.rope.freq_base f32 = 10000.000000
|
||||
llama_model_loader: - kv 11: llama.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: llama.attention.key_length u32 = 64
|
||||
llama_model_loader: - kv 13: llama.attention.value_length u32 = 64
|
||||
llama_model_loader: - kv 14: general.file_type u32 = 1
|
||||
llama_model_loader: - kv 15: llama.vocab_size u32 = 158626
|
||||
llama_model_loader: - kv 16: llama.rope.dimension_count u32 = 64
|
||||
llama_model_loader: - kv 17: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.tokens arr[str,158626] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.token_type arr[i32,158626] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 24: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 25: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 26: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 27: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 28: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 29: tokenizer.ggml.add_space_prefix bool = false
|
||||
llama_model_loader: - type f32: 50 tensors
|
||||
llama_model_loader: - type f16: 143 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = F16
|
||||
print_info: file size = 1.07 GiB (20.70 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = llama
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 4096
|
||||
print_info: n_embd = 768
|
||||
print_info: n_layer = 20
|
||||
print_info: n_head = 12
|
||||
print_info: n_head_kv = 12
|
||||
print_info: n_rot = 64
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 64
|
||||
print_info: n_embd_head_v = 64
|
||||
print_info: n_gqa = 1
|
||||
print_info: n_embd_k_gqa = 768
|
||||
print_info: n_embd_v_gqa = 768
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 3072
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = 0
|
||||
print_info: rope type = 0
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 10000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 4096
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = ?B
|
||||
print_info: model params = 444.94 M
|
||||
print_info: general.name = Tts
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 158626
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
load_tensors: skipping TTS tensor emb_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor emb_text.weight (size = 467140608 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.weight (size = 12582912 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.weight (size = 2359296 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.weight (size = 6291456 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.weight (size = 1179648 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor head_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 0 repeating layers to GPU
|
||||
load_tensors: offloaded 0/21 layers to GPU
|
||||
load_tensors: CPU_Mapped model buffer size = 592.48 MiB
|
||||
..............................................................
|
||||
llama_init_from_model: model default pooling_type is [0], but [-1] was specified
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 10000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (4096) -- the full capacity of the model will not be utilized
|
||||
llama_context: CPU output buffer size = 0.61 MiB
|
||||
llama_kv_cache: CPU KV buffer size = 120.00 MiB
|
||||
llama_kv_cache: size = 120.00 MiB ( 2048 cells, 20 layers, 1/1 seqs), K (f16): 60.00 MiB, V (f16): 60.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 30.75 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 7.01 MiB
|
||||
llama_context: graph nodes = 613
|
||||
llama_context: graph splits = 216 (with bs=512), 1 (with bs=1)
|
||||
vision_model_loader: model name:
|
||||
vision_model_loader: description: image encoder for MiniCPM-V
|
||||
vision_model_loader: GGUF version: 3
|
||||
vision_model_loader: alignment: 32
|
||||
vision_model_loader: n_tensors: 455
|
||||
vision_model_loader: n_kv: 20
|
||||
|
||||
vision_ctx: vision using Vulkan0 backend
|
||||
load_hparams: model_type: 0
|
||||
load_hparams: n_embd: 1152
|
||||
load_hparams: n_head: 16
|
||||
load_hparams: n_ff: 4304
|
||||
load_hparams: n_layer: 27
|
||||
load_hparams: ffn_op: gelu
|
||||
load_hparams: projection_dim: 0
|
||||
|
||||
--- vision hparams ---
|
||||
load_hparams: image_size: 448
|
||||
load_hparams: patch_size: 14
|
||||
load_hparams: minicpmv_version: 100045
|
||||
load_hparams: proj_scale_factor: 0
|
||||
load_hparams: n_wa_pattern: 0
|
||||
|
||||
load_hparams: model size: 1044.36 MiB
|
||||
load_hparams: metadata size: 0.16 MiB
|
||||
load_tensors: ffn up/down are swapped
|
||||
alloc_compute_meta: Vulkan0 compute buffer size = 100.30 MiB
|
||||
alloc_compute_meta: CPU compute buffer size = 18.30 MiB
|
||||
init tts....init t2w....voc_hg2_model: CPU backend using 8 threads
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=32000
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=200, n_mel=80, total_size=16000
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=200, n_mel=80, total_elements=16000
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=200, n_tokens=100, current_iter=0, current_total=0, new_total=100, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 1 (total_tokens=100, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 20]
|
||||
audition_audio_batch_encode: Final output: 20 tokens x 4096 dims
|
||||
=== Model Paths ===
|
||||
Base dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf
|
||||
LLM: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf [OK]
|
||||
Vision: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/vision/MiniCPM-o-4_5-vision-F16.gguf [OK]
|
||||
Audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf [OK]
|
||||
TTS: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf [OK]
|
||||
Projector: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf [OK]
|
||||
===================
|
||||
=== Initializing Omni Context ===
|
||||
Media type: 2 (omni: audio+vision)
|
||||
TTS enabled: yes
|
||||
Context size: 2048
|
||||
GPU layers: 99
|
||||
Token2Wav device: cpu
|
||||
Vision backend: metal
|
||||
TTS bin dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts
|
||||
Ref audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\vllm-omni-edge\benchmarks\edge_harness\results\e2e_expansion_20260923\evidence\minicpmo_cpp_radeon890m_prompted\fixture_0000.wav
|
||||
02:37:26.764 === omni_init start
|
||||
02:37:26.764 media_type = 2, duplex_mode = 0, base_output_dir = ./tools/omni/output
|
||||
02:37:26.764 === omni_init: loading new LLM model
|
||||
02:38:17.790 === omni_init: loading TTS model
|
||||
02:38:17.790 TTS model: loading with n_gpu_layers=0
|
||||
02:38:21.557 TTS sampler: temp=0.80, top_p=0.85, top_k=25, rep_penalty=1.05
|
||||
02:38:21.557 TTS: loading weights from GGUF (emb_code, emb_text, projector_semantic, head_code)...
|
||||
02:38:27.138 TTS: head_code shape: dim0=6562, dim1=768, will transpose to [6562, 768]
|
||||
02:38:27.143 TTS: head_code loaded, verifying first few values:
|
||||
02:38:27.143 head_code_weight[0] = 0.01361084 (expect ~0.01385498)
|
||||
02:38:27.143 head_code_weight[1] = -0.01660156 (expect ~-0.01647949)
|
||||
02:38:27.143 head_code_weight[768] = 0.04101562 (this is weight[1, 0], expect ~0.04150391)
|
||||
02:38:27.188 TTS: weights loaded successfully
|
||||
02:38:27.188 Projector: loading from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf
|
||||
02:38:27.290 Projector: loaded successfully
|
||||
02:38:27.290 === omni_init: loading APM model
|
||||
02:38:30.411 APM: init from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf
|
||||
02:38:36.023 Token2Wav: found models in \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf
|
||||
02:38:36.075 Token2Wav: all model files found, initializing session...
|
||||
02:38:36.075 Token2Wav: non-CUDA backend, vocoder using CPU for better performance
|
||||
02:38:36.075 Token2Wav: using prompt_cache from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf/prompt_cache.gguf
|
||||
02:38:40.781 Token2Wav: initialized successfully
|
||||
02:38:40.781 Python T2W: script_dir=./tools/omni/pyt2w, model_dir=./tools/omni/pyt2w/token2wav
|
||||
02:38:40.781 Token2Wav: 浣跨敤 C++ 瀹炵幇
|
||||
02:38:40.781 === omni_init success: ctx_llama = 0000016f6c97fe80
|
||||
=== Running test case ===
|
||||
Audio prefix: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-7-ynrdupcw\fixture_
|
||||
Count: 2
|
||||
02:38:40.781 stream_prefill: n_past = 0
|
||||
voice_clone_prompt = <|im_start|>system
|
||||
妯′豢闊抽�鏍锋湰鐨勯煶鑹插苟鐢熸垚鏂扮殑鍐呭�銆�
|
||||
<|audio_start|>
|
||||
assistant_prompt = <|audio_end|>浣犵殑浠诲姟鏄�敤杩欑�澹伴煶妯″紡鏉ュ綋涓€涓�姪鎵嬨€傝�璁ょ湡銆侀珮璐ㄩ噺鍦板洖澶嶇敤鎴风殑闂��銆傝�鐢ㄩ珮鑷�劧搴︾殑鏂瑰紡鍜岀敤鎴疯亰澶┿€�<|im_end|>
|
||||
<|im_start|>user
|
||||
|
||||
02:38:40.782 system prompt ref_audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-7-ynrdupcw\fixture_0000.wav
|
||||
02:38:41.285 system prompt ref_audio embedding: n_pos=20
|
||||
02:38:41.787 馃敀 n_keep 璁剧疆涓� 72 (system prompt tokens)锛岃繖閮ㄥ垎姘歌繙涓嶄細琚�粦鍔ㄧ獥鍙ｅ垹闄�
|
||||
02:38:41.787 stream_prefill(index=0): system prompt 鍒濆�鍖栧畬鎴愶紝ref_audio 宸插湪鍏朵腑 prefill
|
||||
02:38:41.787 [SW] system_prompt registered: preserve_length=72 (will be protected from sliding)
|
||||
02:38:41.787 n_past = 72
|
||||
02:38:41.787
|
||||
|
||||
c++ finish stream_prefill(index=0). n_past=72, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 0 (audio) : 1.00564 s
|
||||
encode_image_with_vision_chunks: step 1 of 1 encoded in 631.04 ms
|
||||
encode_image_with_vision_chunks: all 1 chunks encoded in 649.10 ms (grid: 1x1)
|
||||
|
||||
encode_image_with_vision_chunks: image encoded in 649.10 ms by vision ( 649.10 ms per chunk, 64 total tokens)
|
||||
omni_image_embed_make_chunks_with_filename: created 1 vision chunks from C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-7-ynrdupcw\fixture_0001.jpg
|
||||
stream_prefill: prefilled 1 vision chunks (64 tokens each)
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=49339
|
||||
audition_audio_preprocess: Audio not aligned to 100ms (49339 samples = 3083.7ms), padded with silence to 49600 samples (3100ms)
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=310, n_mel=80, total_size=24800
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=310, n_mel=80, total_elements=24800
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=310, n_tokens=155, current_iter=1, current_total=155, new_total=310, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 2 (total_tokens=310, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 31]
|
||||
audition_audio_batch_encode: Final output: 31 tokens x 4096 dims
|
||||
02:38:42.956 stream_prefill(index=1): processing user audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-7-ynrdupcw\fixture_0001.wav
|
||||
02:38:43.237 stream_prefill(index=1): user audio embedding: n_pos=31
|
||||
02:38:43.550
|
||||
|
||||
c++ finish stream_prefill(index=1). n_past=172, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 1 (audio+vision) : 1.76079 s
|
||||
02:38:43.550 馃搷 stream_decode 寮€濮�: n_past=172, n_keep=72, n_ctx=2048, duplex_mode=0
|
||||
02:38:43.550 stream_decode: create tts thread (simplex mode)
|
||||
02:38:43.550 stream_decode: create t2w thread
|
||||
02:38:43.550 wait prefill done
|
||||
02:38:43.550 馃搷 [鍗曞伐TTS] 娣诲姞 assistant prompt: "<|im_end|>
|
||||
<|im_start|>assistant
|
||||
<think>
|
||||
|
||||
</think>
|
||||
|
||||
<|tts_bos|>", n_past=172
|
||||
02:38:43.551 T2W thread (C++) started
|
||||
<user>
|
||||
<assistant>02:38:43.554 TTS: 鍒涘缓鍗曞伐妯″紡杈撳嚭鐩�綍: ./tools/omni/output/round_000
|
||||
02:38:43.555 TTS thread started
|
||||
02:38:43.711 馃搷 [鍗曞伐TTS] assistant prompt 瀹屾垚, n_past=182
|
||||
02:38:43.711 LLM decode: max_tgt_len = 96, n_predict = 96, n_ctx = 2048
|
||||
02:38:44.229 LLM: detected end token, set llm_generation_done=true
|
||||
02:38:44.229 LLM->TTS: text='The square in the image is red.', n_tokens=8, hidden_size=32768, n_embd=4096, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:38:44.229 馃搷 Decode 缁撴潫: n_past=191, 鍓╀綑绌洪棿=1857, 鏃犻渶婊戠獥
|
||||
02:38:44.229 馃搷 杞�� 1 缁撴潫锛岃�褰曡竟鐣屼簬 n_past=191
|
||||
02:38:44.229 TTS<-LLM: chunk_idx=0, text='The square in the image is red.', n_tokens=8, hidden_size=32768, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:38:44.229 TTS: after queue pop - speek_done=0, llm_finish=1, llm_text.empty=0, token_ids.size=8
|
||||
02:38:44.229 TTS: before empty check - response.empty=0, response='The square in the image is red.', llm_finish=1
|
||||
Waiting for audio generation to complete...
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_0.wav does not exist or is empty
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_1.wav does not exist or is empty
|
||||
TTS: no valid WAV files to merge
|
||||
02:38:44.229 TTS: DEBUG before has_llm_data - token_ids.size=8, hidden_states.size=32768, n_embd=4096
|
||||
02:38:44.229 TTS: has_llm_data=1, token_ids=8, hidden_states=32768, n_embd=4096, llm_finish=1
|
||||
02:38:44.230 TTS: n_tokens_orig=8, n_tokens_filtered=8 (filtered 0 special tokens)
|
||||
02:38:44.262 TTS: saved merged embeddings (text) to ./tools/omni/output/round_000/llm_debug/chunk_0/merged_embeddings.txt
|
||||
02:38:44.262 TTS Simplex: generating audio tokens for chunk 0 (n_tokens=8, tts_n_embd=768)
|
||||
02:38:44.262 TTS Simplex: 鍦� prefill 鍓嶆坊鍔� audio_bos (chunk_idx=0, new_size=9)
|
||||
02:38:44.262 TTS Simplex: first chunk - cleared KV cache
|
||||
02:38:44.262 TTS Simplex: first chunk - cleared tts_all_generated_tokens and tts_token_buffer
|
||||
02:38:44.322 馃搷 涓轰笅涓€杞�噯澶�: eval <|im_end|>\n<|im_start|>user\n, n_past=196
|
||||
02:38:44.531 TTS Simplex: prefill completed, n_past_tts=9
|
||||
02:38:44.531 TTS Simplex: sampler created
|
||||
02:38:44.531 TTS simplex: is_audio_bos=true (first audio token)
|
||||
02:38:44.557 TTS Simplex Phase1: token 1/500: rel_id=1487
|
||||
02:38:44.568 TTS Simplex Phase1: token 2/500: rel_id=4137
|
||||
02:38:44.579 TTS Simplex Phase1: token 3/500: rel_id=4218
|
||||
02:38:44.590 TTS Simplex Phase1: token 4/500: rel_id=4218
|
||||
02:38:44.601 TTS Simplex Phase1: token 5/500: rel_id=4218
|
||||
02:38:44.820 TTS Simplex Phase1: yield 25 tokens 鍒� T2W (step 25, buffer=25)
|
||||
02:38:44.820 TTS Simplex Phase1: token 25/500: rel_id=4913
|
||||
02:38:44.820 T2W绾跨▼(C++): 杞��鍒囨崲 (-1 -> 0)锛屾洿鏂拌緭鍑虹洰褰�
|
||||
02:38:44.820 T2W绾跨▼(C++): 鏂拌緭鍑虹洰褰� ./tools/omni/output/round_000/tts_wav
|
||||
02:38:45.093 TTS Simplex Phase1: yield 25 tokens 鍒� T2W (step 50, buffer=25)
|
||||
02:38:45.093 TTS Simplex Phase1: token 50/500: rel_id=300
|
||||
02:38:45.153 TTS Simplex Phase1: EOS token at step 56
|
||||
02:38:45.153 TTS Simplex: is_final_text_chunk=true, will enter Phase 2 for text_eos_embed
|
||||
02:38:45.153 TTS Simplex Phase2: injecting text_eos_embed + audio_bos at n_past=64
|
||||
02:38:45.161 TTS Simplex Phase2: text_eos_embed injected, n_past=65
|
||||
02:38:45.169 TTS Simplex Phase2: audio_bos injected, n_past=66, starting final generation
|
||||
02:38:45.201 TTS Simplex Phase2: EOS at step 3 (final end)
|
||||
02:38:45.201 TTS Simplex: chunk 缁撴潫锛宼ts_token_buffer 鍓╀綑 7 tokens (淇濈暀绛変笅涓€涓� chunk)
|
||||
02:38:45.201 TTS Simplex: is_final_text_chunk=true, flush 鍓╀綑 7 tokens 浠� tts_token_buffer
|
||||
02:38:45.201 TTS Simplex: generated 57 audio tokens for chunk 0
|
||||
02:38:45.201 TTS Simplex: chunk 0 first tokens: [1487 4137 4218 4218 4218 ], last tokens: [2031 4218 4137 1959 1959 ]
|
||||
02:38:45.201 TTS Simplex: updated tts_n_past_accumulated=69, total_generated_tokens=57
|
||||
02:38:45.212 TTS: finished processing all chunks
|
||||
02:38:45.212 TTS: 鍗曞伐妯″紡杞��缁撴潫锛屼笅涓€杞� round_idx=1
|
||||
02:38:45.212 TTS: sent is_final=true to T2W queue (turn end)
|
||||
02:38:47.162 馃帀 棣栧搷鏃堕棿 (First Audio Response): 3612ms
|
||||
02:38:47.162 T2W绾跨▼: wav_0.wav | 0.84s audio | 2340.5ms inference | RTF=2.79 | t=3612ms
|
||||
02:38:49.398 T2W绾跨▼: wav_1.wav | 1.00s audio | 2235.3ms inference | RTF=2.24 | t=5848ms
|
||||
02:38:51.488 T2W绾跨▼: wav_2.wav | 0.56s audio | 2089.2ms inference | RTF=3.73 | t=7937ms
|
||||
02:38:51.488 T2W绾跨▼: 鍐欏叆缁撴潫鏍囪� ./tools/omni/output/round_000/tts_wav/generation_done.flag (last_wav=2)
|
||||
02:38:51.488 T2W绾跨▼: 杞��缁撴潫鍚庢洿鏂拌緭鍑虹洰褰曚负 ./tools/omni/output/round_001/tts_wav
|
||||
02:38:51.574 omni_stop_threads: stop signals sent
|
||||
02:38:51.574 T2W(C++) 绾跨▼: 鍋滄�
|
||||
[timing] call=0 tokens=28 final=0 token2mel=2014.146ms vocoder=326.350ms total=2340.510ms audio=20160
|
||||
[timing] call=1 tokens=28 final=0 token2mel=1847.373ms vocoder=387.862ms total=2235.245ms audio=24000
|
||||
[timing] call=2 tokens=10 final=1 token2mel=1888.264ms vocoder=200.886ms total=2089.160ms audio=13440
|
||||
Audio generation completed.
|
||||
llama_perf_context_print: load time = 25776.92 ms
|
||||
llama_perf_context_print: prompt eval time = 2928.68 ms / 182 tokens ( 16.09 ms per token, 62.14 tokens per second)
|
||||
llama_perf_context_print: eval time = 515.44 ms / 9 runs ( 57.27 ms per token, 17.46 tokens per second)
|
||||
llama_perf_context_print: total time = 33639.40 ms / 191 tokens
|
||||
llama_perf_context_print: graphs reused = 10
|
||||
Token2Wav (C++): session released
|
||||
tts thread end
|
||||
t2w thread end
|
||||
+480
@@ -0,0 +1,480 @@
|
||||
build: 64 (5202b7b) with x86_64-w64-mingw32-gcc-posix (GCC) 13-posix for x86_64-w64-mingw32
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 30720 MiB free
|
||||
llama_model_loader: loaded meta data with 27 key-value pairs and 399 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = qwen3
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Llm
|
||||
llama_model_loader: - kv 3: general.size_label str = 8.2B
|
||||
llama_model_loader: - kv 4: qwen3.block_count u32 = 36
|
||||
llama_model_loader: - kv 5: qwen3.context_length u32 = 40960
|
||||
llama_model_loader: - kv 6: qwen3.embedding_length u32 = 4096
|
||||
llama_model_loader: - kv 7: qwen3.feed_forward_length u32 = 12288
|
||||
llama_model_loader: - kv 8: qwen3.attention.head_count u32 = 32
|
||||
llama_model_loader: - kv 9: qwen3.attention.head_count_kv u32 = 8
|
||||
llama_model_loader: - kv 10: qwen3.rope.freq_base f32 = 1000000.000000
|
||||
llama_model_loader: - kv 11: qwen3.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: qwen3.attention.key_length u32 = 128
|
||||
llama_model_loader: - kv 13: qwen3.attention.value_length u32 = 128
|
||||
llama_model_loader: - kv 14: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 15: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 16: tokenizer.ggml.tokens arr[str,151748] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 17: tokenizer.ggml.token_type arr[i32,151748] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 24: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 25: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 26: general.file_type u32 = 15
|
||||
llama_model_loader: - type f32: 145 tensors
|
||||
llama_model_loader: - type q4_K: 217 tensors
|
||||
llama_model_loader: - type q6_K: 37 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = Q4_K - Medium
|
||||
print_info: file size = 4.68 GiB (4.90 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = qwen3
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 40960
|
||||
print_info: n_embd = 4096
|
||||
print_info: n_layer = 36
|
||||
print_info: n_head = 32
|
||||
print_info: n_head_kv = 8
|
||||
print_info: n_rot = 128
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 128
|
||||
print_info: n_embd_head_v = 128
|
||||
print_info: n_gqa = 4
|
||||
print_info: n_embd_k_gqa = 1024
|
||||
print_info: n_embd_v_gqa = 1024
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 12288
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = -1
|
||||
print_info: rope type = 2
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 1000000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 40960
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = 8B
|
||||
print_info: model params = 8.19 B
|
||||
print_info: general.name = Llm
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 151748
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 36 repeating layers to GPU
|
||||
load_tensors: offloading output layer to GPU
|
||||
load_tensors: offloaded 37/37 layers to GPU
|
||||
load_tensors: Vulkan0 model buffer size = 4454.74 MiB
|
||||
load_tensors: CPU_Mapped model buffer size = 333.43 MiB
|
||||
.....................................................................................
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 1000000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (40960) -- the full capacity of the model will not be utilized
|
||||
llama_context: Vulkan_Host output buffer size = 0.58 MiB
|
||||
llama_kv_cache: Vulkan0 KV buffer size = 288.00 MiB
|
||||
llama_kv_cache: size = 288.00 MiB ( 2048 cells, 36 layers, 1/1 seqs), K (f16): 144.00 MiB, V (f16): 144.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 304.38 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 12.01 MiB
|
||||
llama_context: graph nodes = 1267
|
||||
llama_context: graph splits = 2
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 25673 MiB free
|
||||
llama_model_loader: loaded meta data with 30 key-value pairs and 193 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = llama
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Tts
|
||||
llama_model_loader: - kv 3: general.size_label str = 445M
|
||||
llama_model_loader: - kv 4: llama.block_count u32 = 20
|
||||
llama_model_loader: - kv 5: llama.context_length u32 = 4096
|
||||
llama_model_loader: - kv 6: llama.embedding_length u32 = 768
|
||||
llama_model_loader: - kv 7: llama.feed_forward_length u32 = 3072
|
||||
llama_model_loader: - kv 8: llama.attention.head_count u32 = 12
|
||||
llama_model_loader: - kv 9: llama.attention.head_count_kv u32 = 12
|
||||
llama_model_loader: - kv 10: llama.rope.freq_base f32 = 10000.000000
|
||||
llama_model_loader: - kv 11: llama.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: llama.attention.key_length u32 = 64
|
||||
llama_model_loader: - kv 13: llama.attention.value_length u32 = 64
|
||||
llama_model_loader: - kv 14: general.file_type u32 = 1
|
||||
llama_model_loader: - kv 15: llama.vocab_size u32 = 158626
|
||||
llama_model_loader: - kv 16: llama.rope.dimension_count u32 = 64
|
||||
llama_model_loader: - kv 17: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.tokens arr[str,158626] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.token_type arr[i32,158626] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 24: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 25: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 26: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 27: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 28: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 29: tokenizer.ggml.add_space_prefix bool = false
|
||||
llama_model_loader: - type f32: 50 tensors
|
||||
llama_model_loader: - type f16: 143 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = F16
|
||||
print_info: file size = 1.07 GiB (20.70 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = llama
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 4096
|
||||
print_info: n_embd = 768
|
||||
print_info: n_layer = 20
|
||||
print_info: n_head = 12
|
||||
print_info: n_head_kv = 12
|
||||
print_info: n_rot = 64
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 64
|
||||
print_info: n_embd_head_v = 64
|
||||
print_info: n_gqa = 1
|
||||
print_info: n_embd_k_gqa = 768
|
||||
print_info: n_embd_v_gqa = 768
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 3072
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = 0
|
||||
print_info: rope type = 0
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 10000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 4096
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = ?B
|
||||
print_info: model params = 444.94 M
|
||||
print_info: general.name = Tts
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 158626
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
load_tensors: skipping TTS tensor emb_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor emb_text.weight (size = 467140608 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.weight (size = 12582912 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.weight (size = 2359296 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.weight (size = 6291456 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.weight (size = 1179648 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor head_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 0 repeating layers to GPU
|
||||
load_tensors: offloaded 0/21 layers to GPU
|
||||
load_tensors: CPU_Mapped model buffer size = 592.48 MiB
|
||||
..............................................................
|
||||
llama_init_from_model: model default pooling_type is [0], but [-1] was specified
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 10000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (4096) -- the full capacity of the model will not be utilized
|
||||
llama_context: CPU output buffer size = 0.61 MiB
|
||||
llama_kv_cache: CPU KV buffer size = 120.00 MiB
|
||||
llama_kv_cache: size = 120.00 MiB ( 2048 cells, 20 layers, 1/1 seqs), K (f16): 60.00 MiB, V (f16): 60.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 30.75 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 7.01 MiB
|
||||
llama_context: graph nodes = 613
|
||||
llama_context: graph splits = 216 (with bs=512), 1 (with bs=1)
|
||||
vision_model_loader: model name:
|
||||
vision_model_loader: description: image encoder for MiniCPM-V
|
||||
vision_model_loader: GGUF version: 3
|
||||
vision_model_loader: alignment: 32
|
||||
vision_model_loader: n_tensors: 455
|
||||
vision_model_loader: n_kv: 20
|
||||
|
||||
vision_ctx: vision using Vulkan0 backend
|
||||
load_hparams: model_type: 0
|
||||
load_hparams: n_embd: 1152
|
||||
load_hparams: n_head: 16
|
||||
load_hparams: n_ff: 4304
|
||||
load_hparams: n_layer: 27
|
||||
load_hparams: ffn_op: gelu
|
||||
load_hparams: projection_dim: 0
|
||||
|
||||
--- vision hparams ---
|
||||
load_hparams: image_size: 448
|
||||
load_hparams: patch_size: 14
|
||||
load_hparams: minicpmv_version: 100045
|
||||
load_hparams: proj_scale_factor: 0
|
||||
load_hparams: n_wa_pattern: 0
|
||||
|
||||
load_hparams: model size: 1044.36 MiB
|
||||
load_hparams: metadata size: 0.16 MiB
|
||||
load_tensors: ffn up/down are swapped
|
||||
alloc_compute_meta: Vulkan0 compute buffer size = 100.30 MiB
|
||||
alloc_compute_meta: CPU compute buffer size = 18.30 MiB
|
||||
init tts....init t2w....voc_hg2_model: CPU backend using 8 threads
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=32000
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=200, n_mel=80, total_size=16000
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=200, n_mel=80, total_elements=16000
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=200, n_tokens=100, current_iter=0, current_total=0, new_total=100, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 1 (total_tokens=100, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 20]
|
||||
audition_audio_batch_encode: Final output: 20 tokens x 4096 dims
|
||||
=== Model Paths ===
|
||||
Base dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf
|
||||
LLM: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf [OK]
|
||||
Vision: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/vision/MiniCPM-o-4_5-vision-F16.gguf [OK]
|
||||
Audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf [OK]
|
||||
TTS: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf [OK]
|
||||
Projector: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf [OK]
|
||||
===================
|
||||
=== Initializing Omni Context ===
|
||||
Media type: 2 (omni: audio+vision)
|
||||
TTS enabled: yes
|
||||
Context size: 2048
|
||||
GPU layers: 99
|
||||
Token2Wav device: cpu
|
||||
Vision backend: metal
|
||||
TTS bin dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts
|
||||
Ref audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\vllm-omni-edge\benchmarks\edge_harness\results\e2e_expansion_20260923\evidence\minicpmo_cpp_radeon890m_prompted\fixture_0000.wav
|
||||
02:38:53.490 === omni_init start
|
||||
02:38:53.490 media_type = 2, duplex_mode = 0, base_output_dir = ./tools/omni/output
|
||||
02:38:53.490 === omni_init: loading new LLM model
|
||||
02:39:47.565 === omni_init: loading TTS model
|
||||
02:39:47.565 TTS model: loading with n_gpu_layers=0
|
||||
02:39:51.249 TTS sampler: temp=0.80, top_p=0.85, top_k=25, rep_penalty=1.05
|
||||
02:39:51.249 TTS: loading weights from GGUF (emb_code, emb_text, projector_semantic, head_code)...
|
||||
02:39:56.717 TTS: head_code shape: dim0=6562, dim1=768, will transpose to [6562, 768]
|
||||
02:39:56.722 TTS: head_code loaded, verifying first few values:
|
||||
02:39:56.722 head_code_weight[0] = 0.01361084 (expect ~0.01385498)
|
||||
02:39:56.722 head_code_weight[1] = -0.01660156 (expect ~-0.01647949)
|
||||
02:39:56.722 head_code_weight[768] = 0.04101562 (this is weight[1, 0], expect ~0.04150391)
|
||||
02:39:56.767 TTS: weights loaded successfully
|
||||
02:39:56.767 Projector: loading from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf
|
||||
02:39:56.868 Projector: loaded successfully
|
||||
02:39:56.869 === omni_init: loading APM model
|
||||
02:39:59.931 APM: init from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf
|
||||
02:40:05.413 Token2Wav: found models in \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf
|
||||
02:40:05.467 Token2Wav: all model files found, initializing session...
|
||||
02:40:05.467 Token2Wav: non-CUDA backend, vocoder using CPU for better performance
|
||||
02:40:05.467 Token2Wav: using prompt_cache from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf/prompt_cache.gguf
|
||||
02:40:10.039 Token2Wav: initialized successfully
|
||||
02:40:10.039 Python T2W: script_dir=./tools/omni/pyt2w, model_dir=./tools/omni/pyt2w/token2wav
|
||||
02:40:10.040 Token2Wav: 浣跨敤 C++ 瀹炵幇
|
||||
02:40:10.040 === omni_init success: ctx_llama = 000001509bdefe80
|
||||
=== Running test case ===
|
||||
Audio prefix: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-8-08wp6hsv\fixture_
|
||||
Count: 2
|
||||
02:40:10.040 stream_prefill: n_past = 0
|
||||
voice_clone_prompt = <|im_start|>system
|
||||
妯′豢闊抽�鏍锋湰鐨勯煶鑹插苟鐢熸垚鏂扮殑鍐呭�銆�
|
||||
<|audio_start|>
|
||||
assistant_prompt = <|audio_end|>浣犵殑浠诲姟鏄�敤杩欑�澹伴煶妯″紡鏉ュ綋涓€涓�姪鎵嬨€傝�璁ょ湡銆侀珮璐ㄩ噺鍦板洖澶嶇敤鎴风殑闂��銆傝�鐢ㄩ珮鑷�劧搴︾殑鏂瑰紡鍜岀敤鎴疯亰澶┿€�<|im_end|>
|
||||
<|im_start|>user
|
||||
|
||||
02:40:10.040 system prompt ref_audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-8-08wp6hsv\fixture_0000.wav
|
||||
02:40:10.659 system prompt ref_audio embedding: n_pos=20
|
||||
02:40:11.156 馃敀 n_keep 璁剧疆涓� 72 (system prompt tokens)锛岃繖閮ㄥ垎姘歌繙涓嶄細琚�粦鍔ㄧ獥鍙ｅ垹闄�
|
||||
02:40:11.156 stream_prefill(index=0): system prompt 鍒濆�鍖栧畬鎴愶紝ref_audio 宸插湪鍏朵腑 prefill
|
||||
02:40:11.156 [SW] system_prompt registered: preserve_length=72 (will be protected from sliding)
|
||||
02:40:11.156 n_past = 72
|
||||
02:40:11.156
|
||||
|
||||
c++ finish stream_prefill(index=0). n_past=72, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 0 (audio) : 1.11626 s
|
||||
encode_image_with_vision_chunks: step 1 of 1 encoded in 632.38 ms
|
||||
encode_image_with_vision_chunks: all 1 chunks encoded in 650.82 ms (grid: 1x1)
|
||||
|
||||
encode_image_with_vision_chunks: image encoded in 650.83 ms by vision ( 650.83 ms per chunk, 64 total tokens)
|
||||
omni_image_embed_make_chunks_with_filename: created 1 vision chunks from C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-8-08wp6hsv\fixture_0001.jpg
|
||||
stream_prefill: prefilled 1 vision chunks (64 tokens each)
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=49339
|
||||
audition_audio_preprocess: Audio not aligned to 100ms (49339 samples = 3083.7ms), padded with silence to 49600 samples (3100ms)
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=310, n_mel=80, total_size=24800
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=310, n_mel=80, total_elements=24800
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=310, n_tokens=155, current_iter=1, current_total=155, new_total=310, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 2 (total_tokens=310, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 31]
|
||||
audition_audio_batch_encode: Final output: 31 tokens x 4096 dims
|
||||
02:40:12.322 stream_prefill(index=1): processing user audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-8-08wp6hsv\fixture_0001.wav
|
||||
02:40:12.601 stream_prefill(index=1): user audio embedding: n_pos=31
|
||||
02:40:12.914
|
||||
|
||||
c++ finish stream_prefill(index=1). n_past=172, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 1 (audio+vision) : 1.75555 s
|
||||
02:40:12.914 馃搷 stream_decode 寮€濮�: n_past=172, n_keep=72, n_ctx=2048, duplex_mode=0
|
||||
02:40:12.914 stream_decode: create tts thread (simplex mode)
|
||||
02:40:12.914 stream_decode: create t2w thread
|
||||
02:40:12.914 wait prefill done
|
||||
02:40:12.914 馃搷 [鍗曞伐TTS] 娣诲姞 assistant prompt: "<|im_end|>
|
||||
<|im_start|>assistant
|
||||
<think>
|
||||
|
||||
</think>
|
||||
|
||||
<|tts_bos|>", n_past=172
|
||||
02:40:12.914 T2W thread (C++) started
|
||||
<user>
|
||||
<assistant>02:40:12.917 TTS: 鍒涘缓鍗曞伐妯″紡杈撳嚭鐩�綍: ./tools/omni/output/round_000
|
||||
02:40:12.918 TTS thread started
|
||||
02:40:13.075 馃搷 [鍗曞伐TTS] assistant prompt 瀹屾垚, n_past=182
|
||||
02:40:13.075 LLM decode: max_tgt_len = 96, n_predict = 96, n_ctx = 2048
|
||||
02:40:13.595 LLM: detected end token, set llm_generation_done=true
|
||||
02:40:13.595 LLM->TTS: text='The square in the image is red.', n_tokens=8, hidden_size=32768, n_embd=4096, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:40:13.595 馃搷 Decode 缁撴潫: n_past=191, 鍓╀綑绌洪棿=1857, 鏃犻渶婊戠獥
|
||||
02:40:13.595 馃搷 杞�� 1 缁撴潫锛岃�褰曡竟鐣屼簬 n_past=191
|
||||
02:40:13.595 TTS<-LLM: chunk_idx=0, text='The square in the image is red.', n_tokens=8, hidden_size=32768, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:40:13.595 TTS: after queue pop - speek_done=0, llm_finish=1, llm_text.empty=0, token_ids.size=8
|
||||
02:40:13.596 TTS: before empty check - response.empty=0, response='The square in the image is red.', llm_finish=1
|
||||
Waiting for audio generation to complete...
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_0.wav does not exist or is empty
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_1.wav does not exist or is empty
|
||||
TTS: no valid WAV files to merge
|
||||
02:40:13.596 TTS: DEBUG before has_llm_data - token_ids.size=8, hidden_states.size=32768, n_embd=4096
|
||||
02:40:13.596 TTS: has_llm_data=1, token_ids=8, hidden_states=32768, n_embd=4096, llm_finish=1
|
||||
02:40:13.596 TTS: n_tokens_orig=8, n_tokens_filtered=8 (filtered 0 special tokens)
|
||||
02:40:13.637 TTS: saved merged embeddings (text) to ./tools/omni/output/round_000/llm_debug/chunk_0/merged_embeddings.txt
|
||||
02:40:13.637 TTS Simplex: generating audio tokens for chunk 0 (n_tokens=8, tts_n_embd=768)
|
||||
02:40:13.637 TTS Simplex: 鍦� prefill 鍓嶆坊鍔� audio_bos (chunk_idx=0, new_size=9)
|
||||
02:40:13.637 TTS Simplex: first chunk - cleared KV cache
|
||||
02:40:13.637 TTS Simplex: first chunk - cleared tts_all_generated_tokens and tts_token_buffer
|
||||
02:40:13.687 馃搷 涓轰笅涓€杞�噯澶�: eval <|im_end|>\n<|im_start|>user\n, n_past=196
|
||||
02:40:13.904 TTS Simplex: prefill completed, n_past_tts=9
|
||||
02:40:13.904 TTS Simplex: sampler created
|
||||
02:40:13.904 TTS simplex: is_audio_bos=true (first audio token)
|
||||
02:40:13.932 TTS Simplex Phase1: token 1/500: rel_id=1978
|
||||
02:40:13.943 TTS Simplex Phase1: token 2/500: rel_id=3459
|
||||
02:40:13.954 TTS Simplex Phase1: token 3/500: rel_id=155
|
||||
02:40:13.966 TTS Simplex Phase1: token 4/500: rel_id=2990
|
||||
02:40:13.977 TTS Simplex Phase1: token 5/500: rel_id=4995
|
||||
02:40:14.200 TTS Simplex Phase1: yield 25 tokens 鍒� T2W (step 25, buffer=25)
|
||||
02:40:14.200 TTS Simplex Phase1: token 25/500: rel_id=6501
|
||||
02:40:14.200 T2W绾跨▼(C++): 杞��鍒囨崲 (-1 -> 0)锛屾洿鏂拌緭鍑虹洰褰�
|
||||
02:40:14.200 T2W绾跨▼(C++): 鏂拌緭鍑虹洰褰� ./tools/omni/output/round_000/tts_wav
|
||||
02:40:14.301 TTS Simplex Phase1: EOS token at step 35
|
||||
02:40:14.301 TTS Simplex: is_final_text_chunk=true, will enter Phase 2 for text_eos_embed
|
||||
02:40:14.301 TTS Simplex Phase2: injecting text_eos_embed + audio_bos at n_past=43
|
||||
02:40:14.309 TTS Simplex Phase2: text_eos_embed injected, n_past=44
|
||||
02:40:14.317 TTS Simplex Phase2: audio_bos injected, n_past=45, starting final generation
|
||||
02:40:14.492 TTS Simplex Phase2: EOS at step 16 (final end)
|
||||
02:40:14.492 TTS Simplex: chunk 缁撴潫锛宼ts_token_buffer 鍓╀綑 24 tokens (淇濈暀绛変笅涓€涓� chunk)
|
||||
02:40:14.492 TTS Simplex: is_final_text_chunk=true, flush 鍓╀綑 24 tokens 浠� tts_token_buffer
|
||||
02:40:14.492 TTS Simplex: generated 49 audio tokens for chunk 0
|
||||
02:40:14.492 TTS Simplex: chunk 0 first tokens: [1978 3459 155 2990 4995 ], last tokens: [4543 35 71 876 1038 ]
|
||||
02:40:14.492 TTS Simplex: updated tts_n_past_accumulated=61, total_generated_tokens=49
|
||||
02:40:14.503 TTS: finished processing all chunks
|
||||
02:40:14.503 TTS: 鍗曞伐妯″紡杞��缁撴潫锛屼笅涓€杞� round_idx=1
|
||||
02:40:14.503 TTS: sent is_final=true to T2W queue (turn end)
|
||||
02:40:16.519 馃帀 棣栧搷鏃堕棿 (First Audio Response): 3605ms
|
||||
02:40:16.519 T2W绾跨▼: wav_0.wav | 0.84s audio | 2317.4ms inference | RTF=2.76 | t=3605ms
|
||||
02:40:18.803 T2W绾跨▼: wav_1.wav | 1.24s audio | 2283.2ms inference | RTF=1.84 | t=5889ms
|
||||
02:40:18.803 T2W绾跨▼: 鍐欏叆缁撴潫鏍囪� ./tools/omni/output/round_000/tts_wav/generation_done.flag (last_wav=1)
|
||||
02:40:18.803 T2W绾跨▼: 杞��缁撴潫鍚庢洿鏂拌緭鍑虹洰褰曚负 ./tools/omni/output/round_001/tts_wav
|
||||
02:40:18.884 omni_stop_threads: stop signals sent
|
||||
02:40:18.885 T2W(C++) 绾跨▼: 鍋滄�
|
||||
[timing] call=0 tokens=28 final=0 token2mel=1988.870ms vocoder=328.484ms total=2317.370ms audio=20160
|
||||
[timing] call=1 tokens=27 final=1 token2mel=1875.420ms vocoder=407.799ms total=2283.231ms audio=29760
|
||||
Audio generation completed.
|
||||
llama_perf_context_print: load time = 25375.98 ms
|
||||
llama_perf_context_print: prompt eval time = 3034.76 ms / 182 tokens ( 16.67 ms per token, 59.97 tokens per second)
|
||||
llama_perf_context_print: eval time = 517.07 ms / 9 runs ( 57.45 ms per token, 17.41 tokens per second)
|
||||
llama_perf_context_print: total time = 31184.80 ms / 191 tokens
|
||||
llama_perf_context_print: graphs reused = 10
|
||||
Token2Wav (C++): session released
|
||||
tts thread end
|
||||
t2w thread end
|
||||
+484
@@ -0,0 +1,484 @@
|
||||
build: 64 (5202b7b) with x86_64-w64-mingw32-gcc-posix (GCC) 13-posix for x86_64-w64-mingw32
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 30720 MiB free
|
||||
llama_model_loader: loaded meta data with 27 key-value pairs and 399 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = qwen3
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Llm
|
||||
llama_model_loader: - kv 3: general.size_label str = 8.2B
|
||||
llama_model_loader: - kv 4: qwen3.block_count u32 = 36
|
||||
llama_model_loader: - kv 5: qwen3.context_length u32 = 40960
|
||||
llama_model_loader: - kv 6: qwen3.embedding_length u32 = 4096
|
||||
llama_model_loader: - kv 7: qwen3.feed_forward_length u32 = 12288
|
||||
llama_model_loader: - kv 8: qwen3.attention.head_count u32 = 32
|
||||
llama_model_loader: - kv 9: qwen3.attention.head_count_kv u32 = 8
|
||||
llama_model_loader: - kv 10: qwen3.rope.freq_base f32 = 1000000.000000
|
||||
llama_model_loader: - kv 11: qwen3.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: qwen3.attention.key_length u32 = 128
|
||||
llama_model_loader: - kv 13: qwen3.attention.value_length u32 = 128
|
||||
llama_model_loader: - kv 14: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 15: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 16: tokenizer.ggml.tokens arr[str,151748] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 17: tokenizer.ggml.token_type arr[i32,151748] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 24: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 25: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 26: general.file_type u32 = 15
|
||||
llama_model_loader: - type f32: 145 tensors
|
||||
llama_model_loader: - type q4_K: 217 tensors
|
||||
llama_model_loader: - type q6_K: 37 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = Q4_K - Medium
|
||||
print_info: file size = 4.68 GiB (4.90 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = qwen3
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 40960
|
||||
print_info: n_embd = 4096
|
||||
print_info: n_layer = 36
|
||||
print_info: n_head = 32
|
||||
print_info: n_head_kv = 8
|
||||
print_info: n_rot = 128
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 128
|
||||
print_info: n_embd_head_v = 128
|
||||
print_info: n_gqa = 4
|
||||
print_info: n_embd_k_gqa = 1024
|
||||
print_info: n_embd_v_gqa = 1024
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 12288
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = -1
|
||||
print_info: rope type = 2
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 1000000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 40960
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = 8B
|
||||
print_info: model params = 8.19 B
|
||||
print_info: general.name = Llm
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 151748
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 36 repeating layers to GPU
|
||||
load_tensors: offloading output layer to GPU
|
||||
load_tensors: offloaded 37/37 layers to GPU
|
||||
load_tensors: Vulkan0 model buffer size = 4454.74 MiB
|
||||
load_tensors: CPU_Mapped model buffer size = 333.43 MiB
|
||||
.....................................................................................
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 1000000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (40960) -- the full capacity of the model will not be utilized
|
||||
llama_context: Vulkan_Host output buffer size = 0.58 MiB
|
||||
llama_kv_cache: Vulkan0 KV buffer size = 288.00 MiB
|
||||
llama_kv_cache: size = 288.00 MiB ( 2048 cells, 36 layers, 1/1 seqs), K (f16): 144.00 MiB, V (f16): 144.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 304.38 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 12.01 MiB
|
||||
llama_context: graph nodes = 1267
|
||||
llama_context: graph splits = 2
|
||||
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon(TM) 890M Graphics) (unknown id) - 25673 MiB free
|
||||
llama_model_loader: loaded meta data with 30 key-value pairs and 193 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf (version GGUF V3 (latest))
|
||||
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
llama_model_loader: - kv 0: general.architecture str = llama
|
||||
llama_model_loader: - kv 1: general.type str = model
|
||||
llama_model_loader: - kv 2: general.name str = Tts
|
||||
llama_model_loader: - kv 3: general.size_label str = 445M
|
||||
llama_model_loader: - kv 4: llama.block_count u32 = 20
|
||||
llama_model_loader: - kv 5: llama.context_length u32 = 4096
|
||||
llama_model_loader: - kv 6: llama.embedding_length u32 = 768
|
||||
llama_model_loader: - kv 7: llama.feed_forward_length u32 = 3072
|
||||
llama_model_loader: - kv 8: llama.attention.head_count u32 = 12
|
||||
llama_model_loader: - kv 9: llama.attention.head_count_kv u32 = 12
|
||||
llama_model_loader: - kv 10: llama.rope.freq_base f32 = 10000.000000
|
||||
llama_model_loader: - kv 11: llama.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
llama_model_loader: - kv 12: llama.attention.key_length u32 = 64
|
||||
llama_model_loader: - kv 13: llama.attention.value_length u32 = 64
|
||||
llama_model_loader: - kv 14: general.file_type u32 = 1
|
||||
llama_model_loader: - kv 15: llama.vocab_size u32 = 158626
|
||||
llama_model_loader: - kv 16: llama.rope.dimension_count u32 = 64
|
||||
llama_model_loader: - kv 17: general.quantization_version u32 = 2
|
||||
llama_model_loader: - kv 18: tokenizer.ggml.model str = gpt2
|
||||
llama_model_loader: - kv 19: tokenizer.ggml.pre str = qwen2
|
||||
llama_model_loader: - kv 20: tokenizer.ggml.tokens arr[str,158626] = ["!", "\"", "#", "$", "%", "&", "'", ...
|
||||
llama_model_loader: - kv 21: tokenizer.ggml.token_type arr[i32,158626] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
|
||||
llama_model_loader: - kv 22: tokenizer.ggml.merges arr[str,151387] = ["臓 臓", "臓臓 臓臓", "i n", "臓 t",...
|
||||
llama_model_loader: - kv 23: tokenizer.ggml.bos_token_id u32 = 151644
|
||||
llama_model_loader: - kv 24: tokenizer.ggml.eos_token_id u32 = 151645
|
||||
llama_model_loader: - kv 25: tokenizer.ggml.unknown_token_id u32 = 128244
|
||||
llama_model_loader: - kv 26: tokenizer.ggml.padding_token_id u32 = 151643
|
||||
llama_model_loader: - kv 27: tokenizer.ggml.add_bos_token bool = false
|
||||
llama_model_loader: - kv 28: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
|
||||
llama_model_loader: - kv 29: tokenizer.ggml.add_space_prefix bool = false
|
||||
llama_model_loader: - type f32: 50 tensors
|
||||
llama_model_loader: - type f16: 143 tensors
|
||||
print_info: file format = GGUF V3 (latest)
|
||||
print_info: file type = F16
|
||||
print_info: file size = 1.07 GiB (20.70 BPW)
|
||||
load: printing all EOG tokens:
|
||||
load: - 151643 ('<|endoftext|>')
|
||||
load: - 151645 ('<|im_end|>')
|
||||
load: - 151662 ('<|fim_pad|>')
|
||||
load: - 151663 ('<|repo_name|>')
|
||||
load: - 151664 ('<|file_sep|>')
|
||||
load: special tokens cache size = 106
|
||||
load: token to piece cache size = 0.9319 MB
|
||||
print_info: arch = llama
|
||||
print_info: vocab_only = 0
|
||||
print_info: n_ctx_train = 4096
|
||||
print_info: n_embd = 768
|
||||
print_info: n_layer = 20
|
||||
print_info: n_head = 12
|
||||
print_info: n_head_kv = 12
|
||||
print_info: n_rot = 64
|
||||
print_info: n_swa = 0
|
||||
print_info: is_swa_any = 0
|
||||
print_info: n_embd_head_k = 64
|
||||
print_info: n_embd_head_v = 64
|
||||
print_info: n_gqa = 1
|
||||
print_info: n_embd_k_gqa = 768
|
||||
print_info: n_embd_v_gqa = 768
|
||||
print_info: f_norm_eps = 0.0e+00
|
||||
print_info: f_norm_rms_eps = 1.0e-06
|
||||
print_info: f_clamp_kqv = 0.0e+00
|
||||
print_info: f_max_alibi_bias = 0.0e+00
|
||||
print_info: f_logit_scale = 0.0e+00
|
||||
print_info: f_attn_scale = 0.0e+00
|
||||
print_info: n_ff = 3072
|
||||
print_info: n_expert = 0
|
||||
print_info: n_expert_used = 0
|
||||
print_info: causal attn = 1
|
||||
print_info: pooling type = 0
|
||||
print_info: rope type = 0
|
||||
print_info: rope scaling = linear
|
||||
print_info: freq_base_train = 10000.0
|
||||
print_info: freq_scale_train = 1
|
||||
print_info: n_ctx_orig_yarn = 4096
|
||||
print_info: rope_finetuned = unknown
|
||||
print_info: model type = ?B
|
||||
print_info: model params = 444.94 M
|
||||
print_info: general.name = Tts
|
||||
print_info: vocab type = BPE
|
||||
print_info: n_vocab = 158626
|
||||
print_info: n_merges = 151387
|
||||
print_info: BOS token = 151644 '<|im_start|>'
|
||||
print_info: EOS token = 151645 '<|im_end|>'
|
||||
print_info: EOT token = 151645 '<|im_end|>'
|
||||
print_info: UNK token = 128244 '<unk>'
|
||||
print_info: PAD token = 151643 '<|endoftext|>'
|
||||
print_info: LF token = 198 '膴'
|
||||
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
|
||||
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
|
||||
print_info: FIM MID token = 151660 '<|fim_middle|>'
|
||||
print_info: FIM PAD token = 151662 '<|fim_pad|>'
|
||||
print_info: FIM REP token = 151663 '<|repo_name|>'
|
||||
print_info: FIM SEP token = 151664 '<|file_sep|>'
|
||||
print_info: EOG token = 151643 '<|endoftext|>'
|
||||
print_info: EOG token = 151645 '<|im_end|>'
|
||||
print_info: EOG token = 151662 '<|fim_pad|>'
|
||||
print_info: EOG token = 151663 '<|repo_name|>'
|
||||
print_info: EOG token = 151664 '<|file_sep|>'
|
||||
print_info: max token length = 256
|
||||
load_tensors: loading model tensors, this can take a while... (mmap = true)
|
||||
load_tensors: skipping TTS tensor emb_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor emb_text.weight (size = 467140608 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.weight (size = 12582912 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.weight (size = 2359296 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_semantic.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.weight (size = 6291456 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear1.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.weight (size = 1179648 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor projector_spk.linear2.bias (size = 3072 bytes) -- will be loaded separately
|
||||
load_tensors: skipping TTS tensor head_code.0.weight (size = 20158464 bytes) -- will be loaded separately
|
||||
warning: PrefetchVirtualMemory failed: 内存资源不足，无法处理此命令。
|
||||
|
||||
load_tensors: offloading 0 repeating layers to GPU
|
||||
load_tensors: offloaded 0/21 layers to GPU
|
||||
load_tensors: CPU_Mapped model buffer size = 592.48 MiB
|
||||
..............................................................
|
||||
llama_init_from_model: model default pooling_type is [0], but [-1] was specified
|
||||
llama_context: constructing llama_context
|
||||
llama_context: n_seq_max = 1
|
||||
llama_context: n_ctx = 2048
|
||||
llama_context: n_ctx_per_seq = 2048
|
||||
llama_context: n_batch = 2048
|
||||
llama_context: n_ubatch = 512
|
||||
llama_context: causal_attn = 1
|
||||
llama_context: flash_attn = auto
|
||||
llama_context: kv_unified = false
|
||||
llama_context: freq_base = 10000.0
|
||||
llama_context: freq_scale = 1
|
||||
llama_context: n_ctx_per_seq (2048) < n_ctx_train (4096) -- the full capacity of the model will not be utilized
|
||||
llama_context: CPU output buffer size = 0.61 MiB
|
||||
llama_kv_cache: CPU KV buffer size = 120.00 MiB
|
||||
llama_kv_cache: size = 120.00 MiB ( 2048 cells, 20 layers, 1/1 seqs), K (f16): 60.00 MiB, V (f16): 60.00 MiB
|
||||
llama_context: Flash Attention was auto, set to enabled
|
||||
llama_context: Vulkan0 compute buffer size = 30.75 MiB
|
||||
llama_context: Vulkan_Host compute buffer size = 7.01 MiB
|
||||
llama_context: graph nodes = 613
|
||||
llama_context: graph splits = 216 (with bs=512), 1 (with bs=1)
|
||||
vision_model_loader: model name:
|
||||
vision_model_loader: description: image encoder for MiniCPM-V
|
||||
vision_model_loader: GGUF version: 3
|
||||
vision_model_loader: alignment: 32
|
||||
vision_model_loader: n_tensors: 455
|
||||
vision_model_loader: n_kv: 20
|
||||
|
||||
vision_ctx: vision using Vulkan0 backend
|
||||
load_hparams: model_type: 0
|
||||
load_hparams: n_embd: 1152
|
||||
load_hparams: n_head: 16
|
||||
load_hparams: n_ff: 4304
|
||||
load_hparams: n_layer: 27
|
||||
load_hparams: ffn_op: gelu
|
||||
load_hparams: projection_dim: 0
|
||||
|
||||
--- vision hparams ---
|
||||
load_hparams: image_size: 448
|
||||
load_hparams: patch_size: 14
|
||||
load_hparams: minicpmv_version: 100045
|
||||
load_hparams: proj_scale_factor: 0
|
||||
load_hparams: n_wa_pattern: 0
|
||||
|
||||
load_hparams: model size: 1044.36 MiB
|
||||
load_hparams: metadata size: 0.16 MiB
|
||||
load_tensors: ffn up/down are swapped
|
||||
alloc_compute_meta: Vulkan0 compute buffer size = 100.30 MiB
|
||||
alloc_compute_meta: CPU compute buffer size = 18.30 MiB
|
||||
init tts....init t2w....voc_hg2_model: CPU backend using 8 threads
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=32000
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=200, n_mel=80, total_size=16000
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=200, n_mel=80, total_elements=16000
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=200, n_tokens=100, current_iter=0, current_total=0, new_total=100, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 1 (total_tokens=100, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 20]
|
||||
audition_audio_batch_encode: Final output: 20 tokens x 4096 dims
|
||||
=== Model Paths ===
|
||||
Base dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf
|
||||
LLM: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf\MiniCPM-o-4_5-Q4_K_M.gguf [OK]
|
||||
Vision: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/vision/MiniCPM-o-4_5-vision-F16.gguf [OK]
|
||||
Audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf [OK]
|
||||
TTS: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-tts-F16.gguf [OK]
|
||||
Projector: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf [OK]
|
||||
===================
|
||||
=== Initializing Omni Context ===
|
||||
Media type: 2 (omni: audio+vision)
|
||||
TTS enabled: yes
|
||||
Context size: 2048
|
||||
GPU layers: 99
|
||||
Token2Wav device: cpu
|
||||
Vision backend: metal
|
||||
TTS bin dir: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts
|
||||
Ref audio: \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\vllm-omni-edge\benchmarks\edge_harness\results\e2e_expansion_20260923\evidence\minicpmo_cpp_radeon890m_prompted\fixture_0000.wav
|
||||
02:40:20.639 === omni_init start
|
||||
02:40:20.639 media_type = 2, duplex_mode = 0, base_output_dir = ./tools/omni/output
|
||||
02:40:20.639 === omni_init: loading new LLM model
|
||||
02:40:54.698 === omni_init: loading TTS model
|
||||
02:40:54.698 TTS model: loading with n_gpu_layers=0
|
||||
02:40:58.354 TTS sampler: temp=0.80, top_p=0.85, top_k=25, rep_penalty=1.05
|
||||
02:40:58.354 TTS: loading weights from GGUF (emb_code, emb_text, projector_semantic, head_code)...
|
||||
02:41:03.897 TTS: head_code shape: dim0=6562, dim1=768, will transpose to [6562, 768]
|
||||
02:41:03.901 TTS: head_code loaded, verifying first few values:
|
||||
02:41:03.901 head_code_weight[0] = 0.01361084 (expect ~0.01385498)
|
||||
02:41:03.901 head_code_weight[1] = -0.01660156 (expect ~-0.01647949)
|
||||
02:41:03.901 head_code_weight[768] = 0.04101562 (this is weight[1, 0], expect ~0.04150391)
|
||||
02:41:03.944 TTS: weights loaded successfully
|
||||
02:41:03.945 Projector: loading from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/tts/MiniCPM-o-4_5-projector-F16.gguf
|
||||
02:41:04.044 Projector: loaded successfully
|
||||
02:41:04.045 === omni_init: loading APM model
|
||||
02:41:07.144 APM: init from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/audio/MiniCPM-o-4_5-audio-F16.gguf
|
||||
02:41:12.666 Token2Wav: found models in \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf
|
||||
02:41:12.725 Token2Wav: all model files found, initializing session...
|
||||
02:41:12.725 Token2Wav: non-CUDA backend, vocoder using CPU for better performance
|
||||
02:41:12.725 Token2Wav: using prompt_cache from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\MiniCPM-o-4_5-gguf/token2wav-gguf/prompt_cache.gguf
|
||||
02:41:17.249 Token2Wav: initialized successfully
|
||||
02:41:17.249 Python T2W: script_dir=./tools/omni/pyt2w, model_dir=./tools/omni/pyt2w/token2wav
|
||||
02:41:17.249 Token2Wav: 浣跨敤 C++ 瀹炵幇
|
||||
02:41:17.249 === omni_init success: ctx_llama = 000001bb47d9fe80
|
||||
=== Running test case ===
|
||||
Audio prefix: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-9-7tnit0uk\fixture_
|
||||
Count: 2
|
||||
02:41:17.249 stream_prefill: n_past = 0
|
||||
voice_clone_prompt = <|im_start|>system
|
||||
妯′豢闊抽�鏍锋湰鐨勯煶鑹插苟鐢熸垚鏂扮殑鍐呭�銆�
|
||||
<|audio_start|>
|
||||
assistant_prompt = <|audio_end|>浣犵殑浠诲姟鏄�敤杩欑�澹伴煶妯″紡鏉ュ綋涓€涓�姪鎵嬨€傝�璁ょ湡銆侀珮璐ㄩ噺鍦板洖澶嶇敤鎴风殑闂��銆傝�鐢ㄩ珮鑷�劧搴︾殑鏂瑰紡鍜岀敤鎴疯亰澶┿€�<|im_end|>
|
||||
<|im_start|>user
|
||||
|
||||
02:41:17.249 system prompt ref_audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-9-7tnit0uk\fixture_0000.wav
|
||||
02:41:18.010 system prompt ref_audio embedding: n_pos=20
|
||||
02:41:18.505 馃敀 n_keep 璁剧疆涓� 72 (system prompt tokens)锛岃繖閮ㄥ垎姘歌繙涓嶄細琚�粦鍔ㄧ獥鍙ｅ垹闄�
|
||||
02:41:18.505 stream_prefill(index=0): system prompt 鍒濆�鍖栧畬鎴愶紝ref_audio 宸插湪鍏朵腑 prefill
|
||||
02:41:18.505 [SW] system_prompt registered: preserve_length=72 (will be protected from sliding)
|
||||
02:41:18.505 n_past = 72
|
||||
02:41:18.505
|
||||
|
||||
c++ finish stream_prefill(index=0). n_past=72, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 0 (audio) : 1.256 s
|
||||
encode_image_with_vision_chunks: step 1 of 1 encoded in 611.38 ms
|
||||
encode_image_with_vision_chunks: all 1 chunks encoded in 629.92 ms (grid: 1x1)
|
||||
|
||||
encode_image_with_vision_chunks: image encoded in 629.92 ms by vision ( 629.92 ms per chunk, 64 total tokens)
|
||||
omni_image_embed_make_chunks_with_filename: created 1 vision chunks from C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-9-7tnit0uk\fixture_0001.jpg
|
||||
stream_prefill: prefilled 1 vision chunks (64 tokens each)
|
||||
audition_audio_preprocess: Decoded audio - sample_rate=16000, n_samples=49339
|
||||
audition_audio_preprocess: Audio not aligned to 100ms (49339 samples = 3083.7ms), padded with silence to 49600 samples (3100ms)
|
||||
audition_get_mel_filters: Retrieved mel filters from model: 80 mel bins x 201 fft bins
|
||||
audition_audio_preprocess: Using mel filters - n_mel=80, n_fft=201
|
||||
audition_audio_preprocess: Generated 1 mel spectrogram chunk(s)
|
||||
audition_audio_preprocess: Mel spectrogram ready - n_len=310, n_mel=80, total_size=24800
|
||||
build_whisper: Whisper encoder graph built (projector output, pre-pooling)
|
||||
audition_audio_batch_encode: Audio input data - n_step=310, n_mel=80, total_elements=24800
|
||||
audition_audio_batch_encode: Before incrementing iter: input_frames=310, n_tokens=155, current_iter=1, current_total=155, new_total=310, max_tokens=1500
|
||||
audition_audio_batch_encode: KV cache iter incremented to 2 (total_tokens=310, max_tokens=1500)
|
||||
audition_audio_batch_encode: Final output shape: [4096, 31]
|
||||
audition_audio_batch_encode: Final output: 31 tokens x 4096 dims
|
||||
02:41:19.644 stream_prefill(index=1): processing user audio: C:\Users\zhout\w2\minicpmo_omni_profile20_work_radeon\omni-minicpmo-9-7tnit0uk\fixture_0001.wav
|
||||
02:41:19.942 stream_prefill(index=1): user audio embedding: n_pos=31
|
||||
02:41:20.256
|
||||
|
||||
c++ finish stream_prefill(index=1). n_past=172, n_keep=72, n_ctx=2048
|
||||
|
||||
prefill 1 (audio+vision) : 1.74985 s
|
||||
02:41:20.256 馃搷 stream_decode 寮€濮�: n_past=172, n_keep=72, n_ctx=2048, duplex_mode=0
|
||||
02:41:20.257 stream_decode: create tts thread (simplex mode)
|
||||
02:41:20.257 stream_decode: create t2w thread
|
||||
02:41:20.257 wait prefill done
|
||||
02:41:20.257 馃搷 [鍗曞伐TTS] 娣诲姞 assistant prompt: "<|im_end|>
|
||||
<|im_start|>assistant
|
||||
<think>
|
||||
|
||||
</think>
|
||||
|
||||
<|tts_bos|>", n_past=172
|
||||
02:41:20.257 T2W thread (C++) started
|
||||
<user>
|
||||
<assistant>02:41:20.261 TTS: 鍒涘缓鍗曞伐妯″紡杈撳嚭鐩�綍: ./tools/omni/output/round_000
|
||||
02:41:20.262 TTS thread started
|
||||
02:41:20.416 馃搷 [鍗曞伐TTS] assistant prompt 瀹屾垚, n_past=182
|
||||
02:41:20.416 LLM decode: max_tgt_len = 96, n_predict = 96, n_ctx = 2048
|
||||
02:41:20.935 LLM: detected end token, set llm_generation_done=true
|
||||
02:41:20.935 LLM->TTS: text='The square in the image is red.', n_tokens=8, hidden_size=32768, n_embd=4096, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:41:20.935 馃搷 Decode 缁撴潫: n_past=191, 鍓╀綑绌洪棿=1857, 鏃犻渶婊戠獥
|
||||
02:41:20.935 馃搷 杞�� 1 缁撴潫锛岃�褰曡竟鐣屼簬 n_past=191
|
||||
02:41:20.935 TTS<-LLM: chunk_idx=0, text='The square in the image is red.', n_tokens=8, hidden_size=32768, token_ids=[785 9334 304 279 2168 374 2518 13]
|
||||
02:41:20.935 TTS: after queue pop - speek_done=0, llm_finish=1, llm_text.empty=0, token_ids.size=8
|
||||
02:41:20.935 TTS: before empty check - response.empty=0, response='The square in the image is red.', llm_finish=1
|
||||
Waiting for audio generation to complete...
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_0.wav does not exist or is empty
|
||||
TTS: chunk file ./tools/omni/output/round_000/tts_wav/tts_output_chunk_1.wav does not exist or is empty
|
||||
TTS: no valid WAV files to merge
|
||||
02:41:20.935 TTS: DEBUG before has_llm_data - token_ids.size=8, hidden_states.size=32768, n_embd=4096
|
||||
02:41:20.935 TTS: has_llm_data=1, token_ids=8, hidden_states=32768, n_embd=4096, llm_finish=1
|
||||
02:41:20.936 TTS: n_tokens_orig=8, n_tokens_filtered=8 (filtered 0 special tokens)
|
||||
02:41:20.969 TTS: saved merged embeddings (text) to ./tools/omni/output/round_000/llm_debug/chunk_0/merged_embeddings.txt
|
||||
02:41:20.969 TTS Simplex: generating audio tokens for chunk 0 (n_tokens=8, tts_n_embd=768)
|
||||
02:41:20.969 TTS Simplex: 鍦� prefill 鍓嶆坊鍔� audio_bos (chunk_idx=0, new_size=9)
|
||||
02:41:20.969 TTS Simplex: first chunk - cleared KV cache
|
||||
02:41:20.969 TTS Simplex: first chunk - cleared tts_all_generated_tokens and tts_token_buffer
|
||||
02:41:21.029 馃搷 涓轰笅涓€杞�噯澶�: eval <|im_end|>\n<|im_start|>user\n, n_past=196
|
||||
02:41:21.239 TTS Simplex: prefill completed, n_past_tts=9
|
||||
02:41:21.239 TTS Simplex: sampler created
|
||||
02:41:21.239 TTS simplex: is_audio_bos=true (first audio token)
|
||||
02:41:21.264 TTS Simplex Phase1: token 1/500: rel_id=1604
|
||||
02:41:21.275 TTS Simplex Phase1: token 2/500: rel_id=4218
|
||||
02:41:21.286 TTS Simplex Phase1: token 3/500: rel_id=6405
|
||||
02:41:21.296 TTS Simplex Phase1: token 4/500: rel_id=6405
|
||||
02:41:21.307 TTS Simplex Phase1: token 5/500: rel_id=4218
|
||||
02:41:21.525 TTS Simplex Phase1: yield 25 tokens 鍒� T2W (step 25, buffer=25)
|
||||
02:41:21.525 TTS Simplex Phase1: token 25/500: rel_id=1950
|
||||
02:41:21.525 T2W绾跨▼(C++): 杞��鍒囨崲 (-1 -> 0)锛屾洿鏂拌緭鍑虹洰褰�
|
||||
02:41:21.525 T2W绾跨▼(C++): 鏂拌緭鍑虹洰褰� ./tools/omni/output/round_000/tts_wav
|
||||
02:41:21.801 TTS Simplex Phase1: yield 25 tokens 鍒� T2W (step 50, buffer=25)
|
||||
02:41:21.801 TTS Simplex Phase1: token 50/500: rel_id=2993
|
||||
02:41:21.813 TTS Simplex Phase1: EOS token at step 52
|
||||
02:41:21.814 TTS Simplex: is_final_text_chunk=true, will enter Phase 2 for text_eos_embed
|
||||
02:41:21.814 TTS Simplex Phase2: injecting text_eos_embed + audio_bos at n_past=60
|
||||
02:41:21.821 TTS Simplex Phase2: text_eos_embed injected, n_past=61
|
||||
02:41:21.828 TTS Simplex Phase2: audio_bos injected, n_past=62, starting final generation
|
||||
02:41:22.071 TTS Simplex Phase2: EOS at step 22 (final end)
|
||||
02:41:22.071 TTS Simplex: chunk 缁撴潫锛宼ts_token_buffer 鍓╀綑 22 tokens (淇濈暀绛変笅涓€涓� chunk)
|
||||
02:41:22.071 TTS Simplex: is_final_text_chunk=true, flush 鍓╀綑 22 tokens 浠� tts_token_buffer
|
||||
02:41:22.071 TTS Simplex: generated 72 audio tokens for chunk 0
|
||||
02:41:22.071 TTS Simplex: chunk 0 first tokens: [1604 4218 6405 6405 4218 ], last tokens: [2109 4299 4218 4218 3894 ]
|
||||
02:41:22.071 TTS Simplex: updated tts_n_past_accumulated=84, total_generated_tokens=72
|
||||
02:41:22.081 TTS: finished processing all chunks
|
||||
02:41:22.081 TTS: 鍗曞伐妯″紡杞��缁撴潫锛屼笅涓€杞� round_idx=1
|
||||
02:41:22.081 TTS: sent is_final=true to T2W queue (turn end)
|
||||
02:41:23.885 馃帀 棣栧搷鏃堕棿 (First Audio Response): 3629ms
|
||||
02:41:23.885 T2W绾跨▼: wav_0.wav | 0.84s audio | 2358.9ms inference | RTF=2.81 | t=3629ms
|
||||
02:41:26.120 T2W绾跨▼: wav_1.wav | 1.00s audio | 2234.0ms inference | RTF=2.23 | t=5863ms
|
||||
02:41:28.369 T2W绾跨▼: wav_2.wav | 1.16s audio | 2247.7ms inference | RTF=1.94 | t=8112ms
|
||||
02:41:28.369 T2W绾跨▼: 鍐欏叆缁撴潫鏍囪� ./tools/omni/output/round_000/tts_wav/generation_done.flag (last_wav=2)
|
||||
02:41:28.369 T2W绾跨▼: 杞��缁撴潫鍚庢洿鏂拌緭鍑虹洰褰曚负 ./tools/omni/output/round_001/tts_wav
|
||||
02:41:28.395 omni_stop_threads: stop signals sent
|
||||
02:41:28.395 T2W(C++) 绾跨▼: 鍋滄�
|
||||
[timing] call=0 tokens=28 final=0 token2mel=2030.517ms vocoder=328.321ms total=2358.857ms audio=20160
|
||||
[timing] call=1 tokens=28 final=0 token2mel=1853.947ms vocoder=380.063ms total=2234.021ms audio=24000
|
||||
[timing] call=2 tokens=25 final=1 token2mel=1868.863ms vocoder=378.787ms total=2247.660ms audio=27840
|
||||
Audio generation completed.
|
||||
llama_perf_context_print: load time = 25571.93 ms
|
||||
llama_perf_context_print: prompt eval time = 3166.14 ms / 182 tokens ( 17.40 ms per token, 57.48 tokens per second)
|
||||
llama_perf_context_print: eval time = 516.26 ms / 9 runs ( 57.36 ms per token, 17.43 tokens per second)
|
||||
llama_perf_context_print: total time = 33550.58 ms / 191 tokens
|
||||
llama_perf_context_print: graphs reused = 10
|
||||
Token2Wav (C++): session released
|
||||
tts thread end
|
||||
t2w thread end
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
All 60 pairings have a disposition. The timing tables below remain the 2026-09-22 measurements. The configuration matrix and affected per-cell reasons incorporate the [2026-09-23 scoped recovery runs](../../../e2e_recovery_20260923/README.md) and [WSL CPU/native Windows expansion](../../../e2e_expansion_20260923/README.md). The expansion adds separate 20-request serial MiniCPM-o text-to-speech profiles on WSL CPU and native Windows GPU, a Spark output-head component profile on the HX370 AMD NPU, complete Spark text-request profiles through standalone llama.cpp and a bounded Omni whole-session stage on native Windows CPU and Radeon 890M, native Windows CPU Qwen3-TTS text-to-WAV profiles through both standalone PyTorch and a bounded Omni whole-session stage, and a hybrid CPU+Radeon 890M Qwen3-TTS profile likewise repeated through a bounded Omni whole-session stage. A pinned GGUF MiniCPM-o route adds combined image+audio to text+speech runs on WSL CPU, native Windows CPU and native Windows Radeon 890M+CPU. All three now pass a bounded complete request through an Omni graph stage; the WSL CPU path additionally passed one warmup plus 20 serial measured requests. These are scoped synthetic cases, not full release qualification.
|
||||
|
||||
The WSL CPU and CPU+Radeon InternVLA cells also have [bounded Omni complete-policy profiles](../../../e2e_expansion_20260923/evidence/internvla_omni_policy/README.md): one warmup and 20 serial patterned synthetic requests per placement, plus admission refusal, cancellation and a public hybrid `AsyncOmni.generate` request. Separate native Windows [CPU](../../../e2e_expansion_20260923/evidence/internvla_omni_windows_cpu/README.md) and [CPU+Radeon](../../../e2e_expansion_20260923/evidence/internvla_omni_windows_hybrid/README.md) Omni profiles add 20 serial patterned requests per placement, refusal, cancellation and public action requests. These advance only the synthetic action-policy path. Real observations/reference actions, physical action units/order/step time, task quality, loading peak and sustained power/thermal behavior remain unverified. The earlier [Cosmos component and direct hybrid results](../../../e2e_expansion_20260923/evidence/internvla_cosmos_radeon890m/README.md) remain separate experiments.
|
||||
The WSL CPU and CPU+Radeon InternVLA cells also have [bounded Omni complete-policy profiles](../../../e2e_expansion_20260923/evidence/internvla_omni_policy/README.md): one warmup and 20 serial patterned synthetic requests per placement, plus admission refusal, cancellation and a public hybrid `AsyncOmni.generate` request. Separate native Windows [CPU](../../../e2e_expansion_20260923/evidence/internvla_omni_windows_cpu/README.md) and [CPU+Radeon](../../../e2e_expansion_20260923/evidence/internvla_omni_windows_hybrid/README.md) Omni profiles add 20 serial patterned requests per placement, refusal, cancellation and public action requests. These advance only the synthetic action-policy path. Real observations/reference actions, physical action units/order/step time, task quality, loading peak and sustained power/thermal behavior remain unverified. The earlier [Cosmos component and direct hybrid results](../../../e2e_expansion_20260923/evidence/internvla_cosmos_radeon890m/README.md) remain separate experiments. A new [20-request native Radeon MiniCPM-o profile](../../../e2e_expansion_20260923/evidence/minicpmo_omni_radeon_profile20/README.md) passed the scoped combined input/output path but exposed a large memory-sensitive latency regression versus its earlier one-request smoke; its row below supersedes the smoke for repeated-request timing.
|
||||
|
||||
## Configuration × model
|
||||
|
||||
@@ -56,7 +56,7 @@ NOT E2E means no complete device/model workload was verified. “Scoped E2E” i
|
||||
| PC HX370 + Radeon 890M / Windows worker | Spark-X2.5 | SCOPED_E2E_OMNI_COMPLETE_REQUEST | The same pinned Spark-X2.5-1.7B Q4_K_M GGUF completed text prefill/decode through a bounded Omni whole-session stage with llama.cpp Vulkan1. The log identifies Radeon 890M and 29/29 layers offloaded there, rather than RTX 5090; an intentionally wrong device name was refused. `Paris` and `1511` passed; one warmup plus 20 serial complete requests were 20/20 correct, nearest-rank wall p50/p95 2.10/2.14 s. A public AsyncOmni request passed; an oversized context was rejected with a later correct request; in-flight cancellation produced no stale output and released the ledger. The prior standalone run was 2.18/2.20 s. No streaming, restart-after-cancel, GPU-memory peak or sustained power/thermal check is claimed. | Qualify public configuration, broader text/token quality, streaming and cancellation recovery, shared-RAM/GPU peak and sustained use; [Omni raw evidence](../../../e2e_expansion_20260923/evidence/spark_omni_llamacpp/README.md), [standalone evidence](../../../e2e_expansion_20260923/evidence/spark_radeon890m_llamacpp/README.md). |
|
||||
| PC HX370 + Radeon 890M / Windows worker | Qwen3-TTS 0.6B CustomVoice | SCOPED_E2E_OMNI_COMPLETE_REQUEST_HYBRID | The pinned Q8_0 CustomVoice talker, F16 codec and punctuation GGUFs completed two named text-to-WAV requests through an Omni whole-session CrispASR stage on native Windows. The runtime log identifies Radeon 890M Vulkan0 for talker/codec and the 432 MB code predictor pinned to CPU FP32; this is not all-Radeon or NPU execution. Both PCM hashes exactly match the earlier standalone outputs. Whisper tiny.en transcribed the first Omni WAV and both standalone WAVs exactly (WER 0 as an intelligibility proxy). One warmup plus 20 serial complete Omni requests for 2.56 s of audio had identical PCM and nearest-rank wall p50/p95 3.298/3.336 s (RTF 1.29/1.30). A public AsyncOmni audio request passed; cancellation drained the worker without stale output or remaining reservation. The earlier standalone profile was 3.310/3.345 s. The default Vulkan configuration previously fell back to CPU, all-Vulkan ran away to a 360-frame cap, and DirectML failed waveform quality. No playable stream, post-cancel restart, loading peak, broad speech quality or sustained power/thermal run is claimed. | Validate playable streaming, restart/recovery, broader voice quality, shared-RAM/GPU peak, concurrency and sustained real-time behavior before wider qualification; [Omni raw evidence](../../../e2e_expansion_20260923/evidence/qwen_tts_omni_hybrid/README.md), [standalone evidence](../../../e2e_expansion_20260923/evidence/qwen_tts_radeon890m_joint/README.md), [failed DirectML route](../../../e2e_expansion_20260923/evidence/qwen_tts_radeon890m_directml/README.md). |
|
||||
| PC HX370 + Radeon 890M / Windows worker | Qwen3.8-27B | NOT_E2E_PROFILEABLE | FP32 448x448 vision tower: DirectML 85/113 nodes; normalized max error 9.095e-06 versus torch CPU. Historical component path, not full 27B or current graph-stage image-to-text integration. | Recover the hashed tower bundle and qualify its current Omni graph handoff with the language stage. |
|
||||
| PC HX370 + Radeon 890M / Windows worker | MiniCPM-o 4.5 | SCOPED_E2E_OMNI_COMPLETE_REQUEST_HYBRID | The same pinned GGUF set ran one combined synthetic spoken-question+JPEG → text+speech request through a bounded Omni whole-session graph stage. Logs verify `GGML_VK_VISIBLE_DEVICES=1`, Radeon 890M Vulkan0 language (37/37 layers) and vision, with CPU TTS/Token2Wav/vocoder. It answered “The square in the image is red.” and produced 2.60 s of 24 kHz speech in 18.27 s complete-request wall time (one cold sample, zero warmups); sampled worker RSS peaked at 15.57 GB. The stage verified artifact and binary hashes, reserved 24 GiB shared RAM, emitted a terminal audio event, and in-flight cancellation left no stale output or reservation. The prior standalone prompted run took 18.63 s and its output transcribed with WER 0. This is CPU+iGPU joint execution, not all-Radeon or NPU execution; neither run qualifies general quality or sustained performance. | Validate real multimodal quality, playable streaming, post-cancel restart, loading peak, repeated/concurrent latency and sustained thermal behavior. [Omni raw evidence](../../../e2e_expansion_20260923/evidence/minicpmo_omni_cpp/README.md), [standalone prompted evidence](../../../e2e_expansion_20260923/evidence/minicpmo_cpp_radeon890m_prompted/README.md). |
|
||||
| PC HX370 + Radeon 890M / Windows worker | MiniCPM-o 4.5 | SCOPED_E2E_OMNI_COMPLETE_REQUEST_HYBRID; PERFORMANCE_UNSTABLE | The pinned GGUF set ran a combined synthetic spoken-question+JPEG → text+speech workload through a bounded Omni whole-session graph stage. All 21 successful worker logs in a new serial profile verified Radeon 890M Vulkan0 language (37/37 layers) and vision, with CPU TTS/Token2Wav/vocoder; each request launched a fresh C++ model session. After one warmup, 20/20 requests answered “The square in the image is red.” and produced nonzero 2.08–3.00 s 24 kHz speech with terminal audio events. Nearest-rank complete-request wall p50/p95 was **87.27/91.45 s**, range 69.91–97.90 s, excluding 51.92 s controller startup. A separate cold rerun was 88.01 s. The first measured WAV transcribed exactly with pinned Whisper tiny.en (WER 0, one-output intelligibility proxy). A 21 GiB shared-RAM demand/capacity fit the 23.80 GB OS-available pre-load sample; maximum sampled worker RSS was 15.30 GB. Every successful new worker logged `PrefetchVirtualMemory failed` twice. The older one-cold-request Omni result was 18.27 s under a 24 GiB reservation/30 GiB declared ceiling and had no prefetch warning. The cause of the roughly fivefold regression is unqualified; no matched CPU control, OS memory/power time series or shared-iGPU allocation trace exists. In-flight cancellation left no stale output or reservation. This is CPU+iGPU joint execution, not all-Radeon, NPU, general quality or sustained performance qualification. | Measure Windows available/commit/pagefile and GPU shared-memory over matched CPU/Radeon runs; isolate UNC weight I/O, then validate real multimodal quality, playable streaming, post-cancel restart, loading peak, concurrency and sustained thermal behavior. [20-request Omni evidence](../../../e2e_expansion_20260923/evidence/minicpmo_omni_radeon_profile20/README.md), [earlier single-request Omni evidence](../../../e2e_expansion_20260923/evidence/minicpmo_omni_cpp/README.md). |
|
||||
| PC HX370 + Radeon 890M / Windows worker | InternVLA-A1 | SYNTHETIC_POLICY_PASS_HYBRID_OMNI; NOT_TASK_E2E | The real Place_Markpen checkpoint ran in admitted Omni whole-policy stages on both WSL CPU+Radeon and native Windows CPU+Radeon. In each, the BF16 Qwen3-VL/flow action policy stayed on CPU and a pinned six-frame FP32 Cosmos encoder used Omni's external DirectML worker; the selected Radeon 890M and graph output device were verified, without internal per-op attribution. Each placement returned 20/20 identical finite `[1,50,32]` actions after one warmup. WSL hybrid complete-request p50/p95 was 3.120/3.197 s versus its separate CPU baseline 3.175/3.219 s, with action relative L2 difference 2.305%. Native Windows hybrid was 4.575/4.725 s versus its separate CPU baseline 4.878/4.902 s, with action relative L2 difference 2.4391%. Native hybrid startup was 140.31 s versus CPU 135.60 s. These are separate one-fixture serial runs, not paired speedup or task-quality proof. FP32 encoder is an explicit precision change after DirectML BF16 aborted. Both stages verified hashes, reserved 16 GiB shared host RAM, rejected insufficient demand before launch, emitted terminal actions through public `AsyncOmni.generate`, and released reservations after in-flight cancellation/shutdown. An initial WSL hybrid shutdown quarantined 16 GiB; the lifecycle fix and reruns cleared it. The older direct hybrid zero-input profile is a different experiment. | Validate real observations/reference actions, physical units/order/step time, post-cancel restart, loading peak, concurrency and sustained tail/power before deployment; [WSL Omni evidence](../../../e2e_expansion_20260923/evidence/internvla_omni_policy/README.md), [native Windows Omni evidence](../../../e2e_expansion_20260923/evidence/internvla_omni_windows_hybrid/README.md), [earlier direct component evidence](../../../e2e_expansion_20260923/evidence/internvla_cosmos_radeon890m/README.md). |
|
||||
| PC HX370 + AMD NPU / Windows worker | Spark-X2.5 | COMPONENT_PASS; NOT_E2E_PROFILEABLE | The real 1.7B output head (RMS norm and full 131,072-logit projection) ran through Omni `external.graph.v1` with VitisAI: one fused NPU partition and seven CPU nodes per request. The A16W8 graph matched top-1 on four captured activations (full-logit SNR 55.43–56.81 dB). One warmup plus 20 serial repeated-input component requests gave p50/p95 10.21/12.44 ms, versus 17.97/20.29 ms for the original FP32 output head on the same Omni CPU stage. NPU startup was 18.21 s versus 4.48 s for FP32 CPU, and NPU worker peak RSS was 3.52 GiB versus 2.06 GiB. No full prefill/decode/KV/sampling, real handoff, power, or sustained run was verified. | Measure an actual vLLM-to-output-head handoff and whole-generation cost, state/cancellation, memory admission and quality before considering a split plan; [component evidence](../../../e2e_expansion_20260923/evidence/spark_amd_npu_output_head/README.md). |
|
||||
| PC HX370 + AMD NPU / Windows worker | Qwen3-TTS 0.6B CustomVoice | NOT_E2E_PROFILEABLE | No placement- and quality-verified whole-model graph bundle/stateful adapter for this model on this AMD route. Small backend graphs do not establish model support. | Supply and validate the missing matching artifact/backend, then run the complete specified workload. |
|
||||
|
||||
@@ -76,6 +76,17 @@ Radeon 890M, but per-operator placement was not measured. Action relative
|
||||
L2 difference was 2.4391% without a task tolerance; the hybrid startup was
|
||||
4.72 s longer. Real task quality and sustained benefit remain open.
|
||||
|
||||
The [native Windows Radeon MiniCPM-o repeated profile](../../benchmarks/edge_harness/results/e2e_expansion_20260923/evidence/minicpmo_omni_radeon_profile20/README.md)
|
||||
passed 20 serial combined synthetic audio+image to text+speech requests after
|
||||
one warmup, but complete-request p50/p95 was 87.27/91.45 s and a separate
|
||||
cold rerun was 88.01 s. The earlier single cold Omni request was 18.27 s;
|
||||
all 21 successful new worker logs show Windows prefetch-memory warnings and
|
||||
roughly 25–27 s model loads rather than 6.07 s. The placement remains scoped
|
||||
functionally, while repeated latency and its cause remain unqualified. The
|
||||
next M4a performance experiment must capture matched CPU/Radeon host-memory,
|
||||
pagefile, shared-GPU-memory, file-I/O and power traces, not infer a speedup
|
||||
from the earlier single sample.
|
||||
|
||||
The WSL RTX MiniCPM-o image-to-text+speech path now passes for one 448×448
|
||||
synthetic red-square image under an explicit 0.61 thinker GPU budget. The
|
||||
previous 0.58 budget failed the 2,048-token KV admission gate; the successful
|
||||
|
||||
Reference in new issue
Block a user