Qualify Spark GGUF Omni stage on WSL CPU
This commit is contained in:
tzhouam committed
1 parent
36ef53d9b1
commit
d5c1f2bdc1
19 files changed
+3777
-4
No files matched your search
@@ -18,6 +18,8 @@ A later [native Windows Radeon MiniCPM-o Omni profile](evidence/minicpmo_omni_ra
|
||||
|
||||
The [native-storage follow-up](evidence/minicpmo_omni_native_weights_io/README.md) used the same ten hashed GGUF weights copied to Windows NTFS. Native Windows CPU and Radeon+CPU both passed one warmup and 20 serial combined requests through Omni at nearest-rank complete-request p50/p95 **23.62/23.88 s** and **16.35/17.20 s**, respectively, with no prefetch warnings. First measured WAVs matched the expected sentence using pinned local ASR; both paths passed in-flight abort with a zero resource ledger. Four sequential one-request controls using native versus WSL UNC weights associate UNC I/O with the older slowdown, but cache, run order and power were not controlled. The slow UNC profile remains valid for its storage path; the CPU/Radeon profiles are not a paired speedup experiment. An initial 21 GiB Radeon repeat was correctly refused at admission under temporarily low available RAM and preserved as failure evidence.
|
||||
|
||||
A separate [WSL CPU Spark GGUF Omni profile](evidence/spark_omni_wsl_cpu/README.md) ran the same pinned Q4_K_M artifact through a Linux CPU-only llama.cpp server and current Omni controller. It passed one warmup and 20/20 serial inventory requests at nearest-rank complete-request wall p50/p95 **7.82/7.97 s**, a public `AsyncOmni.generate` text request, explicit context refusal and in-flight cancellation cleanup with a 4 GiB host reservation. The Linux server omitted the Windows offload-count line; the stage now requires positive CPU tensor-buffer evidence for that log layout and rejects accelerator buffers. This is independent of the earlier WSL vLLM INT8 result and does not qualify incremental streaming or general text quality.
|
||||
|
||||
| Device / model | Verified depth | Observed output and time | Remaining gate |
|
||||
|---|---|---|---|
|
||||
| Native Windows HX370 Radeon 890M / InternVLA-A1 Cosmos CI8x8 encoder | Component C; unchanged Omni encoder source in separate DirectML/PyTorch 2.4 environment, not an Omni full-policy stage | One warmup + 20 serial patterned 256×256 image encodes, including upload/readback: Radeon p50/p95 26.56/27.49 ms versus CPU 156.49/158.71 ms; relative L2 error 4.73e-07. Encoder weights and output device checked. | Complete policy/action and real-observation parity; policy/controller runtime compatibility, bounded handoff, shared-RAM admission and end-to-end benefit. [Raw evidence](evidence/internvla_cosmos_radeon890m/README.md). |
|
||||
|
||||
+17
@@ -0,0 +1,17 @@
|
||||
# Spark-X2.5 GGUF through Omni on WSL CPU
|
||||
|
||||
The pinned Spark-X2.5-1.7B Q4_K_M GGUF now completes bounded text requests through Omni `StageRuntime`/`StagePool` on WSL CPU. A separate public `AsyncOmni.generate` request passed. This adds an admitted current-Omni GGUF path to the WSL CPU matrix cell; it does not replace the older vLLM INT8 profile or qualify broad text quality, incremental token streaming or post-cancel restart.
|
||||
|
||||
Host: Ryzen AI 9 HX 370, Ubuntu 26.04 on WSL2 kernel `6.18.33.2-microsoft-standard-WSL2`. The RTX 5090 Laptop GPU was not selected; `CUDA_VISIBLE_DEVICES` was empty and llama.cpp used `-dev none -ngl 0`. The installed controller packages were vLLM `0.29.0` and Omni `0.29.0rc2.dev71+g00510ce38.d20260922`. The executed Linux CPU-only llama.cpp server reported build `10849`, commit `e71b80510`, and SHA256 `0d6bb6ade8da8f3231917ce23261da5331333df0007dc4bda74c00e0774e092c`. The [model](https://huggingface.co/XHToken/Spark-X2.5-1.7B-GGUF/tree/1f7fa33b1245c14730da39e125714ad3a327901b) revision was `1f7fa33b1245c14730da39e125714ad3a327901b`, file `Spark-X2.5-1.7B-Q4_K_M.gguf`, SHA256 `902bde2522394954ac17821b3e5fd0df02defbc6944f122253f2580acf0503f4`. Model and executable remain outside Git; the stage verified both hashes before loading.
|
||||
|
||||
The Linux CPU-only build omits the Windows llama.cpp `0/29 layers offloaded` log line. The first [attempt](placement_parser_refusal.json) therefore failed closed during Omni placement verification even though the [server log](placement_parser_refusal_server.log) showed only CPU buffers. The verifier now accepts this layout only when the server explicitly reports no usable GPU, records one or more `CPU*` model tensor buffers, records no accelerator model buffer or accelerator layer assignment, and was launched with `-dev none -ngl 0`. The successful [profile server log](profile20_server.log) identifies `CPU_Mapped` and `CPU_REPACK` model buffers, a CPU compute buffer and a CPU-only device list. The [profile plan](profile20_report.json) records `cpu_only_layout=true`, `offloaded_layers=null`, loaded worker RSS 1.91 GB, a 4 GiB host-RAM reservation and a one-slot server. This fixes an evidence-format mismatch without treating missing GPU data as CPU proof.
|
||||
|
||||
A post-change [native Windows CPU regression request](windows_cpu_parser_regression_report.json) also passed the two named prompts with the earlier b11124 executable. Its [server log](windows_cpu_parser_regression_server.log) retained the `0/29` offload line and CPU buffers; the plan recorded `cpu_only_layout=false`. This one-request check tests the alternate log layout, not Windows performance.
|
||||
|
||||
The profile used a 4,096-token context, 96-token generation cap, temperature zero, no prompt cache and serial concurrency one. Both named requests passed: `Paris` for the short question and `1511` for district 73 of the synthetic 120-item inventory. After one warmup, **20/20** measured inventory requests were correct; nearest-rank complete-request wall p50/p95 was **7.82/7.97 s** (range 7.71–8.01 s), excluding 1.61 s stage startup. The server reported 2,463 prompt and 5 completion tokens for the measured request. The prompt was checked before submission; an over-context request was refused without truncation and a following short request passed. In-flight cancellation terminated the worker, emitted no stale output and released the ledger without quarantine. The [public entrypoint report](public_report.json) separately returned `Paris` and a terminal text event using the real Spark HF config directory for pipeline resolution while executing the pinned GGUF. The two OS builds have different llama.cpp commits and environments, so these timings are not a paired WSL-versus-Windows comparison.
|
||||
|
||||
The reservation includes the 1.11 GB GGUF and a declared 2 GiB for KV/workspace/transfer/headroom; loaded RSS is a snapshot, not a loading peak. Power and thermal conditions were not sampled. The backend returns one complete text response and retires the worker on a live-request cancellation. Broader reference-quality prompts, long contexts, incremental streaming, restart after cancellation, concurrent admission and sustained power/thermal measurements remain open.
|
||||
|
||||
Reproduce from this checkout with [`probe_omni_llamacpp_spark.py`](../../../../experiments/probe_omni_llamacpp_spark.py) and [`probe_omni_llamacpp_entrypoint.py`](../../../../experiments/probe_omni_llamacpp_entrypoint.py). Set `PYTHONPATH` to this checkout, `LD_LIBRARY_PATH=/home/zhout/project/edge_infer/llama.cpp/build/bin` for this locally built shared-library server, `VLLM_ENABLE_V1_MULTIPROCESSING=0` and `CUDA_VISIBLE_DEVICES=''`. Use the exact model/server paths and hashes above, `--device cpu`, `--capacity-gib 12 --reserve-gib 4 --warmups 1 --repeats 20 --context-check --abort-check` for the profile. The [driver log](profile20_driver.log), [smoke report](smoke_report.json) and [initial refusal trace](placement_parser_refusal_driver.log) retain the execution and failure records.
|
||||
|
||||
Checked-in log copies have line endings and trailing whitespace normalized for review; the JSON reports retain per-request timing and event data.
|
||||
+27
@@ -0,0 +1,27 @@
|
||||
{
|
||||
"scope": "complete Spark GGUF text requests through Omni StageRuntime/StagePool and external llama.cpp backend",
|
||||
"model_file": "/home/zhout/project/edge_infer/models/Spark-X2.5-1.7B-GGUF/Spark-X2.5-1.7B-Q4_K_M.gguf",
|
||||
"device": "cpu",
|
||||
"memory_budget": {
|
||||
"capacities": {
|
||||
"host_ram": 12884901888
|
||||
},
|
||||
"demands": {
|
||||
"host_ram": 4294967296
|
||||
}
|
||||
},
|
||||
"warmup_count": 0,
|
||||
"measured_count": 1,
|
||||
"status": "failed",
|
||||
"error": "RuntimeError: CPU stage assigned model layers to an accelerator",
|
||||
"ledger_after_shutdown": {
|
||||
"capacities": {
|
||||
"host_ram": 12884901888
|
||||
},
|
||||
"reserved": {
|
||||
"host_ram": 0
|
||||
},
|
||||
"owners": [],
|
||||
"quarantined": []
|
||||
}
|
||||
}
|
||||
+84
@@ -0,0 +1,84 @@
|
||||
INFO 09-24 04:54:28 [patch.py:259] NVFP4 W4A4 weight_scale NaN-clamp: installed.
|
||||
INFO 09-24 04:54:28 [patch.py:510] inductor factorable-divisibility patch: installed.
|
||||
INFO 09-24 04:54:28 [patch.py:589] [cumem-cuda] CuMemAllocator._python_free_callback patched: asleep guard extended to all platforms.
|
||||
INFO 09-24 04:54:29 [stage_init_utils.py:768] [stage_init] Set VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
WARNING 09-24 04:54:29 [stage_init_utils.py:1887] [stage_init] Failed to load transfer config: [Errno 21] Is a directory: '/home/zhout/project/edge_infer/vllm-omni-edge'
|
||||
ERROR 09-24 04:54:31 [stage_runtime.py:265] [StageRuntime] Stage initialization failed; shutting down 0 initialized client(s)
|
||||
ERROR 09-24 04:54:31 [stage_runtime.py:265] Traceback (most recent call last):
|
||||
ERROR 09-24 04:54:31 [stage_runtime.py:265] File "/home/zhout/project/edge_infer/vllm-omni-edge/vllm_omni/engine/stage_runtime.py", line 252, in initialize
|
||||
ERROR 09-24 04:54:31 [stage_runtime.py:265] initialized_clients = self._initialize_stage_replicas(stage_plans, self._stage_init_timeout)
|
||||
ERROR 09-24 04:54:31 [stage_runtime.py:265] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
ERROR 09-24 04:54:31 [stage_runtime.py:265] File "/home/zhout/project/edge_infer/vllm-omni-edge/vllm_omni/engine/stage_runtime.py", line 589, in _initialize_stage_replicas
|
||||
ERROR 09-24 04:54:31 [stage_runtime.py:265] raise primary_exc
|
||||
ERROR 09-24 04:54:31 [stage_runtime.py:265] File "/home/zhout/project/edge_infer/vllm-omni-edge/vllm_omni/engine/stage_runtime.py", line 550, in _init_group
|
||||
ERROR 09-24 04:54:31 [stage_runtime.py:265] client = self._initialize_replica(
|
||||
ERROR 09-24 04:54:31 [stage_runtime.py:265] ^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
ERROR 09-24 04:54:31 [stage_runtime.py:265] File "/home/zhout/project/edge_infer/vllm-omni-edge/vllm_omni/engine/stage_runtime.py", line 817, in _initialize_replica
|
||||
ERROR 09-24 04:54:31 [stage_runtime.py:265] return create_graph_client(
|
||||
ERROR 09-24 04:54:31 [stage_runtime.py:265] ^^^^^^^^^^^^^^^^^^^^
|
||||
ERROR 09-24 04:54:31 [stage_runtime.py:265] File "/home/zhout/project/edge_infer/vllm-omni-edge/vllm_omni/engine/backends/__init__.py", line 15, in create_graph_client
|
||||
ERROR 09-24 04:54:31 [stage_runtime.py:265] return LlamaCppTextStageClient(metadata, backend, ledger, reservation)
|
||||
ERROR 09-24 04:54:31 [stage_runtime.py:265] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
ERROR 09-24 04:54:31 [stage_runtime.py:265] File "/home/zhout/project/edge_infer/vllm-omni-edge/vllm_omni/engine/backends/llamacpp.py", line 168, in __init__
|
||||
ERROR 09-24 04:54:31 [stage_runtime.py:265] raise RuntimeError("CPU stage assigned model layers to an accelerator")
|
||||
ERROR 09-24 04:54:31 [stage_runtime.py:265] RuntimeError: CPU stage assigned model layers to an accelerator
|
||||
{
|
||||
"scope": "complete Spark GGUF text requests through Omni StageRuntime/StagePool and external llama.cpp backend",
|
||||
"model_file": "/home/zhout/project/edge_infer/models/Spark-X2.5-1.7B-GGUF/Spark-X2.5-1.7B-Q4_K_M.gguf",
|
||||
"device": "cpu",
|
||||
"memory_budget": {
|
||||
"capacities": {
|
||||
"host_ram": 12884901888
|
||||
},
|
||||
"demands": {
|
||||
"host_ram": 4294967296
|
||||
}
|
||||
},
|
||||
"warmup_count": 0,
|
||||
"measured_count": 1,
|
||||
"status": "failed",
|
||||
"error": "RuntimeError: CPU stage assigned model layers to an accelerator",
|
||||
"ledger_after_shutdown": {
|
||||
"capacities": {
|
||||
"host_ram": 12884901888
|
||||
},
|
||||
"reserved": {
|
||||
"host_ram": 0
|
||||
},
|
||||
"owners": [],
|
||||
"quarantined": []
|
||||
}
|
||||
}
|
||||
Traceback (most recent call last):
|
||||
File "/home/zhout/project/edge_infer/vllm-omni-edge/benchmarks/edge_harness/experiments/probe_omni_llamacpp_spark.py", line 192, in <module>
|
||||
asyncio.run(main())
|
||||
File "/home/zhout/.local/share/uv/python/cpython-3.12.13-linux-x86_64-gnu/lib/python3.12/asyncio/runners.py", line 195, in run
|
||||
return runner.run(main)
|
||||
^^^^^^^^^^^^^^^^
|
||||
File "/home/zhout/.local/share/uv/python/cpython-3.12.13-linux-x86_64-gnu/lib/python3.12/asyncio/runners.py", line 118, in run
|
||||
return self._loop.run_until_complete(task)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/home/zhout/.local/share/uv/python/cpython-3.12.13-linux-x86_64-gnu/lib/python3.12/asyncio/base_events.py", line 691, in run_until_complete
|
||||
return future.result()
|
||||
^^^^^^^^^^^^^^^
|
||||
File "/home/zhout/project/edge_infer/vllm-omni-edge/benchmarks/edge_harness/experiments/probe_omni_llamacpp_spark.py", line 102, in main
|
||||
runtime.initialize()
|
||||
File "/home/zhout/project/edge_infer/vllm-omni-edge/vllm_omni/engine/stage_runtime.py", line 278, in initialize
|
||||
raise exc
|
||||
File "/home/zhout/project/edge_infer/vllm-omni-edge/vllm_omni/engine/stage_runtime.py", line 252, in initialize
|
||||
initialized_clients = self._initialize_stage_replicas(stage_plans, self._stage_init_timeout)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/home/zhout/project/edge_infer/vllm-omni-edge/vllm_omni/engine/stage_runtime.py", line 589, in _initialize_stage_replicas
|
||||
raise primary_exc
|
||||
File "/home/zhout/project/edge_infer/vllm-omni-edge/vllm_omni/engine/stage_runtime.py", line 550, in _init_group
|
||||
client = self._initialize_replica(
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/home/zhout/project/edge_infer/vllm-omni-edge/vllm_omni/engine/stage_runtime.py", line 817, in _initialize_replica
|
||||
return create_graph_client(
|
||||
^^^^^^^^^^^^^^^^^^^^
|
||||
File "/home/zhout/project/edge_infer/vllm-omni-edge/vllm_omni/engine/backends/__init__.py", line 15, in create_graph_client
|
||||
return LlamaCppTextStageClient(metadata, backend, ledger, reservation)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/home/zhout/project/edge_infer/vllm-omni-edge/vllm_omni/engine/backends/llamacpp.py", line 168, in __init__
|
||||
raise RuntimeError("CPU stage assigned model layers to an accelerator")
|
||||
RuntimeError: CPU stage assigned model layers to an accelerator
|
||||
+196
@@ -0,0 +1,196 @@
|
||||
warning: no usable GPU found, --gpu-layers option will be ignored
|
||||
warning: one possible reason is that llama.cpp was compiled without GPU support
|
||||
warning: consult docs/build.md for compilation instructions
|
||||
0.00.013.195 I cmn common_param: common_params_print_info: build 10849 (e71b80510) with GNU 11.4.0 for Linux x86_64
|
||||
0.00.013.222 I cmn common_param: common_params_print_info: verbosity = 4 (adjust with the `-lv N` CLI arg)
|
||||
0.00.013.223 I cmn common_param: device_info:
|
||||
0.00.013.245 I cmn common_param: - CPU : AMD Ryzen AI 9 HX 370 w/ Radeon 890M (31647 MiB, 31647 MiB free)
|
||||
0.00.013.347 I cmn common_param: system_info: n_threads = 12 (n_threads_batch = 12) / 24 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX_VNNI = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | AMX_INT8 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
|
||||
0.00.013.482 I srv init: using 23 threads for HTTP server
|
||||
0.00.013.500 I srv init: The UI is disabled
|
||||
0.00.013.501 I srv init: Use --ui/--no-ui (or deprecated --webui/--no-webui) to enable/disable
|
||||
0.00.013.567 W srv llama_server: -----------------
|
||||
0.00.013.585 W srv llama_server: CORS is set to allow all origins ('*') and no API key is set
|
||||
0.00.013.586 W srv llama_server: this can be a security risk (cross-origin attacks)
|
||||
0.00.013.586 W srv llama_server: more info: https://github.com/ggml-org/llama.cpp/pull/25655
|
||||
0.00.013.586 W srv llama_server: -----------------
|
||||
0.00.013.610 I srv start: binding port with default address family
|
||||
0.00.015.010 I srv load_model: loading model '/home/zhout/project/edge_infer/models/Spark-X2.5-1.7B-GGUF/Spark-X2.5-1.7B-Q4_K_M.gguf'
|
||||
0.00.015.032 I srv load_model: local path '/home/zhout/project/edge_infer/models/Spark-X2.5-1.7B-GGUF/Spark-X2.5-1.7B-Q4_K_M.gguf'
|
||||
0.00.041.091 I llama_model_loader: loaded meta data with 37 key-value pairs and 226 tensors from /home/zhout/project/edge_infer/models/Spark-X2.5-1.7B-GGUF/Spark-X2.5-1.7B-Q4_K_M.gguf (version GGUF V3 (latest))
|
||||
0.00.041.136 I llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
0.00.041.147 I llama_model_loader: - kv 0: general.architecture str = spark2_5
|
||||
0.00.041.148 I llama_model_loader: - kv 1: general.type str = model
|
||||
0.00.041.152 I llama_model_loader: - kv 2: general.sampling.top_k i32 = 0
|
||||
0.00.041.158 I llama_model_loader: - kv 3: general.sampling.top_p f32 = 0.950000
|
||||
0.00.041.159 I llama_model_loader: - kv 4: general.sampling.temp f32 = 1.000000
|
||||
0.00.041.159 I llama_model_loader: - kv 5: general.name str = Hf_Format
|
||||
0.00.041.160 I llama_model_loader: - kv 6: general.size_label str = 1.7B
|
||||
0.00.041.160 I llama_model_loader: - kv 7: spark2_5.block_count u32 = 28
|
||||
0.00.041.161 I llama_model_loader: - kv 8: spark2_5.context_length u32 = 1048576
|
||||
0.00.041.161 I llama_model_loader: - kv 9: spark2_5.embedding_length u32 = 2048
|
||||
0.00.041.162 I llama_model_loader: - kv 10: spark2_5.feed_forward_length u32 = 6656
|
||||
0.00.041.162 I llama_model_loader: - kv 11: spark2_5.attention.head_count u32 = 8
|
||||
0.00.041.162 I llama_model_loader: - kv 12: spark2_5.attention.head_count_kv u32 = 2
|
||||
0.00.041.164 I llama_model_loader: - kv 13: spark2_5.rope.freq_base f32 = 5000000.000000
|
||||
0.00.041.165 I llama_model_loader: - kv 14: spark2_5.rope.freq_base_swa f32 = 10000.000000
|
||||
0.00.041.166 I llama_model_loader: - kv 15: spark2_5.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
0.00.041.166 I llama_model_loader: - kv 16: spark2_5.attention.key_length u32 = 256
|
||||
0.00.041.166 I llama_model_loader: - kv 17: spark2_5.attention.value_length u32 = 256
|
||||
0.00.041.166 I llama_model_loader: - kv 18: spark2_5.vocab_size u32 = 131072
|
||||
0.00.041.167 I llama_model_loader: - kv 19: spark2_5.attention.sliding_window u32 = 512
|
||||
0.00.041.189 I llama_model_loader: - kv 20: spark2_5.attention.sliding_window_pattern arr[bool,28] = [true, true, true, false, true, true,...
|
||||
0.00.041.189 I llama_model_loader: - kv 21: spark2_5.rope.dimension_count u32 = 64
|
||||
0.00.041.189 I llama_model_loader: - kv 22: spark2_5.rope.dimension_count_swa u32 = 256
|
||||
0.00.041.189 I llama_model_loader: - kv 23: tokenizer.ggml.model str = gpt2
|
||||
0.00.041.190 I llama_model_loader: - kv 24: tokenizer.ggml.pre str = spark2_5
|
||||
0.00.050.050 I llama_model_loader: - kv 25: tokenizer.ggml.tokens arr[str,131072] = ["<｜startâ–�ofâ–�sentence｜>", "<ï...
|
||||
0.00.051.814 I llama_model_loader: - kv 26: tokenizer.ggml.token_type arr[i32,131072] = [3, 3, 3, 4, 4, 3, 3, 3, 4, 4, 4, 3, ...
|
||||
0.00.059.243 I llama_model_loader: - kv 27: tokenizer.ggml.merges arr[str,130716] = ["Ä Ä ", "Ä t", "i n", "Ä a", "Ä Ä...
|
||||
0.00.059.276 I llama_model_loader: - kv 28: tokenizer.ggml.bos_token_id u32 = 0
|
||||
0.00.059.277 I llama_model_loader: - kv 29: tokenizer.ggml.eos_token_id u32 = 1
|
||||
0.00.059.278 I llama_model_loader: - kv 30: tokenizer.ggml.unknown_token_id u32 = 5
|
||||
0.00.059.278 I llama_model_loader: - kv 31: tokenizer.ggml.padding_token_id u32 = 2
|
||||
0.00.059.279 I llama_model_loader: - kv 32: tokenizer.ggml.add_bos_token bool = false
|
||||
0.00.059.279 I llama_model_loader: - kv 33: tokenizer.ggml.add_eos_token bool = false
|
||||
0.00.059.283 I llama_model_loader: - kv 34: tokenizer.chat_template str = {%- if not messages %}\n {{- raise_...
|
||||
0.00.059.284 I llama_model_loader: - kv 35: general.quantization_version u32 = 2
|
||||
0.00.059.285 I llama_model_loader: - kv 36: general.file_type u32 = 15
|
||||
0.00.059.293 I llama_model_loader: - type f32: 57 tensors
|
||||
0.00.059.294 I llama_model_loader: - type q4_K: 140 tensors
|
||||
0.00.059.294 I llama_model_loader: - type q6_K: 29 tensors
|
||||
0.00.059.299 I print_info: file format = GGUF V3 (latest)
|
||||
0.00.059.302 I print_info: file type = Q4_K - Medium
|
||||
0.00.059.305 I print_info: file size = 1.03 GiB (5.16 BPW)
|
||||
0.00.161.869 I load: 86 unused tokens
|
||||
0.00.189.894 I load: printing all EOG tokens:
|
||||
0.00.189.924 I load: - 1 ('<｜end�of�sentence｜>')
|
||||
0.00.190.068 I load: special tokens cache size = 113
|
||||
0.00.215.365 I load: token to piece cache size = 0.7975 MB
|
||||
0.00.215.424 I print_info: arch = spark2_5
|
||||
0.00.215.425 I print_info: vocab_only = 0
|
||||
0.00.215.425 I print_info: no_alloc = 0
|
||||
0.00.215.426 I print_info: n_ctx_train = 1048576
|
||||
0.00.215.428 I print_info: n_embd_inp = 2048
|
||||
0.00.215.428 I print_info: n_embd = 2048
|
||||
0.00.215.430 I print_info: n_embd_out = 2048
|
||||
0.00.215.430 I print_info: n_layer = 28
|
||||
0.00.215.430 I print_info: n_layer_all = 28
|
||||
0.00.215.444 I print_info: n_head = 8
|
||||
0.00.215.447 I print_info: n_head_kv = 2
|
||||
0.00.215.447 I print_info: n_rot = 64
|
||||
0.00.215.447 I print_info: n_swa = 512
|
||||
0.00.215.448 I print_info: is_swa_any = 1
|
||||
0.00.215.449 I print_info: non_causal_type = 0
|
||||
0.00.215.449 I print_info: n_embd_head_k = 256
|
||||
0.00.215.449 I print_info: n_embd_head_v = 256
|
||||
0.00.215.451 I print_info: n_gqa = 4
|
||||
0.00.215.456 I print_info: n_embd_k_gqa = 512
|
||||
0.00.215.460 I print_info: n_embd_v_gqa = 512
|
||||
0.00.215.461 I print_info: f_norm_eps = 0.0e+00
|
||||
0.00.215.462 I print_info: f_norm_rms_eps = 1.0e-06
|
||||
0.00.215.462 I print_info: f_clamp_kqv = 0.0e+00
|
||||
0.00.215.462 I print_info: f_max_alibi_bias = 0.0e+00
|
||||
0.00.215.462 I print_info: f_logit_scale = 0.0e+00
|
||||
0.00.215.462 I print_info: f_attn_scale = 0.0e+00
|
||||
0.00.215.463 I print_info: f_attn_value_scale = 0.0000
|
||||
0.00.215.464 I print_info: n_ff = 6656
|
||||
0.00.215.464 I print_info: n_expert = 0
|
||||
0.00.215.465 I print_info: n_expert_used = 0
|
||||
0.00.215.465 I print_info: n_expert_groups = 0
|
||||
0.00.215.465 I print_info: n_group_used = 0
|
||||
0.00.215.465 I print_info: causal attn = 1
|
||||
0.00.215.465 I print_info: pooling type = -1
|
||||
0.00.215.465 I print_info: rope type = 2
|
||||
0.00.215.466 I print_info: rope scaling = linear
|
||||
0.00.215.467 I print_info: freq_base_train = 5000000.0
|
||||
0.00.215.467 I print_info: freq_scale_train = 1
|
||||
0.00.215.468 I print_info: freq_base_swa = 10000.0
|
||||
0.00.215.468 I print_info: freq_scale_swa = 1
|
||||
0.00.215.468 I print_info: n_embd_head_k_swa = 256
|
||||
0.00.215.468 I print_info: n_embd_head_v_swa = 256
|
||||
0.00.215.468 I print_info: n_rot_swa = 256
|
||||
0.00.215.468 I print_info: n_ctx_orig_yarn = 1048576
|
||||
0.00.215.468 I print_info: rope_yarn_log_mul = 0.0000
|
||||
0.00.215.469 I print_info: rope_finetuned = unknown
|
||||
0.00.215.470 I print_info: model type = 1.7B
|
||||
0.00.215.470 I print_info: model params = 1.71 B
|
||||
0.00.215.470 I print_info: general.name = Hf_Format
|
||||
0.00.215.475 I print_info: vocab type = BPE
|
||||
0.00.215.477 I print_info: n_vocab = 131072
|
||||
0.00.215.477 I print_info: n_merges = 130716
|
||||
0.00.215.478 I print_info: BOS token = 0 '<｜start�of�sentence｜>'
|
||||
0.00.215.478 I print_info: EOS token = 1 '<｜end�of�sentence｜>'
|
||||
0.00.215.479 I print_info: EOT token = 1 '<｜end�of�sentence｜>'
|
||||
0.00.215.479 I print_info: UNK token = 5 '<unk>'
|
||||
0.00.215.479 I print_info: PAD token = 2 '<｜�pad�｜>'
|
||||
0.00.215.480 I print_info: LF token = 198 'ÄŠ'
|
||||
0.00.215.480 I print_info: FIM PRE token = 11 '<｜fim�begin｜>'
|
||||
0.00.215.480 I print_info: FIM SUF token = 12 '<｜fim�hole｜>'
|
||||
0.00.215.481 I print_info: FIM MID token = 13 '<｜fim�end｜>'
|
||||
0.00.215.481 I print_info: EOG token = 1 '<｜end�of�sentence｜>'
|
||||
0.00.215.482 I print_info: max token length = 210
|
||||
0.00.215.485 I load_tensors: loading model tensors, this can take a while... (load_mode = mmap)
|
||||
AMX is not ready to be used!
|
||||
0.00.218.801 I load_tensors: CPU_Mapped model buffer size = 1043.71 MiB
|
||||
0.00.218.809 I load_tensors: CPU_REPACK model buffer size = 622.37 MiB
|
||||
0.00.876.942 I cmn common_init_: added <｜end�of�sentence｜> logit bias = -inf
|
||||
0.00.877.450 I llama_context: constructing llama_context
|
||||
0.00.877.473 I llama_context: n_seq_max = 1
|
||||
0.00.877.474 I llama_context: n_ctx = 4096
|
||||
0.00.877.474 I llama_context: n_ctx_seq = 4096
|
||||
0.00.877.474 I llama_context: n_batch = 2048
|
||||
0.00.877.475 I llama_context: n_ubatch = 512
|
||||
0.00.877.475 I llama_context: causal_attn = 1
|
||||
0.00.877.476 I llama_context: flash_attn = auto
|
||||
0.00.877.477 I llama_context: kv_unified = false
|
||||
0.00.877.480 I llama_context: freq_base = 5000000.0
|
||||
0.00.877.481 I llama_context: freq_scale = 1
|
||||
0.00.877.481 I llama_context: n_rs_seq = 0
|
||||
0.00.877.481 I llama_context: n_outputs_max = 1
|
||||
0.00.877.481 I llama_context: n_outputs_max_per_seq = 1
|
||||
0.00.877.482 I llama_context: n_ctx_seq (4096) < n_ctx_train (1048576) -- the full capacity of the model will not be utilized
|
||||
0.00.877.543 I llama_context: CPU output buffer size = 0.50 MiB
|
||||
0.00.877.575 I llama_kv_cache_iswa: creating non-SWA KV cache, size = 4096 cells
|
||||
0.00.877.654 I llama_kv_cache: CPU KV buffer size = 56.00 MiB
|
||||
0.00.894.348 I llama_kv_cache: size = 56.00 MiB ( 4096 cells, 7 layers, 1/1 seqs), K (f16): 28.00 MiB, V (f16): 28.00 MiB
|
||||
0.00.894.374 I llama_kv_cache: attn_rot_k = 0, n_embd_head_k_all = 256
|
||||
0.00.894.374 I llama_kv_cache: attn_rot_v = 0, n_embd_head_k_all = 256
|
||||
0.00.894.376 I llama_kv_cache_iswa: creating SWA KV cache, size = 1024 cells
|
||||
0.00.894.429 I llama_kv_cache: CPU KV buffer size = 42.00 MiB
|
||||
0.00.907.290 I llama_kv_cache: size = 42.00 MiB ( 1024 cells, 21 layers, 1/1 seqs), K (f16): 21.00 MiB, V (f16): 21.00 MiB
|
||||
0.00.907.313 I llama_kv_cache: attn_rot_k = 0, n_embd_head_k_all = 256
|
||||
0.00.907.313 I llama_kv_cache: attn_rot_v = 0, n_embd_head_k_all = 256
|
||||
0.00.907.337 I sched_reserve: reserving ...
|
||||
0.00.907.889 I resolve_fused_ops: Flash Attention enabled
|
||||
0.00.907.911 I resolve_fused_ops: resolving fused DeepSeek V4 HC support:
|
||||
0.00.908.068 I resolve_fused_ops: fused DeepSeek V4 HC pre enabled
|
||||
0.00.908.243 I resolve_fused_ops: fused DeepSeek V4 HC comb enabled
|
||||
0.00.908.399 I resolve_fused_ops: fused DeepSeek V4 HC post enabled
|
||||
0.00.908.972 I sched_reserve: CPU compute buffer size = 60.02 MiB
|
||||
0.00.908.994 I sched_reserve: graph nodes = 986
|
||||
0.00.908.994 I sched_reserve: graph splits = 1
|
||||
0.00.908.996 I sched_reserve: reserve took 1.65 ms, sched copies = 1
|
||||
0.00.909.035 I cmn init: llama threadpool init, n_threads = 12
|
||||
0.00.909.080 I cmn common_init_: warming up the model with an empty run - please wait ... (--no-warmup to disable)
|
||||
0.00.967.212 I srv load_model: initializing, n_slots = 1, n_ctx_slot = 4096, kv_unified = 'false'
|
||||
0.00.967.288 I spec common_specu: no implementations specified for speculative decoding
|
||||
0.00.967.294 I slot load_model: id 0 | task -1 | new slot, n_ctx = 4096
|
||||
0.00.967.379 I srv load_model: prompt cache is disabled - use `--cache-ram N` to enable it
|
||||
0.00.967.413 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
|
||||
0.00.967.414 I srv load_model: context checkpoints enabled, max = 32, min spacing = 8192
|
||||
0.00.967.439 W srv init: --cache-idle-slots requires --cache-ram, disabling
|
||||
0.00.976.714 I srv init: init: chat template, example_format: '<｜start�of�sentence｜><|System|>
|
||||
you are a helpful assistant.
|
||||
|
||||
You are a helpful assistant<｜end�of�sentence｜><｜start�of�sentence｜><|User|>Hello<｜end�of�sentence｜><｜start�of�sentence｜><|Bot|></think>Hi there<｜end�of�sentence｜><｜start�of�sentence｜><|User|>How are you?<｜end�of�sentence｜><｜start�of�sentence｜><|Bot|></think>'
|
||||
0.00.982.113 I srv init: init: chat template, thinking = 0
|
||||
0.00.982.144 I srv init: preserve_reasoning kwarg: true
|
||||
0.00.982.145 W srv init: chat template supports preserving reasoning, it is enabled by default (may use more tokens, disable via --no-reasoning-preserve)
|
||||
0.00.982.199 I srv llama_server: model loaded
|
||||
0.00.982.222 I srv llama_server: listening on http://127.0.0.1:42309
|
||||
0.00.982.352 I srv update_slots: all slots are idle
|
||||
0.01.005.536 I srv operator(): operator(): cleaning up before exit...
|
||||
0.01.006.628 I common_memory_breakdown_print: | memory breakdown [MiB] | total free self model context compute unaccounted |
|
||||
0.01.006.647 I common_memory_breakdown_print: | - Host | 1201 = 1043 + 98 + 60 |
|
||||
0.01.006.648 I common_memory_breakdown_print: | - CPU_REPACK | 622 = 622 + 0 + 0 |
|
||||
+172
@@ -0,0 +1,172 @@
|
||||
INFO 09-24 04:56:35 [patch.py:259] NVFP4 W4A4 weight_scale NaN-clamp: installed.
|
||||
INFO 09-24 04:56:35 [patch.py:510] inductor factorable-divisibility patch: installed.
|
||||
INFO 09-24 04:56:35 [patch.py:589] [cumem-cuda] CuMemAllocator._python_free_callback patched: asleep guard extended to all platforms.
|
||||
INFO 09-24 04:56:36 [stage_init_utils.py:768] [stage_init] Set VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
WARNING 09-24 04:56:36 [stage_init_utils.py:1887] [stage_init] Failed to load transfer config: [Errno 21] Is a directory: '/home/zhout/project/edge_infer/vllm-omni-edge'
|
||||
{
|
||||
"scope": "complete Spark GGUF text requests through Omni StageRuntime/StagePool and external llama.cpp backend",
|
||||
"model_file": "/home/zhout/project/edge_infer/models/Spark-X2.5-1.7B-GGUF/Spark-X2.5-1.7B-Q4_K_M.gguf",
|
||||
"device": "cpu",
|
||||
"memory_budget": {
|
||||
"capacities": {
|
||||
"host_ram": 12884901888
|
||||
},
|
||||
"demands": {
|
||||
"host_ram": 4294967296
|
||||
}
|
||||
},
|
||||
"warmup_count": 1,
|
||||
"measured_count": 20,
|
||||
"startup_s": 1.6053575539845042,
|
||||
"execution_plan": {
|
||||
"backend": "external.llamacpp.text.v1",
|
||||
"stage_id": 0,
|
||||
"worker_generation": "be69859f6a6147efbdc38a2c99cecbea",
|
||||
"model_sha256": "902bde2522394954ac17821b3e5fd0df02defbc6944f122253f2580acf0503f4",
|
||||
"server_sha256": "0d6bb6ade8da8f3231917ce23261da5331333df0007dc4bda74c00e0774e092c",
|
||||
"model_alias": "/home/zhout/project/edge_infer/models/Spark-X2.5-1.7B-GGUF/Spark-X2.5-1.7B-Q4_K_M.gguf",
|
||||
"requested_device": "cpu",
|
||||
"expected_device_name": null,
|
||||
"layer_assignments": 0,
|
||||
"offloaded_layers": null,
|
||||
"model_buffer_devices": [],
|
||||
"all_model_buffers": [
|
||||
"CPU_Mapped",
|
||||
"CPU_REPACK"
|
||||
],
|
||||
"cpu_only_layout": true,
|
||||
"worker_pid": 2121780,
|
||||
"loaded_rss_bytes": 1914826752,
|
||||
"reserved_bytes": {
|
||||
"host_ram": 4294967296
|
||||
},
|
||||
"memory_pool": "host_ram",
|
||||
"memory_overhead_bytes": 2147483648,
|
||||
"context_tokens": 4096,
|
||||
"max_new_tokens": 96,
|
||||
"max_io_bytes": 1048576,
|
||||
"request_capacity": 1,
|
||||
"stateful_session": "llama.cpp-owned; reset for each complete request",
|
||||
"evidence": "B"
|
||||
},
|
||||
"checks": [
|
||||
{
|
||||
"case": "short",
|
||||
"answer": "Paris",
|
||||
"expected": "Paris",
|
||||
"correct": true,
|
||||
"wall_s": 0.14942718797829002,
|
||||
"stage_event": {
|
||||
"request_id": "short-451018974904008",
|
||||
"stage_id": 0,
|
||||
"epoch": 1,
|
||||
"seq": 1,
|
||||
"kind": "text",
|
||||
"worker_generation": "be69859f6a6147efbdc38a2c99cecbea",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"llamacpp_wall_s": 0.1457545809680596,
|
||||
"usage": {
|
||||
"completion_tokens": 2,
|
||||
"prompt_tokens": 28,
|
||||
"total_tokens": 30,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"case": "inventory_120",
|
||||
"answer": "1511",
|
||||
"expected": "1511",
|
||||
"correct": true,
|
||||
"wall_s": 7.643903894000687,
|
||||
"stage_event": {
|
||||
"request_id": "inventory_120-451019124457275",
|
||||
"stage_id": 0,
|
||||
"epoch": 2,
|
||||
"seq": 1,
|
||||
"kind": "text",
|
||||
"worker_generation": "be69859f6a6147efbdc38a2c99cecbea",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"llamacpp_wall_s": 7.639271399995778,
|
||||
"usage": {
|
||||
"completion_tokens": 5,
|
||||
"prompt_tokens": 2463,
|
||||
"total_tokens": 2468,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"nearest_rank_p50_wall_s": 7.820265209011268,
|
||||
"nearest_rank_p95_wall_s": 7.972199406009167,
|
||||
"context_check": {
|
||||
"rejected": true,
|
||||
"error": "ResourceUnavailable: Spark prompt plus generation bound exceeds context; refusing truncation",
|
||||
"recovery": {
|
||||
"case": "short",
|
||||
"answer": "Paris",
|
||||
"expected": "Paris",
|
||||
"correct": true,
|
||||
"wall_s": 0.14603868400445208,
|
||||
"stage_event": {
|
||||
"request_id": "short-451191246405564",
|
||||
"stage_id": 0,
|
||||
"epoch": 24,
|
||||
"seq": 1,
|
||||
"kind": "text",
|
||||
"worker_generation": "be69859f6a6147efbdc38a2c99cecbea",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"llamacpp_wall_s": 0.14326041197637096,
|
||||
"usage": {
|
||||
"completion_tokens": 2,
|
||||
"prompt_tokens": 28,
|
||||
"total_tokens": 30,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"abort_check": {
|
||||
"stale_output": false,
|
||||
"worker_exited": true,
|
||||
"ledger_after_abort": {
|
||||
"capacities": {
|
||||
"host_ram": 12884901888
|
||||
},
|
||||
"reserved": {
|
||||
"host_ram": 0
|
||||
},
|
||||
"owners": [],
|
||||
"quarantined": []
|
||||
}
|
||||
},
|
||||
"status": "passed",
|
||||
"ledger_after_shutdown": {
|
||||
"capacities": {
|
||||
"host_ram": 12884901888
|
||||
},
|
||||
"reserved": {
|
||||
"host_ram": 0
|
||||
},
|
||||
"owners": [],
|
||||
"quarantined": []
|
||||
}
|
||||
}
|
||||
+780
@@ -0,0 +1,780 @@
|
||||
{
|
||||
"scope": "complete Spark GGUF text requests through Omni StageRuntime/StagePool and external llama.cpp backend",
|
||||
"model_file": "/home/zhout/project/edge_infer/models/Spark-X2.5-1.7B-GGUF/Spark-X2.5-1.7B-Q4_K_M.gguf",
|
||||
"device": "cpu",
|
||||
"memory_budget": {
|
||||
"capacities": {
|
||||
"host_ram": 12884901888
|
||||
},
|
||||
"demands": {
|
||||
"host_ram": 4294967296
|
||||
}
|
||||
},
|
||||
"warmup_count": 1,
|
||||
"measured_count": 20,
|
||||
"startup_s": 1.6053575539845042,
|
||||
"execution_plan": {
|
||||
"backend": "external.llamacpp.text.v1",
|
||||
"stage_id": 0,
|
||||
"worker_generation": "be69859f6a6147efbdc38a2c99cecbea",
|
||||
"model_sha256": "902bde2522394954ac17821b3e5fd0df02defbc6944f122253f2580acf0503f4",
|
||||
"server_sha256": "0d6bb6ade8da8f3231917ce23261da5331333df0007dc4bda74c00e0774e092c",
|
||||
"model_alias": "/home/zhout/project/edge_infer/models/Spark-X2.5-1.7B-GGUF/Spark-X2.5-1.7B-Q4_K_M.gguf",
|
||||
"requested_device": "cpu",
|
||||
"expected_device_name": null,
|
||||
"layer_assignments": 0,
|
||||
"offloaded_layers": null,
|
||||
"model_buffer_devices": [],
|
||||
"all_model_buffers": [
|
||||
"CPU_Mapped",
|
||||
"CPU_REPACK"
|
||||
],
|
||||
"cpu_only_layout": true,
|
||||
"worker_pid": 2121780,
|
||||
"loaded_rss_bytes": 1914826752,
|
||||
"reserved_bytes": {
|
||||
"host_ram": 4294967296
|
||||
},
|
||||
"memory_pool": "host_ram",
|
||||
"memory_overhead_bytes": 2147483648,
|
||||
"context_tokens": 4096,
|
||||
"max_new_tokens": 96,
|
||||
"max_io_bytes": 1048576,
|
||||
"request_capacity": 1,
|
||||
"stateful_session": "llama.cpp-owned; reset for each complete request",
|
||||
"evidence": "B"
|
||||
},
|
||||
"checks": [
|
||||
{
|
||||
"case": "short",
|
||||
"answer": "Paris",
|
||||
"expected": "Paris",
|
||||
"correct": true,
|
||||
"wall_s": 0.14942718797829002,
|
||||
"stage_event": {
|
||||
"request_id": "short-451018974904008",
|
||||
"stage_id": 0,
|
||||
"epoch": 1,
|
||||
"seq": 1,
|
||||
"kind": "text",
|
||||
"worker_generation": "be69859f6a6147efbdc38a2c99cecbea",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"llamacpp_wall_s": 0.1457545809680596,
|
||||
"usage": {
|
||||
"completion_tokens": 2,
|
||||
"prompt_tokens": 28,
|
||||
"total_tokens": 30,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"case": "inventory_120",
|
||||
"answer": "1511",
|
||||
"expected": "1511",
|
||||
"correct": true,
|
||||
"wall_s": 7.643903894000687,
|
||||
"stage_event": {
|
||||
"request_id": "inventory_120-451019124457275",
|
||||
"stage_id": 0,
|
||||
"epoch": 2,
|
||||
"seq": 1,
|
||||
"kind": "text",
|
||||
"worker_generation": "be69859f6a6147efbdc38a2c99cecbea",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"llamacpp_wall_s": 7.639271399995778,
|
||||
"usage": {
|
||||
"completion_tokens": 5,
|
||||
"prompt_tokens": 2463,
|
||||
"total_tokens": 2468,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"warmups": [
|
||||
{
|
||||
"case": "inventory_120",
|
||||
"answer": "1511",
|
||||
"expected": "1511",
|
||||
"correct": true,
|
||||
"wall_s": 7.71934891102137,
|
||||
"stage_event": {
|
||||
"request_id": "inventory_120-451026768408600",
|
||||
"stage_id": 0,
|
||||
"epoch": 3,
|
||||
"seq": 1,
|
||||
"kind": "text",
|
||||
"worker_generation": "be69859f6a6147efbdc38a2c99cecbea",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"llamacpp_wall_s": 7.711946224968415,
|
||||
"usage": {
|
||||
"completion_tokens": 5,
|
||||
"prompt_tokens": 2463,
|
||||
"total_tokens": 2468,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"measured": [
|
||||
{
|
||||
"case": "inventory_120",
|
||||
"answer": "1511",
|
||||
"expected": "1511",
|
||||
"correct": true,
|
||||
"wall_s": 7.74671556497924,
|
||||
"stage_event": {
|
||||
"request_id": "inventory_120-451034487807135",
|
||||
"stage_id": 0,
|
||||
"epoch": 4,
|
||||
"seq": 1,
|
||||
"kind": "text",
|
||||
"worker_generation": "be69859f6a6147efbdc38a2c99cecbea",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"llamacpp_wall_s": 7.739578359003644,
|
||||
"usage": {
|
||||
"completion_tokens": 5,
|
||||
"prompt_tokens": 2463,
|
||||
"total_tokens": 2468,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"case": "inventory_120",
|
||||
"answer": "1511",
|
||||
"expected": "1511",
|
||||
"correct": true,
|
||||
"wall_s": 7.707643722009379,
|
||||
"stage_event": {
|
||||
"request_id": "inventory_120-451042234586250",
|
||||
"stage_id": 0,
|
||||
"epoch": 5,
|
||||
"seq": 1,
|
||||
"kind": "text",
|
||||
"worker_generation": "be69859f6a6147efbdc38a2c99cecbea",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"llamacpp_wall_s": 7.70190589199774,
|
||||
"usage": {
|
||||
"completion_tokens": 5,
|
||||
"prompt_tokens": 2463,
|
||||
"total_tokens": 2468,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"case": "inventory_120",
|
||||
"answer": "1511",
|
||||
"expected": "1511",
|
||||
"correct": true,
|
||||
"wall_s": 7.7380492180236615,
|
||||
"stage_event": {
|
||||
"request_id": "inventory_120-451049942277453",
|
||||
"stage_id": 0,
|
||||
"epoch": 6,
|
||||
"seq": 1,
|
||||
"kind": "text",
|
||||
"worker_generation": "be69859f6a6147efbdc38a2c99cecbea",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"llamacpp_wall_s": 7.7311933530145325,
|
||||
"usage": {
|
||||
"completion_tokens": 5,
|
||||
"prompt_tokens": 2463,
|
||||
"total_tokens": 2468,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"case": "inventory_120",
|
||||
"answer": "1511",
|
||||
"expected": "1511",
|
||||
"correct": true,
|
||||
"wall_s": 7.774297729018144,
|
||||
"stage_event": {
|
||||
"request_id": "inventory_120-451057680401342",
|
||||
"stage_id": 0,
|
||||
"epoch": 7,
|
||||
"seq": 1,
|
||||
"kind": "text",
|
||||
"worker_generation": "be69859f6a6147efbdc38a2c99cecbea",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"llamacpp_wall_s": 7.768605509016197,
|
||||
"usage": {
|
||||
"completion_tokens": 5,
|
||||
"prompt_tokens": 2463,
|
||||
"total_tokens": 2468,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"case": "inventory_120",
|
||||
"answer": "1511",
|
||||
"expected": "1511",
|
||||
"correct": true,
|
||||
"wall_s": 7.7508464099955745,
|
||||
"stage_event": {
|
||||
"request_id": "inventory_120-451065454740650",
|
||||
"stage_id": 0,
|
||||
"epoch": 8,
|
||||
"seq": 1,
|
||||
"kind": "text",
|
||||
"worker_generation": "be69859f6a6147efbdc38a2c99cecbea",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"llamacpp_wall_s": 7.745904560026247,
|
||||
"usage": {
|
||||
"completion_tokens": 5,
|
||||
"prompt_tokens": 2463,
|
||||
"total_tokens": 2468,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"case": "inventory_120",
|
||||
"answer": "1511",
|
||||
"expected": "1511",
|
||||
"correct": true,
|
||||
"wall_s": 7.887554449029267,
|
||||
"stage_event": {
|
||||
"request_id": "inventory_120-451073205636945",
|
||||
"stage_id": 0,
|
||||
"epoch": 9,
|
||||
"seq": 1,
|
||||
"kind": "text",
|
||||
"worker_generation": "be69859f6a6147efbdc38a2c99cecbea",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"llamacpp_wall_s": 7.8811626880196854,
|
||||
"usage": {
|
||||
"completion_tokens": 5,
|
||||
"prompt_tokens": 2463,
|
||||
"total_tokens": 2468,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"case": "inventory_120",
|
||||
"answer": "1511",
|
||||
"expected": "1511",
|
||||
"correct": true,
|
||||
"wall_s": 7.887620234978385,
|
||||
"stage_event": {
|
||||
"request_id": "inventory_120-451081093238043",
|
||||
"stage_id": 0,
|
||||
"epoch": 10,
|
||||
"seq": 1,
|
||||
"kind": "text",
|
||||
"worker_generation": "be69859f6a6147efbdc38a2c99cecbea",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"llamacpp_wall_s": 7.880689698038623,
|
||||
"usage": {
|
||||
"completion_tokens": 5,
|
||||
"prompt_tokens": 2463,
|
||||
"total_tokens": 2468,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"case": "inventory_120",
|
||||
"answer": "1511",
|
||||
"expected": "1511",
|
||||
"correct": true,
|
||||
"wall_s": 7.795277512981556,
|
||||
"stage_event": {
|
||||
"request_id": "inventory_120-451088980916479",
|
||||
"stage_id": 0,
|
||||
"epoch": 11,
|
||||
"seq": 1,
|
||||
"kind": "text",
|
||||
"worker_generation": "be69859f6a6147efbdc38a2c99cecbea",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"llamacpp_wall_s": 7.7886199240456335,
|
||||
"usage": {
|
||||
"completion_tokens": 5,
|
||||
"prompt_tokens": 2463,
|
||||
"total_tokens": 2468,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"case": "inventory_120",
|
||||
"answer": "1511",
|
||||
"expected": "1511",
|
||||
"correct": true,
|
||||
"wall_s": 7.820265209011268,
|
||||
"stage_event": {
|
||||
"request_id": "inventory_120-451096776286098",
|
||||
"stage_id": 0,
|
||||
"epoch": 12,
|
||||
"seq": 1,
|
||||
"kind": "text",
|
||||
"worker_generation": "be69859f6a6147efbdc38a2c99cecbea",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"llamacpp_wall_s": 7.814440910995472,
|
||||
"usage": {
|
||||
"completion_tokens": 5,
|
||||
"prompt_tokens": 2463,
|
||||
"total_tokens": 2468,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"case": "inventory_120",
|
||||
"answer": "1511",
|
||||
"expected": "1511",
|
||||
"correct": true,
|
||||
"wall_s": 7.905293180025183,
|
||||
"stage_event": {
|
||||
"request_id": "inventory_120-451104596603887",
|
||||
"stage_id": 0,
|
||||
"epoch": 13,
|
||||
"seq": 1,
|
||||
"kind": "text",
|
||||
"worker_generation": "be69859f6a6147efbdc38a2c99cecbea",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"llamacpp_wall_s": 7.899314830021467,
|
||||
"usage": {
|
||||
"completion_tokens": 5,
|
||||
"prompt_tokens": 2463,
|
||||
"total_tokens": 2468,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"case": "inventory_120",
|
||||
"answer": "1511",
|
||||
"expected": "1511",
|
||||
"correct": true,
|
||||
"wall_s": 7.922793950943742,
|
||||
"stage_event": {
|
||||
"request_id": "inventory_120-451112501939908",
|
||||
"stage_id": 0,
|
||||
"epoch": 14,
|
||||
"seq": 1,
|
||||
"kind": "text",
|
||||
"worker_generation": "be69859f6a6147efbdc38a2c99cecbea",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"llamacpp_wall_s": 7.918431612022687,
|
||||
"usage": {
|
||||
"completion_tokens": 5,
|
||||
"prompt_tokens": 2463,
|
||||
"total_tokens": 2468,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"case": "inventory_120",
|
||||
"answer": "1511",
|
||||
"expected": "1511",
|
||||
"correct": true,
|
||||
"wall_s": 7.972199406009167,
|
||||
"stage_event": {
|
||||
"request_id": "inventory_120-451120424781239",
|
||||
"stage_id": 0,
|
||||
"epoch": 15,
|
||||
"seq": 1,
|
||||
"kind": "text",
|
||||
"worker_generation": "be69859f6a6147efbdc38a2c99cecbea",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"llamacpp_wall_s": 7.967701479967218,
|
||||
"usage": {
|
||||
"completion_tokens": 5,
|
||||
"prompt_tokens": 2463,
|
||||
"total_tokens": 2468,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"case": "inventory_120",
|
||||
"answer": "1511",
|
||||
"expected": "1511",
|
||||
"correct": true,
|
||||
"wall_s": 7.970679979014676,
|
||||
"stage_event": {
|
||||
"request_id": "inventory_120-451128397018036",
|
||||
"stage_id": 0,
|
||||
"epoch": 16,
|
||||
"seq": 1,
|
||||
"kind": "text",
|
||||
"worker_generation": "be69859f6a6147efbdc38a2c99cecbea",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"llamacpp_wall_s": 7.965425668982789,
|
||||
"usage": {
|
||||
"completion_tokens": 5,
|
||||
"prompt_tokens": 2463,
|
||||
"total_tokens": 2468,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"case": "inventory_120",
|
||||
"answer": "1511",
|
||||
"expected": "1511",
|
||||
"correct": true,
|
||||
"wall_s": 8.014964771980885,
|
||||
"stage_event": {
|
||||
"request_id": "inventory_120-451136367749893",
|
||||
"stage_id": 0,
|
||||
"epoch": 17,
|
||||
"seq": 1,
|
||||
"kind": "text",
|
||||
"worker_generation": "be69859f6a6147efbdc38a2c99cecbea",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"llamacpp_wall_s": 8.0075953300111,
|
||||
"usage": {
|
||||
"completion_tokens": 5,
|
||||
"prompt_tokens": 2463,
|
||||
"total_tokens": 2468,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"case": "inventory_120",
|
||||
"answer": "1511",
|
||||
"expected": "1511",
|
||||
"correct": true,
|
||||
"wall_s": 7.900069575989619,
|
||||
"stage_event": {
|
||||
"request_id": "inventory_120-451144382775611",
|
||||
"stage_id": 0,
|
||||
"epoch": 18,
|
||||
"seq": 1,
|
||||
"kind": "text",
|
||||
"worker_generation": "be69859f6a6147efbdc38a2c99cecbea",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"llamacpp_wall_s": 7.895157499995548,
|
||||
"usage": {
|
||||
"completion_tokens": 5,
|
||||
"prompt_tokens": 2463,
|
||||
"total_tokens": 2468,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"case": "inventory_120",
|
||||
"answer": "1511",
|
||||
"expected": "1511",
|
||||
"correct": true,
|
||||
"wall_s": 7.82481507101329,
|
||||
"stage_event": {
|
||||
"request_id": "inventory_120-451152282893469",
|
||||
"stage_id": 0,
|
||||
"epoch": 19,
|
||||
"seq": 1,
|
||||
"kind": "text",
|
||||
"worker_generation": "be69859f6a6147efbdc38a2c99cecbea",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"llamacpp_wall_s": 7.819121725042351,
|
||||
"usage": {
|
||||
"completion_tokens": 5,
|
||||
"prompt_tokens": 2463,
|
||||
"total_tokens": 2468,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"case": "inventory_120",
|
||||
"answer": "1511",
|
||||
"expected": "1511",
|
||||
"correct": true,
|
||||
"wall_s": 7.775801228999626,
|
||||
"stage_event": {
|
||||
"request_id": "inventory_120-451160107752354",
|
||||
"stage_id": 0,
|
||||
"epoch": 20,
|
||||
"seq": 1,
|
||||
"kind": "text",
|
||||
"worker_generation": "be69859f6a6147efbdc38a2c99cecbea",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"llamacpp_wall_s": 7.7705729030421935,
|
||||
"usage": {
|
||||
"completion_tokens": 5,
|
||||
"prompt_tokens": 2463,
|
||||
"total_tokens": 2468,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"case": "inventory_120",
|
||||
"answer": "1511",
|
||||
"expected": "1511",
|
||||
"correct": true,
|
||||
"wall_s": 7.782076460018288,
|
||||
"stage_event": {
|
||||
"request_id": "inventory_120-451167883594700",
|
||||
"stage_id": 0,
|
||||
"epoch": 21,
|
||||
"seq": 1,
|
||||
"kind": "text",
|
||||
"worker_generation": "be69859f6a6147efbdc38a2c99cecbea",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"llamacpp_wall_s": 7.775419605022762,
|
||||
"usage": {
|
||||
"completion_tokens": 5,
|
||||
"prompt_tokens": 2463,
|
||||
"total_tokens": 2468,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"case": "inventory_120",
|
||||
"answer": "1511",
|
||||
"expected": "1511",
|
||||
"correct": true,
|
||||
"wall_s": 7.839946982974652,
|
||||
"stage_event": {
|
||||
"request_id": "inventory_120-451175665748026",
|
||||
"stage_id": 0,
|
||||
"epoch": 22,
|
||||
"seq": 1,
|
||||
"kind": "text",
|
||||
"worker_generation": "be69859f6a6147efbdc38a2c99cecbea",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"llamacpp_wall_s": 7.835599619022105,
|
||||
"usage": {
|
||||
"completion_tokens": 5,
|
||||
"prompt_tokens": 2463,
|
||||
"total_tokens": 2468,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"case": "inventory_120",
|
||||
"answer": "1511",
|
||||
"expected": "1511",
|
||||
"correct": true,
|
||||
"wall_s": 7.722867636010051,
|
||||
"stage_event": {
|
||||
"request_id": "inventory_120-451183505730155",
|
||||
"stage_id": 0,
|
||||
"epoch": 23,
|
||||
"seq": 1,
|
||||
"kind": "text",
|
||||
"worker_generation": "be69859f6a6147efbdc38a2c99cecbea",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"llamacpp_wall_s": 7.717733217985369,
|
||||
"usage": {
|
||||
"completion_tokens": 5,
|
||||
"prompt_tokens": 2463,
|
||||
"total_tokens": 2468,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"nearest_rank_p50_wall_s": 7.820265209011268,
|
||||
"nearest_rank_p95_wall_s": 7.972199406009167,
|
||||
"context_check": {
|
||||
"rejected": true,
|
||||
"error": "ResourceUnavailable: Spark prompt plus generation bound exceeds context; refusing truncation",
|
||||
"recovery": {
|
||||
"case": "short",
|
||||
"answer": "Paris",
|
||||
"expected": "Paris",
|
||||
"correct": true,
|
||||
"wall_s": 0.14603868400445208,
|
||||
"stage_event": {
|
||||
"request_id": "short-451191246405564",
|
||||
"stage_id": 0,
|
||||
"epoch": 24,
|
||||
"seq": 1,
|
||||
"kind": "text",
|
||||
"worker_generation": "be69859f6a6147efbdc38a2c99cecbea",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"llamacpp_wall_s": 0.14326041197637096,
|
||||
"usage": {
|
||||
"completion_tokens": 2,
|
||||
"prompt_tokens": 28,
|
||||
"total_tokens": 30,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"abort_check": {
|
||||
"stale_output": false,
|
||||
"worker_exited": true,
|
||||
"ledger_after_abort": {
|
||||
"capacities": {
|
||||
"host_ram": 12884901888
|
||||
},
|
||||
"reserved": {
|
||||
"host_ram": 0
|
||||
},
|
||||
"owners": [],
|
||||
"quarantined": []
|
||||
}
|
||||
},
|
||||
"status": "passed",
|
||||
"ledger_after_shutdown": {
|
||||
"capacities": {
|
||||
"host_ram": 12884901888
|
||||
},
|
||||
"reserved": {
|
||||
"host_ram": 0
|
||||
},
|
||||
"owners": [],
|
||||
"quarantined": []
|
||||
}
|
||||
}
|
||||
+940
@@ -0,0 +1,940 @@
|
||||
warning: no usable GPU found, --gpu-layers option will be ignored
|
||||
warning: one possible reason is that llama.cpp was compiled without GPU support
|
||||
warning: consult docs/build.md for compilation instructions
|
||||
0.00.002.623 I cmn common_param: common_params_print_info: build 10849 (e71b80510) with GNU 11.4.0 for Linux x86_64
|
||||
0.00.002.644 I cmn common_param: common_params_print_info: verbosity = 4 (adjust with the `-lv N` CLI arg)
|
||||
0.00.002.645 I cmn common_param: device_info:
|
||||
0.00.002.657 I cmn common_param: - CPU : AMD Ryzen AI 9 HX 370 w/ Radeon 890M (31647 MiB, 31647 MiB free)
|
||||
0.00.002.717 I cmn common_param: system_info: n_threads = 12 (n_threads_batch = 12) / 24 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX_VNNI = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | AMX_INT8 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
|
||||
0.00.002.822 I srv init: using 23 threads for HTTP server
|
||||
0.00.002.841 I srv init: The UI is disabled
|
||||
0.00.002.841 I srv init: Use --ui/--no-ui (or deprecated --webui/--no-webui) to enable/disable
|
||||
0.00.002.897 W srv llama_server: -----------------
|
||||
0.00.002.915 W srv llama_server: CORS is set to allow all origins ('*') and no API key is set
|
||||
0.00.002.915 W srv llama_server: this can be a security risk (cross-origin attacks)
|
||||
0.00.002.915 W srv llama_server: more info: https://github.com/ggml-org/llama.cpp/pull/25655
|
||||
0.00.002.915 W srv llama_server: -----------------
|
||||
0.00.002.935 I srv start: binding port with default address family
|
||||
0.00.004.417 I srv load_model: loading model '/home/zhout/project/edge_infer/models/Spark-X2.5-1.7B-GGUF/Spark-X2.5-1.7B-Q4_K_M.gguf'
|
||||
0.00.004.476 I srv load_model: local path '/home/zhout/project/edge_infer/models/Spark-X2.5-1.7B-GGUF/Spark-X2.5-1.7B-Q4_K_M.gguf'
|
||||
0.00.032.268 I llama_model_loader: loaded meta data with 37 key-value pairs and 226 tensors from /home/zhout/project/edge_infer/models/Spark-X2.5-1.7B-GGUF/Spark-X2.5-1.7B-Q4_K_M.gguf (version GGUF V3 (latest))
|
||||
0.00.032.312 I llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
0.00.032.323 I llama_model_loader: - kv 0: general.architecture str = spark2_5
|
||||
0.00.032.324 I llama_model_loader: - kv 1: general.type str = model
|
||||
0.00.032.328 I llama_model_loader: - kv 2: general.sampling.top_k i32 = 0
|
||||
0.00.032.334 I llama_model_loader: - kv 3: general.sampling.top_p f32 = 0.950000
|
||||
0.00.032.335 I llama_model_loader: - kv 4: general.sampling.temp f32 = 1.000000
|
||||
0.00.032.335 I llama_model_loader: - kv 5: general.name str = Hf_Format
|
||||
0.00.032.336 I llama_model_loader: - kv 6: general.size_label str = 1.7B
|
||||
0.00.032.337 I llama_model_loader: - kv 7: spark2_5.block_count u32 = 28
|
||||
0.00.032.337 I llama_model_loader: - kv 8: spark2_5.context_length u32 = 1048576
|
||||
0.00.032.338 I llama_model_loader: - kv 9: spark2_5.embedding_length u32 = 2048
|
||||
0.00.032.338 I llama_model_loader: - kv 10: spark2_5.feed_forward_length u32 = 6656
|
||||
0.00.032.338 I llama_model_loader: - kv 11: spark2_5.attention.head_count u32 = 8
|
||||
0.00.032.339 I llama_model_loader: - kv 12: spark2_5.attention.head_count_kv u32 = 2
|
||||
0.00.032.340 I llama_model_loader: - kv 13: spark2_5.rope.freq_base f32 = 5000000.000000
|
||||
0.00.032.340 I llama_model_loader: - kv 14: spark2_5.rope.freq_base_swa f32 = 10000.000000
|
||||
0.00.032.341 I llama_model_loader: - kv 15: spark2_5.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
0.00.032.341 I llama_model_loader: - kv 16: spark2_5.attention.key_length u32 = 256
|
||||
0.00.032.342 I llama_model_loader: - kv 17: spark2_5.attention.value_length u32 = 256
|
||||
0.00.032.342 I llama_model_loader: - kv 18: spark2_5.vocab_size u32 = 131072
|
||||
0.00.032.342 I llama_model_loader: - kv 19: spark2_5.attention.sliding_window u32 = 512
|
||||
0.00.032.364 I llama_model_loader: - kv 20: spark2_5.attention.sliding_window_pattern arr[bool,28] = [true, true, true, false, true, true,...
|
||||
0.00.032.364 I llama_model_loader: - kv 21: spark2_5.rope.dimension_count u32 = 64
|
||||
0.00.032.364 I llama_model_loader: - kv 22: spark2_5.rope.dimension_count_swa u32 = 256
|
||||
0.00.032.364 I llama_model_loader: - kv 23: tokenizer.ggml.model str = gpt2
|
||||
0.00.032.365 I llama_model_loader: - kv 24: tokenizer.ggml.pre str = spark2_5
|
||||
0.00.040.970 I llama_model_loader: - kv 25: tokenizer.ggml.tokens arr[str,131072] = ["<｜startâ–�ofâ–�sentence｜>", "<ï...
|
||||
0.00.042.769 I llama_model_loader: - kv 26: tokenizer.ggml.token_type arr[i32,131072] = [3, 3, 3, 4, 4, 3, 3, 3, 4, 4, 4, 3, ...
|
||||
0.00.050.040 I llama_model_loader: - kv 27: tokenizer.ggml.merges arr[str,130716] = ["Ä Ä ", "Ä t", "i n", "Ä a", "Ä Ä...
|
||||
0.00.050.073 I llama_model_loader: - kv 28: tokenizer.ggml.bos_token_id u32 = 0
|
||||
0.00.050.074 I llama_model_loader: - kv 29: tokenizer.ggml.eos_token_id u32 = 1
|
||||
0.00.050.074 I llama_model_loader: - kv 30: tokenizer.ggml.unknown_token_id u32 = 5
|
||||
0.00.050.075 I llama_model_loader: - kv 31: tokenizer.ggml.padding_token_id u32 = 2
|
||||
0.00.050.076 I llama_model_loader: - kv 32: tokenizer.ggml.add_bos_token bool = false
|
||||
0.00.050.076 I llama_model_loader: - kv 33: tokenizer.ggml.add_eos_token bool = false
|
||||
0.00.050.080 I llama_model_loader: - kv 34: tokenizer.chat_template str = {%- if not messages %}\n {{- raise_...
|
||||
0.00.050.080 I llama_model_loader: - kv 35: general.quantization_version u32 = 2
|
||||
0.00.050.081 I llama_model_loader: - kv 36: general.file_type u32 = 15
|
||||
0.00.050.086 I llama_model_loader: - type f32: 57 tensors
|
||||
0.00.050.087 I llama_model_loader: - type q4_K: 140 tensors
|
||||
0.00.050.087 I llama_model_loader: - type q6_K: 29 tensors
|
||||
0.00.050.091 I print_info: file format = GGUF V3 (latest)
|
||||
0.00.050.093 I print_info: file type = Q4_K - Medium
|
||||
0.00.050.098 I print_info: file size = 1.03 GiB (5.16 BPW)
|
||||
0.00.140.166 I load: 86 unused tokens
|
||||
0.00.168.733 I load: printing all EOG tokens:
|
||||
0.00.168.767 I load: - 1 ('<｜end�of�sentence｜>')
|
||||
0.00.168.909 I load: special tokens cache size = 113
|
||||
0.00.193.925 I load: token to piece cache size = 0.7975 MB
|
||||
0.00.193.980 I print_info: arch = spark2_5
|
||||
0.00.193.981 I print_info: vocab_only = 0
|
||||
0.00.193.981 I print_info: no_alloc = 0
|
||||
0.00.193.982 I print_info: n_ctx_train = 1048576
|
||||
AMX is not ready to be used!
|
||||
0.00.193.984 I print_info: n_embd_inp = 2048
|
||||
0.00.193.984 I print_info: n_embd = 2048
|
||||
0.00.193.985 I print_info: n_embd_out = 2048
|
||||
0.00.193.986 I print_info: n_layer = 28
|
||||
0.00.193.986 I print_info: n_layer_all = 28
|
||||
0.00.194.002 I print_info: n_head = 8
|
||||
0.00.194.004 I print_info: n_head_kv = 2
|
||||
0.00.194.004 I print_info: n_rot = 64
|
||||
0.00.194.004 I print_info: n_swa = 512
|
||||
0.00.194.005 I print_info: is_swa_any = 1
|
||||
0.00.194.007 I print_info: non_causal_type = 0
|
||||
0.00.194.007 I print_info: n_embd_head_k = 256
|
||||
0.00.194.008 I print_info: n_embd_head_v = 256
|
||||
0.00.194.009 I print_info: n_gqa = 4
|
||||
0.00.194.014 I print_info: n_embd_k_gqa = 512
|
||||
0.00.194.018 I print_info: n_embd_v_gqa = 512
|
||||
0.00.194.018 I print_info: f_norm_eps = 0.0e+00
|
||||
0.00.194.019 I print_info: f_norm_rms_eps = 1.0e-06
|
||||
0.00.194.019 I print_info: f_clamp_kqv = 0.0e+00
|
||||
0.00.194.020 I print_info: f_max_alibi_bias = 0.0e+00
|
||||
0.00.194.020 I print_info: f_logit_scale = 0.0e+00
|
||||
0.00.194.020 I print_info: f_attn_scale = 0.0e+00
|
||||
0.00.194.020 I print_info: f_attn_value_scale = 0.0000
|
||||
0.00.194.022 I print_info: n_ff = 6656
|
||||
0.00.194.022 I print_info: n_expert = 0
|
||||
0.00.194.022 I print_info: n_expert_used = 0
|
||||
0.00.194.023 I print_info: n_expert_groups = 0
|
||||
0.00.194.023 I print_info: n_group_used = 0
|
||||
0.00.194.023 I print_info: causal attn = 1
|
||||
0.00.194.023 I print_info: pooling type = -1
|
||||
0.00.194.023 I print_info: rope type = 2
|
||||
0.00.194.023 I print_info: rope scaling = linear
|
||||
0.00.194.024 I print_info: freq_base_train = 5000000.0
|
||||
0.00.194.025 I print_info: freq_scale_train = 1
|
||||
0.00.194.025 I print_info: freq_base_swa = 10000.0
|
||||
0.00.194.025 I print_info: freq_scale_swa = 1
|
||||
0.00.194.025 I print_info: n_embd_head_k_swa = 256
|
||||
0.00.194.025 I print_info: n_embd_head_v_swa = 256
|
||||
0.00.194.025 I print_info: n_rot_swa = 256
|
||||
0.00.194.026 I print_info: n_ctx_orig_yarn = 1048576
|
||||
0.00.194.026 I print_info: rope_yarn_log_mul = 0.0000
|
||||
0.00.194.026 I print_info: rope_finetuned = unknown
|
||||
0.00.194.027 I print_info: model type = 1.7B
|
||||
0.00.194.027 I print_info: model params = 1.71 B
|
||||
0.00.194.027 I print_info: general.name = Hf_Format
|
||||
0.00.194.032 I print_info: vocab type = BPE
|
||||
0.00.194.033 I print_info: n_vocab = 131072
|
||||
0.00.194.033 I print_info: n_merges = 130716
|
||||
0.00.194.035 I print_info: BOS token = 0 '<｜start�of�sentence｜>'
|
||||
0.00.194.035 I print_info: EOS token = 1 '<｜end�of�sentence｜>'
|
||||
0.00.194.035 I print_info: EOT token = 1 '<｜end�of�sentence｜>'
|
||||
0.00.194.035 I print_info: UNK token = 5 '<unk>'
|
||||
0.00.194.036 I print_info: PAD token = 2 '<｜�pad�｜>'
|
||||
0.00.194.036 I print_info: LF token = 198 'ÄŠ'
|
||||
0.00.194.036 I print_info: FIM PRE token = 11 '<｜fim�begin｜>'
|
||||
0.00.194.037 I print_info: FIM SUF token = 12 '<｜fim�hole｜>'
|
||||
0.00.194.037 I print_info: FIM MID token = 13 '<｜fim�end｜>'
|
||||
0.00.194.037 I print_info: EOG token = 1 '<｜end�of�sentence｜>'
|
||||
0.00.194.038 I print_info: max token length = 210
|
||||
0.00.194.041 I load_tensors: loading model tensors, this can take a while... (load_mode = mmap)
|
||||
0.00.198.187 I load_tensors: CPU_Mapped model buffer size = 1043.71 MiB
|
||||
0.00.198.208 I load_tensors: CPU_REPACK model buffer size = 622.37 MiB
|
||||
0.00.725.650 I cmn common_init_: added <｜end�of�sentence｜> logit bias = -inf
|
||||
0.00.726.171 I llama_context: constructing llama_context
|
||||
0.00.726.194 I llama_context: n_seq_max = 1
|
||||
0.00.726.194 I llama_context: n_ctx = 4096
|
||||
0.00.726.194 I llama_context: n_ctx_seq = 4096
|
||||
0.00.726.195 I llama_context: n_batch = 2048
|
||||
0.00.726.195 I llama_context: n_ubatch = 512
|
||||
0.00.726.195 I llama_context: causal_attn = 1
|
||||
0.00.726.196 I llama_context: flash_attn = auto
|
||||
0.00.726.197 I llama_context: kv_unified = false
|
||||
0.00.726.200 I llama_context: freq_base = 5000000.0
|
||||
0.00.726.201 I llama_context: freq_scale = 1
|
||||
0.00.726.201 I llama_context: n_rs_seq = 0
|
||||
0.00.726.201 I llama_context: n_outputs_max = 1
|
||||
0.00.726.201 I llama_context: n_outputs_max_per_seq = 1
|
||||
0.00.726.202 I llama_context: n_ctx_seq (4096) < n_ctx_train (1048576) -- the full capacity of the model will not be utilized
|
||||
0.00.726.281 I llama_context: CPU output buffer size = 0.50 MiB
|
||||
0.00.726.294 I llama_kv_cache_iswa: creating non-SWA KV cache, size = 4096 cells
|
||||
0.00.726.368 I llama_kv_cache: CPU KV buffer size = 56.00 MiB
|
||||
0.00.742.353 I llama_kv_cache: size = 56.00 MiB ( 4096 cells, 7 layers, 1/1 seqs), K (f16): 28.00 MiB, V (f16): 28.00 MiB
|
||||
0.00.742.382 I llama_kv_cache: attn_rot_k = 0, n_embd_head_k_all = 256
|
||||
0.00.742.383 I llama_kv_cache: attn_rot_v = 0, n_embd_head_k_all = 256
|
||||
0.00.742.386 I llama_kv_cache_iswa: creating SWA KV cache, size = 1024 cells
|
||||
0.00.742.468 I llama_kv_cache: CPU KV buffer size = 42.00 MiB
|
||||
0.00.754.624 I llama_kv_cache: size = 42.00 MiB ( 1024 cells, 21 layers, 1/1 seqs), K (f16): 21.00 MiB, V (f16): 21.00 MiB
|
||||
0.00.754.651 I llama_kv_cache: attn_rot_k = 0, n_embd_head_k_all = 256
|
||||
0.00.754.652 I llama_kv_cache: attn_rot_v = 0, n_embd_head_k_all = 256
|
||||
0.00.754.680 I sched_reserve: reserving ...
|
||||
0.00.755.214 I resolve_fused_ops: Flash Attention enabled
|
||||
0.00.755.236 I resolve_fused_ops: resolving fused DeepSeek V4 HC support:
|
||||
0.00.755.371 I resolve_fused_ops: fused DeepSeek V4 HC pre enabled
|
||||
0.00.755.524 I resolve_fused_ops: fused DeepSeek V4 HC comb enabled
|
||||
0.00.755.676 I resolve_fused_ops: fused DeepSeek V4 HC post enabled
|
||||
0.00.756.297 I sched_reserve: CPU compute buffer size = 60.02 MiB
|
||||
0.00.756.319 I sched_reserve: graph nodes = 986
|
||||
0.00.756.319 I sched_reserve: graph splits = 1
|
||||
0.00.756.321 I sched_reserve: reserve took 1.64 ms, sched copies = 1
|
||||
0.00.756.351 I cmn init: llama threadpool init, n_threads = 12
|
||||
0.00.756.396 I cmn common_init_: warming up the model with an empty run - please wait ... (--no-warmup to disable)
|
||||
0.00.816.978 I srv load_model: initializing, n_slots = 1, n_ctx_slot = 4096, kv_unified = 'false'
|
||||
0.00.817.056 I spec common_specu: no implementations specified for speculative decoding
|
||||
0.00.817.060 I slot load_model: id 0 | task -1 | new slot, n_ctx = 4096
|
||||
0.00.817.138 I srv load_model: prompt cache is disabled - use `--cache-ram N` to enable it
|
||||
0.00.817.148 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
|
||||
0.00.817.148 I srv load_model: context checkpoints enabled, max = 32, min spacing = 8192
|
||||
0.00.817.186 W srv init: --cache-idle-slots requires --cache-ram, disabling
|
||||
0.00.825.915 I srv init: init: chat template, example_format: '<｜start�of�sentence｜><|System|>
|
||||
you are a helpful assistant.
|
||||
|
||||
You are a helpful assistant<｜end�of�sentence｜><｜start�of�sentence｜><|User|>Hello<｜end�of�sentence｜><｜start�of�sentence｜><|Bot|></think>Hi there<｜end�of�sentence｜><｜start�of�sentence｜><|User|>How are you?<｜end�of�sentence｜><｜start�of�sentence｜><|Bot|></think>'
|
||||
0.00.831.361 I srv init: init: chat template, thinking = 0
|
||||
0.00.831.389 I srv init: preserve_reasoning kwarg: true
|
||||
0.00.831.389 W srv init: chat template supports preserving reasoning, it is enabled by default (may use more tokens, disable via --no-reasoning-preserve)
|
||||
0.00.831.443 I srv llama_server: model loaded
|
||||
0.00.831.448 I srv llama_server: listening on http://127.0.0.1:50681
|
||||
0.00.831.635 I srv update_slots: all slots are idle
|
||||
0.01.025.634 I srv server_strea: conv_id= (empty=1)
|
||||
0.01.025.945 I srv operator(): chat format: peg-native
|
||||
0.01.026.164 I slot get_availabl: id 0 | task -1 | - skipping, slot is empty
|
||||
0.01.026.401 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = -1
|
||||
0.01.026.473 I slot launch_slot_: id 0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> ?top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
|
||||
0.01.026.548 I slot launch_slot_: id 0 | task -1 | sampler params:
|
||||
repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
|
||||
dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 64
|
||||
top_k = 0, top_p = 0.950, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.000
|
||||
mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000, adaptive_target = -1.000, adaptive_decay = 0.900
|
||||
0.01.026.551 I slot launch_slot_: id 0 | task 0 | processing task, is_child = 0
|
||||
0.01.026.571 I slot operator(): id 0 | task 0 | new prompt, n_ctx_slot = 4096, n_keep = 0, task.n_tokens = 28
|
||||
0.01.026.575 I slot operator(): id 0 | task 0 | cached n_tokens = 0, memory_seq_rm [0, end)
|
||||
0.01.065.341 I slot operator(): id 0 | task 0 | cached n_tokens = 10, memory_seq_rm [10, end)
|
||||
0.01.065.501 I slot create_check: id 0 | task 0 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 9, n_tokens = 10, size = 0.411 MiB)
|
||||
0.01.116.801 I slot operator(): id 0 | task 0 | cached n_tokens = 24, memory_seq_rm [24, end)
|
||||
0.01.116.893 I slot init_sampler: id 0 | task 0 | init sampler, took 0.02 ms, tokens: text = 28, total = 28
|
||||
0.01.117.832 I slot create_check: id 0 | task 0 | created context checkpoint 2 of 32 (pos_min = 0, pos_max = 23, n_tokens = 24, size = 0.985 MiB)
|
||||
0.01.164.128 I slot print_timing: id 0 | task 0 | prompt eval time = 116.29 ms / 28 tokens ( 4.15 ms per token, 240.77 tokens per second)
|
||||
0.01.164.168 I slot print_timing: id 0 | task 0 | eval time = 21.23 ms / 2 tokens ( 21.23 ms per token, 47.10 tokens per second)
|
||||
0.01.164.169 I slot print_timing: id 0 | task 0 | total time = 137.52 ms / 30 tokens
|
||||
0.01.164.177 I slot print_timing: id 0 | task 0 | graphs reused = 1
|
||||
0.01.164.254 I slot release: id 0 | task 0 | stop processing: n_tokens = 29, truncated = 0
|
||||
0.01.164.293 I srv update_slots: all slots are idle
|
||||
0.01.177.670 I srv server_strea: conv_id= (empty=1)
|
||||
0.01.179.719 I srv operator(): chat format: peg-native
|
||||
0.01.179.879 I slot get_availabl: id 0 | task -1 | - checking sim = 0.005 (12/2463) > 0.100
|
||||
0.01.179.900 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 451019122065
|
||||
0.01.179.944 I slot launch_slot_: id 0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> ?top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
|
||||
0.01.179.968 I slot launch_slot_: id 0 | task -1 | sampler params:
|
||||
repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
|
||||
dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 64
|
||||
top_k = 0, top_p = 0.950, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.000
|
||||
mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000, adaptive_target = -1.000, adaptive_decay = 0.900
|
||||
0.01.179.969 I slot launch_slot_: id 0 | task 5 | processing task, is_child = 0
|
||||
0.01.179.975 I slot operator(): id 0 | task 5 | new prompt, n_ctx_slot = 4096, n_keep = 0, task.n_tokens = 2463
|
||||
0.01.179.977 I slot operator(): id 0 | task 5 | erased invalidated context checkpoint (pos_min = 0, pos_max = 9, n_tokens = 10, n_swa = 512, pos_next = 0, size = 0.411 MiB)
|
||||
0.01.179.977 I slot operator(): id 0 | task 5 | erased invalidated context checkpoint (pos_min = 0, pos_max = 23, n_tokens = 24, n_swa = 512, pos_next = 0, size = 0.985 MiB)
|
||||
0.01.179.978 I slot operator(): id 0 | task 5 | cached n_tokens = 0, memory_seq_rm [0, end)
|
||||
0.01.218.966 I slot operator(): id 0 | task 5 | cached n_tokens = 10, memory_seq_rm [10, end)
|
||||
0.01.219.104 I slot create_check: id 0 | task 5 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 9, n_tokens = 10, size = 0.411 MiB)
|
||||
0.07.034.844 I slot print_timing: id 0 | task 5 | prompt processing, n_tokens = 1947, progress = 0.79, t = 5.85 s / 332.55 tokens per second
|
||||
0.07.034.883 I slot operator(): id 0 | task 5 | cached n_tokens = 1947, memory_seq_rm [1947, end)
|
||||
0.07.045.018 I slot create_check: id 0 | task 5 | created context checkpoint 2 of 32 (pos_min = 923, pos_max = 1946, n_tokens = 1947, size = 21.006 MiB)
|
||||
0.08.661.489 I slot print_timing: id 0 | task 5 | prompt processing, n_tokens = 2459, progress = 1.00, t = 7.48 s / 328.68 tokens per second
|
||||
0.08.661.524 I slot operator(): id 0 | task 5 | cached n_tokens = 2459, memory_seq_rm [2459, end)
|
||||
0.08.661.864 I slot init_sampler: id 0 | task 5 | init sampler, took 0.31 ms, tokens: text = 2463, total = 2463
|
||||
0.08.673.369 I slot create_check: id 0 | task 5 | created context checkpoint 3 of 32 (pos_min = 1435, pos_max = 2458, n_tokens = 2459, size = 21.006 MiB)
|
||||
0.08.809.337 I slot print_timing: id 0 | task 5 | prompt eval time = 7531.07 ms / 2463 tokens ( 3.06 ms per token, 327.04 tokens per second)
|
||||
0.08.809.370 I slot print_timing: id 0 | task 5 | eval time = 98.26 ms / 5 tokens ( 24.56 ms per token, 40.71 tokens per second)
|
||||
0.08.809.371 I slot print_timing: id 0 | task 5 | total time = 7629.33 ms / 2468 tokens
|
||||
0.08.809.373 I slot print_timing: id 0 | task 5 | graphs reused = 4
|
||||
0.08.809.471 I slot release: id 0 | task 5 | stop processing: n_tokens = 2467, truncated = 0
|
||||
0.08.809.491 I srv update_slots: all slots are idle
|
||||
0.08.823.684 I srv server_strea: conv_id= (empty=1)
|
||||
0.08.825.844 I srv operator(): chat format: peg-native
|
||||
0.08.826.036 I slot get_availabl: id 0 | task -1 | - checking sim = 1.000 (2463/2463) > 0.100
|
||||
0.08.826.096 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, f_sim_best = 1.000 (> 0.100 thold), f_keep = 0.998
|
||||
0.08.826.229 I slot launch_slot_: id 0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> ?top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
|
||||
0.08.826.247 I slot launch_slot_: id 0 | task -1 | sampler params:
|
||||
repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
|
||||
dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 64
|
||||
top_k = 0, top_p = 0.950, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.000
|
||||
mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000, adaptive_target = -1.000, adaptive_decay = 0.900
|
||||
0.08.826.249 I slot launch_slot_: id 0 | task 14 | processing task, is_child = 0
|
||||
0.08.826.254 I slot operator(): id 0 | task 14 | new prompt, n_ctx_slot = 4096, n_keep = 0, task.n_tokens = 2463
|
||||
0.08.826.256 I slot operator(): id 0 | task 14 | erased invalidated context checkpoint (pos_min = 0, pos_max = 9, n_tokens = 10, n_swa = 512, pos_next = 0, size = 0.411 MiB)
|
||||
0.08.826.258 I slot operator(): id 0 | task 14 | erased invalidated context checkpoint (pos_min = 923, pos_max = 1946, n_tokens = 1947, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
0.08.827.577 I slot operator(): id 0 | task 14 | erased invalidated context checkpoint (pos_min = 1435, pos_max = 2458, n_tokens = 2459, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
0.08.829.500 I slot operator(): id 0 | task 14 | cached n_tokens = 0, memory_seq_rm [0, end)
|
||||
0.08.869.255 I slot operator(): id 0 | task 14 | cached n_tokens = 10, memory_seq_rm [10, end)
|
||||
0.08.869.397 I slot create_check: id 0 | task 14 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 9, n_tokens = 10, size = 0.411 MiB)
|
||||
0.14.746.681 I slot print_timing: id 0 | task 14 | prompt processing, n_tokens = 1947, progress = 0.79, t = 5.92 s / 328.86 tokens per second
|
||||
0.14.746.735 I slot operator(): id 0 | task 14 | cached n_tokens = 1947, memory_seq_rm [1947, end)
|
||||
0.14.756.381 I slot create_check: id 0 | task 14 | created context checkpoint 2 of 32 (pos_min = 923, pos_max = 1946, n_tokens = 1947, size = 21.006 MiB)
|
||||
0.16.387.250 I slot print_timing: id 0 | task 14 | prompt processing, n_tokens = 2459, progress = 1.00, t = 7.56 s / 325.22 tokens per second
|
||||
0.16.387.288 I slot operator(): id 0 | task 14 | cached n_tokens = 2459, memory_seq_rm [2459, end)
|
||||
0.16.387.699 I slot init_sampler: id 0 | task 14 | init sampler, took 0.38 ms, tokens: text = 2463, total = 2463
|
||||
0.16.397.242 I slot create_check: id 0 | task 14 | created context checkpoint 3 of 32 (pos_min = 1435, pos_max = 2458, n_tokens = 2459, size = 21.006 MiB)
|
||||
0.16.527.635 I slot print_timing: id 0 | task 14 | prompt eval time = 7603.94 ms / 2463 tokens ( 3.09 ms per token, 323.91 tokens per second)
|
||||
0.16.527.676 I slot print_timing: id 0 | task 14 | eval time = 97.41 ms / 5 tokens ( 24.35 ms per token, 41.06 tokens per second)
|
||||
0.16.527.677 I slot print_timing: id 0 | task 14 | total time = 7701.35 ms / 2468 tokens
|
||||
0.16.527.679 I slot print_timing: id 0 | task 14 | graphs reused = 7
|
||||
0.16.527.769 I slot release: id 0 | task 14 | stop processing: n_tokens = 2467, truncated = 0
|
||||
0.16.527.838 I srv update_slots: all slots are idle
|
||||
0.16.541.831 I srv server_strea: conv_id= (empty=1)
|
||||
0.16.544.039 I srv operator(): chat format: peg-native
|
||||
0.16.544.199 I slot get_availabl: id 0 | task -1 | - checking sim = 1.000 (2463/2463) > 0.100
|
||||
0.16.544.219 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, f_sim_best = 1.000 (> 0.100 thold), f_keep = 0.998
|
||||
0.16.544.259 I slot launch_slot_: id 0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> ?top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
|
||||
0.16.544.283 I slot launch_slot_: id 0 | task -1 | sampler params:
|
||||
repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
|
||||
dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 64
|
||||
top_k = 0, top_p = 0.950, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.000
|
||||
mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000, adaptive_target = -1.000, adaptive_decay = 0.900
|
||||
0.16.544.284 I slot launch_slot_: id 0 | task 23 | processing task, is_child = 0
|
||||
0.16.544.290 I slot operator(): id 0 | task 23 | new prompt, n_ctx_slot = 4096, n_keep = 0, task.n_tokens = 2463
|
||||
0.16.544.292 I slot operator(): id 0 | task 23 | erased invalidated context checkpoint (pos_min = 0, pos_max = 9, n_tokens = 10, n_swa = 512, pos_next = 0, size = 0.411 MiB)
|
||||
0.16.544.293 I slot operator(): id 0 | task 23 | erased invalidated context checkpoint (pos_min = 923, pos_max = 1946, n_tokens = 1947, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
0.16.544.295 I slot operator(): id 0 | task 23 | erased invalidated context checkpoint (pos_min = 1435, pos_max = 2458, n_tokens = 2459, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
0.16.546.917 I slot operator(): id 0 | task 23 | cached n_tokens = 0, memory_seq_rm [0, end)
|
||||
0.16.584.635 I slot operator(): id 0 | task 23 | cached n_tokens = 10, memory_seq_rm [10, end)
|
||||
0.16.584.773 I slot create_check: id 0 | task 23 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 9, n_tokens = 10, size = 0.411 MiB)
|
||||
0.22.471.099 I slot print_timing: id 0 | task 23 | prompt processing, n_tokens = 1947, progress = 0.79, t = 5.93 s / 328.51 tokens per second
|
||||
0.22.471.139 I slot operator(): id 0 | task 23 | cached n_tokens = 1947, memory_seq_rm [1947, end)
|
||||
0.22.482.197 I slot create_check: id 0 | task 23 | created context checkpoint 2 of 32 (pos_min = 923, pos_max = 1946, n_tokens = 1947, size = 21.006 MiB)
|
||||
0.24.123.964 I slot print_timing: id 0 | task 23 | prompt processing, n_tokens = 2459, progress = 1.00, t = 7.58 s / 324.42 tokens per second
|
||||
0.24.124.022 I slot operator(): id 0 | task 23 | cached n_tokens = 2459, memory_seq_rm [2459, end)
|
||||
0.24.124.575 I slot init_sampler: id 0 | task 23 | init sampler, took 0.52 ms, tokens: text = 2463, total = 2463
|
||||
0.24.135.154 I slot create_check: id 0 | task 23 | created context checkpoint 3 of 32 (pos_min = 1435, pos_max = 2458, n_tokens = 2459, size = 21.006 MiB)
|
||||
0.24.273.218 I slot print_timing: id 0 | task 23 | prompt eval time = 7624.95 ms / 2463 tokens ( 3.10 ms per token, 323.02 tokens per second)
|
||||
0.24.273.266 I slot print_timing: id 0 | task 23 | eval time = 103.95 ms / 5 tokens ( 25.99 ms per token, 38.48 tokens per second)
|
||||
0.24.273.267 I slot print_timing: id 0 | task 23 | total time = 7728.90 ms / 2468 tokens
|
||||
0.24.273.269 I slot print_timing: id 0 | task 23 | graphs reused = 10
|
||||
0.24.273.376 I slot release: id 0 | task 23 | stop processing: n_tokens = 2467, truncated = 0
|
||||
0.24.273.425 I srv update_slots: all slots are idle
|
||||
0.24.288.027 I srv server_strea: conv_id= (empty=1)
|
||||
0.24.290.016 I srv operator(): chat format: peg-native
|
||||
0.24.290.171 I slot get_availabl: id 0 | task -1 | - checking sim = 1.000 (2463/2463) > 0.100
|
||||
0.24.290.191 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, f_sim_best = 1.000 (> 0.100 thold), f_keep = 0.998
|
||||
0.24.290.255 I slot launch_slot_: id 0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> ?top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
|
||||
0.24.290.279 I slot launch_slot_: id 0 | task -1 | sampler params:
|
||||
repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
|
||||
dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 64
|
||||
top_k = 0, top_p = 0.950, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.000
|
||||
mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000, adaptive_target = -1.000, adaptive_decay = 0.900
|
||||
0.24.290.282 I slot launch_slot_: id 0 | task 32 | processing task, is_child = 0
|
||||
0.24.290.293 I slot operator(): id 0 | task 32 | new prompt, n_ctx_slot = 4096, n_keep = 0, task.n_tokens = 2463
|
||||
0.24.290.297 I slot operator(): id 0 | task 32 | erased invalidated context checkpoint (pos_min = 0, pos_max = 9, n_tokens = 10, n_swa = 512, pos_next = 0, size = 0.411 MiB)
|
||||
0.24.290.301 I slot operator(): id 0 | task 32 | erased invalidated context checkpoint (pos_min = 923, pos_max = 1946, n_tokens = 1947, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
0.24.290.303 I slot operator(): id 0 | task 32 | erased invalidated context checkpoint (pos_min = 1435, pos_max = 2458, n_tokens = 2459, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
0.24.293.666 I slot operator(): id 0 | task 32 | cached n_tokens = 0, memory_seq_rm [0, end)
|
||||
0.24.334.535 I slot operator(): id 0 | task 32 | cached n_tokens = 10, memory_seq_rm [10, end)
|
||||
0.24.334.680 I slot create_check: id 0 | task 32 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 9, n_tokens = 10, size = 0.411 MiB)
|
||||
0.30.182.108 I slot print_timing: id 0 | task 32 | prompt processing, n_tokens = 1947, progress = 0.79, t = 5.89 s / 330.46 tokens per second
|
||||
0.30.182.136 I slot operator(): id 0 | task 32 | cached n_tokens = 1947, memory_seq_rm [1947, end)
|
||||
0.30.190.712 I slot create_check: id 0 | task 32 | created context checkpoint 2 of 32 (pos_min = 923, pos_max = 1946, n_tokens = 1947, size = 21.006 MiB)
|
||||
0.31.837.737 I slot print_timing: id 0 | task 32 | prompt processing, n_tokens = 2459, progress = 1.00, t = 7.55 s / 325.81 tokens per second
|
||||
0.31.837.790 I slot operator(): id 0 | task 32 | cached n_tokens = 2459, memory_seq_rm [2459, end)
|
||||
0.31.838.404 I slot init_sampler: id 0 | task 32 | init sampler, took 0.58 ms, tokens: text = 2463, total = 2463
|
||||
0.31.849.496 I slot create_check: id 0 | task 32 | created context checkpoint 3 of 32 (pos_min = 1435, pos_max = 2458, n_tokens = 2459, size = 21.006 MiB)
|
||||
0.31.982.129 I slot print_timing: id 0 | task 32 | prompt eval time = 7594.26 ms / 2463 tokens ( 3.08 ms per token, 324.32 tokens per second)
|
||||
0.31.982.176 I slot print_timing: id 0 | task 32 | eval time = 97.55 ms / 5 tokens ( 24.39 ms per token, 41.01 tokens per second)
|
||||
0.31.982.177 I slot print_timing: id 0 | task 32 | total time = 7691.81 ms / 2468 tokens
|
||||
0.31.982.179 I slot print_timing: id 0 | task 32 | graphs reused = 13
|
||||
0.31.982.317 I slot release: id 0 | task 32 | stop processing: n_tokens = 2467, truncated = 0
|
||||
0.31.982.363 I srv update_slots: all slots are idle
|
||||
0.31.996.055 I srv server_strea: conv_id= (empty=1)
|
||||
0.31.998.099 I srv operator(): chat format: peg-native
|
||||
0.31.998.366 I slot get_availabl: id 0 | task -1 | - checking sim = 1.000 (2463/2463) > 0.100
|
||||
0.31.998.388 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, f_sim_best = 1.000 (> 0.100 thold), f_keep = 0.998
|
||||
0.31.998.424 I slot launch_slot_: id 0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> ?top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
|
||||
0.31.998.428 I slot launch_slot_: id 0 | task -1 | sampler params:
|
||||
repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
|
||||
dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 64
|
||||
top_k = 0, top_p = 0.950, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.000
|
||||
mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000, adaptive_target = -1.000, adaptive_decay = 0.900
|
||||
0.31.998.429 I slot launch_slot_: id 0 | task 41 | processing task, is_child = 0
|
||||
0.31.998.435 I slot operator(): id 0 | task 41 | new prompt, n_ctx_slot = 4096, n_keep = 0, task.n_tokens = 2463
|
||||
0.31.998.436 I slot operator(): id 0 | task 41 | erased invalidated context checkpoint (pos_min = 0, pos_max = 9, n_tokens = 10, n_swa = 512, pos_next = 0, size = 0.411 MiB)
|
||||
0.31.998.438 I slot operator(): id 0 | task 41 | erased invalidated context checkpoint (pos_min = 923, pos_max = 1946, n_tokens = 1947, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
0.31.998.440 I slot operator(): id 0 | task 41 | erased invalidated context checkpoint (pos_min = 1435, pos_max = 2458, n_tokens = 2459, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
0.32.001.239 I slot operator(): id 0 | task 41 | cached n_tokens = 0, memory_seq_rm [0, end)
|
||||
0.32.038.466 I slot operator(): id 0 | task 41 | cached n_tokens = 10, memory_seq_rm [10, end)
|
||||
0.32.038.607 I slot create_check: id 0 | task 41 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 9, n_tokens = 10, size = 0.411 MiB)
|
||||
0.37.904.913 I slot print_timing: id 0 | task 41 | prompt processing, n_tokens = 1947, progress = 0.79, t = 5.91 s / 329.64 tokens per second
|
||||
0.37.904.966 I slot operator(): id 0 | task 41 | cached n_tokens = 1947, memory_seq_rm [1947, end)
|
||||
0.37.916.227 I slot create_check: id 0 | task 41 | created context checkpoint 2 of 32 (pos_min = 923, pos_max = 1946, n_tokens = 1947, size = 21.006 MiB)
|
||||
0.39.572.388 I slot print_timing: id 0 | task 41 | prompt processing, n_tokens = 2459, progress = 1.00, t = 7.57 s / 324.67 tokens per second
|
||||
0.39.572.435 I slot operator(): id 0 | task 41 | cached n_tokens = 2459, memory_seq_rm [2459, end)
|
||||
0.39.572.775 I slot init_sampler: id 0 | task 41 | init sampler, took 0.32 ms, tokens: text = 2463, total = 2463
|
||||
0.39.583.750 I slot create_check: id 0 | task 41 | created context checkpoint 3 of 32 (pos_min = 1435, pos_max = 2458, n_tokens = 2459, size = 21.006 MiB)
|
||||
0.39.719.177 I slot print_timing: id 0 | task 41 | prompt eval time = 7618.59 ms / 2463 tokens ( 3.09 ms per token, 323.29 tokens per second)
|
||||
0.39.719.216 I slot print_timing: id 0 | task 41 | eval time = 102.12 ms / 5 tokens ( 25.53 ms per token, 39.17 tokens per second)
|
||||
0.39.719.217 I slot print_timing: id 0 | task 41 | total time = 7720.71 ms / 2468 tokens
|
||||
0.39.719.219 I slot print_timing: id 0 | task 41 | graphs reused = 16
|
||||
0.39.719.306 I slot release: id 0 | task 41 | stop processing: n_tokens = 2467, truncated = 0
|
||||
0.39.719.324 I srv update_slots: all slots are idle
|
||||
0.39.734.461 I srv server_strea: conv_id= (empty=1)
|
||||
0.39.736.453 I srv operator(): chat format: peg-native
|
||||
0.39.736.607 I slot get_availabl: id 0 | task -1 | - checking sim = 1.000 (2463/2463) > 0.100
|
||||
0.39.736.628 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, f_sim_best = 1.000 (> 0.100 thold), f_keep = 0.998
|
||||
0.39.736.669 I slot launch_slot_: id 0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> ?top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
|
||||
0.39.736.694 I slot launch_slot_: id 0 | task -1 | sampler params:
|
||||
repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
|
||||
dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 64
|
||||
top_k = 0, top_p = 0.950, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.000
|
||||
mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000, adaptive_target = -1.000, adaptive_decay = 0.900
|
||||
0.39.736.695 I slot launch_slot_: id 0 | task 50 | processing task, is_child = 0
|
||||
0.39.736.700 I slot operator(): id 0 | task 50 | new prompt, n_ctx_slot = 4096, n_keep = 0, task.n_tokens = 2463
|
||||
0.39.736.702 I slot operator(): id 0 | task 50 | erased invalidated context checkpoint (pos_min = 0, pos_max = 9, n_tokens = 10, n_swa = 512, pos_next = 0, size = 0.411 MiB)
|
||||
0.39.736.704 I slot operator(): id 0 | task 50 | erased invalidated context checkpoint (pos_min = 923, pos_max = 1946, n_tokens = 1947, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
0.39.736.705 I slot operator(): id 0 | task 50 | erased invalidated context checkpoint (pos_min = 1435, pos_max = 2458, n_tokens = 2459, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
0.39.739.506 I slot operator(): id 0 | task 50 | cached n_tokens = 0, memory_seq_rm [0, end)
|
||||
0.39.778.089 I slot operator(): id 0 | task 50 | cached n_tokens = 10, memory_seq_rm [10, end)
|
||||
0.39.778.334 I slot create_check: id 0 | task 50 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 9, n_tokens = 10, size = 0.411 MiB)
|
||||
0.45.678.554 I slot print_timing: id 0 | task 50 | prompt processing, n_tokens = 1947, progress = 0.79, t = 5.94 s / 327.68 tokens per second
|
||||
0.45.678.591 I slot operator(): id 0 | task 50 | cached n_tokens = 1947, memory_seq_rm [1947, end)
|
||||
0.45.689.001 I slot create_check: id 0 | task 50 | created context checkpoint 2 of 32 (pos_min = 923, pos_max = 1946, n_tokens = 1947, size = 21.006 MiB)
|
||||
0.47.354.681 I slot print_timing: id 0 | task 50 | prompt processing, n_tokens = 2459, progress = 1.00, t = 7.62 s / 322.79 tokens per second
|
||||
0.47.354.737 I slot operator(): id 0 | task 50 | cached n_tokens = 2459, memory_seq_rm [2459, end)
|
||||
0.47.355.286 I slot init_sampler: id 0 | task 50 | init sampler, took 0.52 ms, tokens: text = 2463, total = 2463
|
||||
0.47.366.513 I slot create_check: id 0 | task 50 | created context checkpoint 3 of 32 (pos_min = 1435, pos_max = 2458, n_tokens = 2459, size = 21.006 MiB)
|
||||
0.47.494.560 I slot print_timing: id 0 | task 50 | prompt eval time = 7662.88 ms / 2463 tokens ( 3.11 ms per token, 321.42 tokens per second)
|
||||
0.47.494.591 I slot print_timing: id 0 | task 50 | eval time = 94.95 ms / 5 tokens ( 23.74 ms per token, 42.13 tokens per second)
|
||||
0.47.494.592 I slot print_timing: id 0 | task 50 | total time = 7757.83 ms / 2468 tokens
|
||||
0.47.494.594 I slot print_timing: id 0 | task 50 | graphs reused = 19
|
||||
0.47.494.692 I slot release: id 0 | task 50 | stop processing: n_tokens = 2467, truncated = 0
|
||||
0.47.494.732 I srv update_slots: all slots are idle
|
||||
0.47.508.532 I srv server_strea: conv_id= (empty=1)
|
||||
0.47.510.636 I srv operator(): chat format: peg-native
|
||||
0.47.510.786 I slot get_availabl: id 0 | task -1 | - checking sim = 1.000 (2463/2463) > 0.100
|
||||
0.47.510.807 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, f_sim_best = 1.000 (> 0.100 thold), f_keep = 0.998
|
||||
0.47.510.840 I slot launch_slot_: id 0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> ?top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
|
||||
0.47.510.864 I slot launch_slot_: id 0 | task -1 | sampler params:
|
||||
repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
|
||||
dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 64
|
||||
top_k = 0, top_p = 0.950, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.000
|
||||
mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000, adaptive_target = -1.000, adaptive_decay = 0.900
|
||||
0.47.510.865 I slot launch_slot_: id 0 | task 59 | processing task, is_child = 0
|
||||
0.47.510.870 I slot operator(): id 0 | task 59 | new prompt, n_ctx_slot = 4096, n_keep = 0, task.n_tokens = 2463
|
||||
0.47.510.872 I slot operator(): id 0 | task 59 | erased invalidated context checkpoint (pos_min = 0, pos_max = 9, n_tokens = 10, n_swa = 512, pos_next = 0, size = 0.411 MiB)
|
||||
0.47.510.873 I slot operator(): id 0 | task 59 | erased invalidated context checkpoint (pos_min = 923, pos_max = 1946, n_tokens = 1947, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
0.47.510.875 I slot operator(): id 0 | task 59 | erased invalidated context checkpoint (pos_min = 1435, pos_max = 2458, n_tokens = 2459, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
0.47.513.765 I slot operator(): id 0 | task 59 | cached n_tokens = 0, memory_seq_rm [0, end)
|
||||
0.47.553.566 I slot operator(): id 0 | task 59 | cached n_tokens = 10, memory_seq_rm [10, end)
|
||||
0.47.553.754 I slot create_check: id 0 | task 59 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 9, n_tokens = 10, size = 0.411 MiB)
|
||||
0.53.445.137 I slot print_timing: id 0 | task 59 | prompt processing, n_tokens = 1947, progress = 0.79, t = 5.93 s / 328.10 tokens per second
|
||||
0.53.445.174 I slot operator(): id 0 | task 59 | cached n_tokens = 1947, memory_seq_rm [1947, end)
|
||||
0.53.455.768 I slot create_check: id 0 | task 59 | created context checkpoint 2 of 32 (pos_min = 923, pos_max = 1946, n_tokens = 1947, size = 21.006 MiB)
|
||||
0.55.099.608 I slot print_timing: id 0 | task 59 | prompt processing, n_tokens = 2459, progress = 1.00, t = 7.59 s / 324.03 tokens per second
|
||||
0.55.099.645 I slot operator(): id 0 | task 59 | cached n_tokens = 2459, memory_seq_rm [2459, end)
|
||||
0.55.099.987 I slot init_sampler: id 0 | task 59 | init sampler, took 0.32 ms, tokens: text = 2463, total = 2463
|
||||
0.55.114.620 I slot create_check: id 0 | task 59 | created context checkpoint 3 of 32 (pos_min = 1435, pos_max = 2458, n_tokens = 2459, size = 21.006 MiB)
|
||||
0.55.246.618 I slot print_timing: id 0 | task 59 | prompt eval time = 7638.78 ms / 2463 tokens ( 3.10 ms per token, 322.43 tokens per second)
|
||||
0.55.246.652 I slot print_timing: id 0 | task 59 | eval time = 96.94 ms / 5 tokens ( 24.23 ms per token, 41.26 tokens per second)
|
||||
0.55.246.653 I slot print_timing: id 0 | task 59 | total time = 7735.72 ms / 2468 tokens
|
||||
0.55.246.654 I slot print_timing: id 0 | task 59 | graphs reused = 22
|
||||
0.55.246.752 I slot release: id 0 | task 59 | stop processing: n_tokens = 2467, truncated = 0
|
||||
0.55.246.791 I srv update_slots: all slots are idle
|
||||
0.55.260.015 I srv server_strea: conv_id= (empty=1)
|
||||
0.55.262.197 I srv operator(): chat format: peg-native
|
||||
0.55.262.386 I slot get_availabl: id 0 | task -1 | - checking sim = 1.000 (2463/2463) > 0.100
|
||||
0.55.262.407 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, f_sim_best = 1.000 (> 0.100 thold), f_keep = 0.998
|
||||
0.55.262.439 I slot launch_slot_: id 0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> ?top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
|
||||
0.55.262.464 I slot launch_slot_: id 0 | task -1 | sampler params:
|
||||
repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
|
||||
dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 64
|
||||
top_k = 0, top_p = 0.950, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.000
|
||||
mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000, adaptive_target = -1.000, adaptive_decay = 0.900
|
||||
0.55.262.465 I slot launch_slot_: id 0 | task 68 | processing task, is_child = 0
|
||||
0.55.262.470 I slot operator(): id 0 | task 68 | new prompt, n_ctx_slot = 4096, n_keep = 0, task.n_tokens = 2463
|
||||
0.55.262.472 I slot operator(): id 0 | task 68 | erased invalidated context checkpoint (pos_min = 0, pos_max = 9, n_tokens = 10, n_swa = 512, pos_next = 0, size = 0.411 MiB)
|
||||
0.55.262.473 I slot operator(): id 0 | task 68 | erased invalidated context checkpoint (pos_min = 923, pos_max = 1946, n_tokens = 1947, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
0.55.262.475 I slot operator(): id 0 | task 68 | erased invalidated context checkpoint (pos_min = 1435, pos_max = 2458, n_tokens = 2459, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
0.55.265.286 I slot operator(): id 0 | task 68 | cached n_tokens = 0, memory_seq_rm [0, end)
|
||||
0.55.304.008 I slot operator(): id 0 | task 68 | cached n_tokens = 10, memory_seq_rm [10, end)
|
||||
0.55.304.147 I slot create_check: id 0 | task 68 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 9, n_tokens = 10, size = 0.411 MiB)
|
||||
1.01.295.930 I slot print_timing: id 0 | task 68 | prompt processing, n_tokens = 1947, progress = 0.79, t = 6.03 s / 322.70 tokens per second
|
||||
1.01.295.969 I slot operator(): id 0 | task 68 | cached n_tokens = 1947, memory_seq_rm [1947, end)
|
||||
1.01.306.204 I slot create_check: id 0 | task 68 | created context checkpoint 2 of 32 (pos_min = 923, pos_max = 1946, n_tokens = 1947, size = 21.006 MiB)
|
||||
1.02.989.169 I slot print_timing: id 0 | task 68 | prompt processing, n_tokens = 2459, progress = 1.00, t = 7.73 s / 318.25 tokens per second
|
||||
1.02.989.208 I slot operator(): id 0 | task 68 | cached n_tokens = 2459, memory_seq_rm [2459, end)
|
||||
1.02.989.571 I slot init_sampler: id 0 | task 68 | init sampler, took 0.33 ms, tokens: text = 2463, total = 2463
|
||||
1.03.000.450 I slot create_check: id 0 | task 68 | created context checkpoint 3 of 32 (pos_min = 1435, pos_max = 2458, n_tokens = 2459, size = 21.006 MiB)
|
||||
1.03.133.202 I slot print_timing: id 0 | task 68 | prompt eval time = 7771.60 ms / 2463 tokens ( 3.16 ms per token, 316.92 tokens per second)
|
||||
1.03.133.240 I slot print_timing: id 0 | task 68 | eval time = 99.10 ms / 5 tokens ( 24.78 ms per token, 40.36 tokens per second)
|
||||
1.03.133.241 I slot print_timing: id 0 | task 68 | total time = 7870.70 ms / 2468 tokens
|
||||
1.03.133.242 I slot print_timing: id 0 | task 68 | graphs reused = 25
|
||||
1.03.133.344 I slot release: id 0 | task 68 | stop processing: n_tokens = 2467, truncated = 0
|
||||
1.03.133.385 I srv update_slots: all slots are idle
|
||||
1.03.146.997 I srv server_strea: conv_id= (empty=1)
|
||||
1.03.149.168 I srv operator(): chat format: peg-native
|
||||
1.03.149.362 I slot get_availabl: id 0 | task -1 | - checking sim = 1.000 (2463/2463) > 0.100
|
||||
1.03.149.423 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, f_sim_best = 1.000 (> 0.100 thold), f_keep = 0.998
|
||||
1.03.149.488 I slot launch_slot_: id 0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> ?top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
|
||||
1.03.149.607 I slot launch_slot_: id 0 | task -1 | sampler params:
|
||||
repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
|
||||
dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 64
|
||||
top_k = 0, top_p = 0.950, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.000
|
||||
mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000, adaptive_target = -1.000, adaptive_decay = 0.900
|
||||
1.03.149.715 I slot launch_slot_: id 0 | task 77 | processing task, is_child = 0
|
||||
1.03.149.781 I slot operator(): id 0 | task 77 | new prompt, n_ctx_slot = 4096, n_keep = 0, task.n_tokens = 2463
|
||||
1.03.149.813 I slot operator(): id 0 | task 77 | erased invalidated context checkpoint (pos_min = 0, pos_max = 9, n_tokens = 10, n_swa = 512, pos_next = 0, size = 0.411 MiB)
|
||||
1.03.149.815 I slot operator(): id 0 | task 77 | erased invalidated context checkpoint (pos_min = 923, pos_max = 1946, n_tokens = 1947, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
1.03.149.816 I slot operator(): id 0 | task 77 | erased invalidated context checkpoint (pos_min = 1435, pos_max = 2458, n_tokens = 2459, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
1.03.152.637 I slot operator(): id 0 | task 77 | cached n_tokens = 0, memory_seq_rm [0, end)
|
||||
1.03.191.595 I slot operator(): id 0 | task 77 | cached n_tokens = 10, memory_seq_rm [10, end)
|
||||
1.03.191.740 I slot create_check: id 0 | task 77 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 9, n_tokens = 10, size = 0.411 MiB)
|
||||
1.09.200.637 I slot print_timing: id 0 | task 77 | prompt processing, n_tokens = 1947, progress = 0.79, t = 6.05 s / 321.77 tokens per second
|
||||
1.09.200.789 I slot operator(): id 0 | task 77 | cached n_tokens = 1947, memory_seq_rm [1947, end)
|
||||
1.09.210.947 I slot create_check: id 0 | task 77 | created context checkpoint 2 of 32 (pos_min = 923, pos_max = 1946, n_tokens = 1947, size = 21.006 MiB)
|
||||
1.10.873.112 I slot print_timing: id 0 | task 77 | prompt processing, n_tokens = 2459, progress = 1.00, t = 7.72 s / 318.39 tokens per second
|
||||
1.10.873.147 I slot operator(): id 0 | task 77 | cached n_tokens = 2459, memory_seq_rm [2459, end)
|
||||
1.10.873.487 I slot init_sampler: id 0 | task 77 | init sampler, took 0.31 ms, tokens: text = 2463, total = 2463
|
||||
1.10.883.963 I slot create_check: id 0 | task 77 | created context checkpoint 3 of 32 (pos_min = 1435, pos_max = 2458, n_tokens = 2459, size = 21.006 MiB)
|
||||
1.11.019.766 I slot print_timing: id 0 | task 77 | prompt eval time = 7769.73 ms / 2463 tokens ( 3.15 ms per token, 317.00 tokens per second)
|
||||
1.11.019.812 I slot print_timing: id 0 | task 77 | eval time = 100.22 ms / 5 tokens ( 25.06 ms per token, 39.91 tokens per second)
|
||||
1.11.019.813 I slot print_timing: id 0 | task 77 | total time = 7869.95 ms / 2468 tokens
|
||||
1.11.019.815 I slot print_timing: id 0 | task 77 | graphs reused = 28
|
||||
1.11.019.907 I slot release: id 0 | task 77 | stop processing: n_tokens = 2467, truncated = 0
|
||||
1.11.019.955 I srv update_slots: all slots are idle
|
||||
1.11.035.338 I srv server_strea: conv_id= (empty=1)
|
||||
1.11.037.478 I srv operator(): chat format: peg-native
|
||||
1.11.037.684 I slot get_availabl: id 0 | task -1 | - checking sim = 1.000 (2463/2463) > 0.100
|
||||
1.11.037.748 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, f_sim_best = 1.000 (> 0.100 thold), f_keep = 0.998
|
||||
1.11.037.823 I slot launch_slot_: id 0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> ?top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
|
||||
1.11.037.943 I slot launch_slot_: id 0 | task -1 | sampler params:
|
||||
repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
|
||||
dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 64
|
||||
top_k = 0, top_p = 0.950, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.000
|
||||
mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000, adaptive_target = -1.000, adaptive_decay = 0.900
|
||||
1.11.038.052 I slot launch_slot_: id 0 | task 86 | processing task, is_child = 0
|
||||
1.11.038.064 I slot operator(): id 0 | task 86 | new prompt, n_ctx_slot = 4096, n_keep = 0, task.n_tokens = 2463
|
||||
1.11.038.175 I slot operator(): id 0 | task 86 | erased invalidated context checkpoint (pos_min = 0, pos_max = 9, n_tokens = 10, n_swa = 512, pos_next = 0, size = 0.411 MiB)
|
||||
1.11.038.181 I slot operator(): id 0 | task 86 | erased invalidated context checkpoint (pos_min = 923, pos_max = 1946, n_tokens = 1947, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
1.11.038.184 I slot operator(): id 0 | task 86 | erased invalidated context checkpoint (pos_min = 1435, pos_max = 2458, n_tokens = 2459, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
1.11.041.061 I slot operator(): id 0 | task 86 | cached n_tokens = 0, memory_seq_rm [0, end)
|
||||
1.11.081.944 I slot operator(): id 0 | task 86 | cached n_tokens = 10, memory_seq_rm [10, end)
|
||||
1.11.082.085 I slot create_check: id 0 | task 86 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 9, n_tokens = 10, size = 0.411 MiB)
|
||||
1.16.986.583 I slot print_timing: id 0 | task 86 | prompt processing, n_tokens = 1947, progress = 0.79, t = 5.95 s / 327.31 tokens per second
|
||||
1.16.986.615 I slot operator(): id 0 | task 86 | cached n_tokens = 1947, memory_seq_rm [1947, end)
|
||||
1.16.996.043 I slot create_check: id 0 | task 86 | created context checkpoint 2 of 32 (pos_min = 923, pos_max = 1946, n_tokens = 1947, size = 21.006 MiB)
|
||||
1.18.668.183 I slot print_timing: id 0 | task 86 | prompt processing, n_tokens = 2459, progress = 1.00, t = 7.63 s / 322.28 tokens per second
|
||||
1.18.668.212 I slot operator(): id 0 | task 86 | cached n_tokens = 2459, memory_seq_rm [2459, end)
|
||||
1.18.668.516 I slot init_sampler: id 0 | task 86 | init sampler, took 0.28 ms, tokens: text = 2463, total = 2463
|
||||
1.18.678.183 I slot create_check: id 0 | task 86 | created context checkpoint 3 of 32 (pos_min = 1435, pos_max = 2458, n_tokens = 2459, size = 21.006 MiB)
|
||||
1.18.816.195 I slot print_timing: id 0 | task 86 | prompt eval time = 7676.35 ms / 2463 tokens ( 3.12 ms per token, 320.86 tokens per second)
|
||||
1.18.816.239 I slot print_timing: id 0 | task 86 | eval time = 101.75 ms / 5 tokens ( 25.44 ms per token, 39.31 tokens per second)
|
||||
1.18.816.241 I slot print_timing: id 0 | task 86 | total time = 7778.10 ms / 2468 tokens
|
||||
1.18.816.242 I slot print_timing: id 0 | task 86 | graphs reused = 31
|
||||
1.18.816.335 I slot release: id 0 | task 86 | stop processing: n_tokens = 2467, truncated = 0
|
||||
1.18.816.380 I srv update_slots: all slots are idle
|
||||
1.18.830.428 I srv server_strea: conv_id= (empty=1)
|
||||
1.18.832.470 I srv operator(): chat format: peg-native
|
||||
1.18.832.618 I slot get_availabl: id 0 | task -1 | - checking sim = 1.000 (2463/2463) > 0.100
|
||||
1.18.832.640 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, f_sim_best = 1.000 (> 0.100 thold), f_keep = 0.998
|
||||
1.18.832.680 I slot launch_slot_: id 0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> ?top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
|
||||
1.18.832.705 I slot launch_slot_: id 0 | task -1 | sampler params:
|
||||
repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
|
||||
dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 64
|
||||
top_k = 0, top_p = 0.950, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.000
|
||||
mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000, adaptive_target = -1.000, adaptive_decay = 0.900
|
||||
1.18.832.706 I slot launch_slot_: id 0 | task 95 | processing task, is_child = 0
|
||||
1.18.832.712 I slot operator(): id 0 | task 95 | new prompt, n_ctx_slot = 4096, n_keep = 0, task.n_tokens = 2463
|
||||
1.18.832.713 I slot operator(): id 0 | task 95 | erased invalidated context checkpoint (pos_min = 0, pos_max = 9, n_tokens = 10, n_swa = 512, pos_next = 0, size = 0.411 MiB)
|
||||
1.18.832.715 I slot operator(): id 0 | task 95 | erased invalidated context checkpoint (pos_min = 923, pos_max = 1946, n_tokens = 1947, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
1.18.832.717 I slot operator(): id 0 | task 95 | erased invalidated context checkpoint (pos_min = 1435, pos_max = 2458, n_tokens = 2459, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
1.18.835.610 I slot operator(): id 0 | task 95 | cached n_tokens = 0, memory_seq_rm [0, end)
|
||||
1.18.872.805 I slot operator(): id 0 | task 95 | cached n_tokens = 10, memory_seq_rm [10, end)
|
||||
1.18.872.958 I slot create_check: id 0 | task 95 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 9, n_tokens = 10, size = 0.411 MiB)
|
||||
1.24.818.085 I slot print_timing: id 0 | task 95 | prompt processing, n_tokens = 1947, progress = 0.79, t = 5.99 s / 325.30 tokens per second
|
||||
1.24.818.123 I slot operator(): id 0 | task 95 | cached n_tokens = 1947, memory_seq_rm [1947, end)
|
||||
1.24.828.404 I slot create_check: id 0 | task 95 | created context checkpoint 2 of 32 (pos_min = 923, pos_max = 1946, n_tokens = 1947, size = 21.006 MiB)
|
||||
1.26.491.223 I slot print_timing: id 0 | task 95 | prompt processing, n_tokens = 2459, progress = 1.00, t = 7.66 s / 321.08 tokens per second
|
||||
1.26.491.262 I slot operator(): id 0 | task 95 | cached n_tokens = 2459, memory_seq_rm [2459, end)
|
||||
1.26.491.675 I slot init_sampler: id 0 | task 95 | init sampler, took 0.38 ms, tokens: text = 2463, total = 2463
|
||||
1.26.501.787 I slot create_check: id 0 | task 95 | created context checkpoint 3 of 32 (pos_min = 1435, pos_max = 2458, n_tokens = 2459, size = 21.006 MiB)
|
||||
1.26.636.457 I slot print_timing: id 0 | task 95 | prompt eval time = 7702.30 ms / 2463 tokens ( 3.13 ms per token, 319.77 tokens per second)
|
||||
1.26.636.512 I slot print_timing: id 0 | task 95 | eval time = 101.41 ms / 5 tokens ( 25.35 ms per token, 39.44 tokens per second)
|
||||
1.26.636.513 I slot print_timing: id 0 | task 95 | total time = 7803.71 ms / 2468 tokens
|
||||
1.26.636.515 I slot print_timing: id 0 | task 95 | graphs reused = 34
|
||||
1.26.636.632 I slot release: id 0 | task 95 | stop processing: n_tokens = 2467, truncated = 0
|
||||
1.26.636.690 I srv update_slots: all slots are idle
|
||||
1.26.650.131 I srv server_strea: conv_id= (empty=1)
|
||||
1.26.652.143 I srv operator(): chat format: peg-native
|
||||
1.26.652.301 I slot get_availabl: id 0 | task -1 | - checking sim = 1.000 (2463/2463) > 0.100
|
||||
1.26.652.321 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, f_sim_best = 1.000 (> 0.100 thold), f_keep = 0.998
|
||||
1.26.652.359 I slot launch_slot_: id 0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> ?top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
|
||||
1.26.652.456 I slot launch_slot_: id 0 | task -1 | sampler params:
|
||||
repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
|
||||
dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 64
|
||||
top_k = 0, top_p = 0.950, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.000
|
||||
mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000, adaptive_target = -1.000, adaptive_decay = 0.900
|
||||
1.26.652.563 I slot launch_slot_: id 0 | task 104 | processing task, is_child = 0
|
||||
1.26.652.629 I slot operator(): id 0 | task 104 | new prompt, n_ctx_slot = 4096, n_keep = 0, task.n_tokens = 2463
|
||||
1.26.652.738 I slot operator(): id 0 | task 104 | erased invalidated context checkpoint (pos_min = 0, pos_max = 9, n_tokens = 10, n_swa = 512, pos_next = 0, size = 0.411 MiB)
|
||||
1.26.652.798 I slot operator(): id 0 | task 104 | erased invalidated context checkpoint (pos_min = 923, pos_max = 1946, n_tokens = 1947, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
1.26.652.857 I slot operator(): id 0 | task 104 | erased invalidated context checkpoint (pos_min = 1435, pos_max = 2458, n_tokens = 2459, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
1.26.655.963 I slot operator(): id 0 | task 104 | cached n_tokens = 0, memory_seq_rm [0, end)
|
||||
1.26.692.745 I slot operator(): id 0 | task 104 | cached n_tokens = 10, memory_seq_rm [10, end)
|
||||
1.26.692.881 I slot create_check: id 0 | task 104 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 9, n_tokens = 10, size = 0.411 MiB)
|
||||
1.32.703.627 I slot print_timing: id 0 | task 104 | prompt processing, n_tokens = 1947, progress = 0.79, t = 6.05 s / 321.77 tokens per second
|
||||
1.32.703.664 I slot operator(): id 0 | task 104 | cached n_tokens = 1947, memory_seq_rm [1947, end)
|
||||
1.32.714.192 I slot create_check: id 0 | task 104 | created context checkpoint 2 of 32 (pos_min = 923, pos_max = 1946, n_tokens = 1947, size = 21.006 MiB)
|
||||
1.34.400.994 I slot print_timing: id 0 | task 104 | prompt processing, n_tokens = 2459, progress = 1.00, t = 7.75 s / 317.36 tokens per second
|
||||
1.34.401.032 I slot operator(): id 0 | task 104 | cached n_tokens = 2459, memory_seq_rm [2459, end)
|
||||
1.34.401.372 I slot init_sampler: id 0 | task 104 | init sampler, took 0.32 ms, tokens: text = 2463, total = 2463
|
||||
1.34.410.575 I slot create_check: id 0 | task 104 | created context checkpoint 3 of 32 (pos_min = 1435, pos_max = 2458, n_tokens = 2459, size = 21.006 MiB)
|
||||
1.34.541.735 I slot print_timing: id 0 | task 104 | prompt eval time = 7791.06 ms / 2463 tokens ( 3.16 ms per token, 316.13 tokens per second)
|
||||
1.34.541.803 I slot print_timing: id 0 | task 104 | eval time = 98.02 ms / 5 tokens ( 24.50 ms per token, 40.81 tokens per second)
|
||||
1.34.541.807 I slot print_timing: id 0 | task 104 | total time = 7889.08 ms / 2468 tokens
|
||||
1.34.541.809 I slot print_timing: id 0 | task 104 | graphs reused = 37
|
||||
1.34.541.900 I slot release: id 0 | task 104 | stop processing: n_tokens = 2467, truncated = 0
|
||||
1.34.541.919 I srv update_slots: all slots are idle
|
||||
1.34.555.443 I srv server_strea: conv_id= (empty=1)
|
||||
1.34.557.541 I srv operator(): chat format: peg-native
|
||||
1.34.557.693 I slot get_availabl: id 0 | task -1 | - checking sim = 1.000 (2463/2463) > 0.100
|
||||
1.34.557.713 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, f_sim_best = 1.000 (> 0.100 thold), f_keep = 0.998
|
||||
1.34.557.750 I slot launch_slot_: id 0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> ?top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
|
||||
1.34.557.775 I slot launch_slot_: id 0 | task -1 | sampler params:
|
||||
repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
|
||||
dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 64
|
||||
top_k = 0, top_p = 0.950, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.000
|
||||
mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000, adaptive_target = -1.000, adaptive_decay = 0.900
|
||||
1.34.557.776 I slot launch_slot_: id 0 | task 113 | processing task, is_child = 0
|
||||
1.34.557.781 I slot operator(): id 0 | task 113 | new prompt, n_ctx_slot = 4096, n_keep = 0, task.n_tokens = 2463
|
||||
1.34.557.783 I slot operator(): id 0 | task 113 | erased invalidated context checkpoint (pos_min = 0, pos_max = 9, n_tokens = 10, n_swa = 512, pos_next = 0, size = 0.411 MiB)
|
||||
1.34.557.785 I slot operator(): id 0 | task 113 | erased invalidated context checkpoint (pos_min = 923, pos_max = 1946, n_tokens = 1947, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
1.34.557.786 I slot operator(): id 0 | task 113 | erased invalidated context checkpoint (pos_min = 1435, pos_max = 2458, n_tokens = 2459, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
1.34.560.510 I slot operator(): id 0 | task 113 | cached n_tokens = 0, memory_seq_rm [0, end)
|
||||
1.34.599.665 I slot operator(): id 0 | task 113 | cached n_tokens = 10, memory_seq_rm [10, end)
|
||||
1.34.599.802 I slot create_check: id 0 | task 113 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 9, n_tokens = 10, size = 0.411 MiB)
|
||||
1.40.632.315 I slot print_timing: id 0 | task 113 | prompt processing, n_tokens = 1947, progress = 0.79, t = 6.07 s / 320.52 tokens per second
|
||||
1.40.632.375 I slot operator(): id 0 | task 113 | cached n_tokens = 1947, memory_seq_rm [1947, end)
|
||||
1.40.642.950 I slot create_check: id 0 | task 113 | created context checkpoint 2 of 32 (pos_min = 923, pos_max = 1946, n_tokens = 1947, size = 21.006 MiB)
|
||||
1.42.314.809 I slot print_timing: id 0 | task 113 | prompt processing, n_tokens = 2459, progress = 1.00, t = 7.76 s / 317.00 tokens per second
|
||||
1.42.314.845 I slot operator(): id 0 | task 113 | cached n_tokens = 2459, memory_seq_rm [2459, end)
|
||||
1.42.315.190 I slot init_sampler: id 0 | task 113 | init sampler, took 0.32 ms, tokens: text = 2463, total = 2463
|
||||
1.42.332.816 I slot create_check: id 0 | task 113 | created context checkpoint 3 of 32 (pos_min = 1435, pos_max = 2458, n_tokens = 2459, size = 21.006 MiB)
|
||||
1.42.465.775 I slot print_timing: id 0 | task 113 | prompt eval time = 7809.40 ms / 2463 tokens ( 3.17 ms per token, 315.39 tokens per second)
|
||||
1.42.465.812 I slot print_timing: id 0 | task 113 | eval time = 98.56 ms / 5 tokens ( 24.64 ms per token, 40.59 tokens per second)
|
||||
1.42.465.813 I slot print_timing: id 0 | task 113 | total time = 7907.95 ms / 2468 tokens
|
||||
1.42.465.814 I slot print_timing: id 0 | task 113 | graphs reused = 40
|
||||
1.42.465.908 I slot release: id 0 | task 113 | stop processing: n_tokens = 2467, truncated = 0
|
||||
1.42.465.926 I srv update_slots: all slots are idle
|
||||
1.42.478.883 I srv server_strea: conv_id= (empty=1)
|
||||
1.42.480.912 I srv operator(): chat format: peg-native
|
||||
1.42.481.065 I slot get_availabl: id 0 | task -1 | - checking sim = 1.000 (2463/2463) > 0.100
|
||||
1.42.481.086 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, f_sim_best = 1.000 (> 0.100 thold), f_keep = 0.998
|
||||
1.42.481.117 I slot launch_slot_: id 0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> ?top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
|
||||
1.42.481.142 I slot launch_slot_: id 0 | task -1 | sampler params:
|
||||
repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
|
||||
dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 64
|
||||
top_k = 0, top_p = 0.950, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.000
|
||||
mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000, adaptive_target = -1.000, adaptive_decay = 0.900
|
||||
1.42.481.143 I slot launch_slot_: id 0 | task 122 | processing task, is_child = 0
|
||||
1.42.481.147 I slot operator(): id 0 | task 122 | new prompt, n_ctx_slot = 4096, n_keep = 0, task.n_tokens = 2463
|
||||
1.42.481.150 I slot operator(): id 0 | task 122 | erased invalidated context checkpoint (pos_min = 0, pos_max = 9, n_tokens = 10, n_swa = 512, pos_next = 0, size = 0.411 MiB)
|
||||
1.42.481.151 I slot operator(): id 0 | task 122 | erased invalidated context checkpoint (pos_min = 923, pos_max = 1946, n_tokens = 1947, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
1.42.481.153 I slot operator(): id 0 | task 122 | erased invalidated context checkpoint (pos_min = 1435, pos_max = 2458, n_tokens = 2459, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
1.42.483.850 I slot operator(): id 0 | task 122 | cached n_tokens = 0, memory_seq_rm [0, end)
|
||||
1.42.524.592 I slot operator(): id 0 | task 122 | cached n_tokens = 10, memory_seq_rm [10, end)
|
||||
1.42.524.779 I slot create_check: id 0 | task 122 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 9, n_tokens = 10, size = 0.411 MiB)
|
||||
1.48.597.129 I slot print_timing: id 0 | task 122 | prompt processing, n_tokens = 1947, progress = 0.79, t = 6.12 s / 318.35 tokens per second
|
||||
1.48.597.170 I slot operator(): id 0 | task 122 | cached n_tokens = 1947, memory_seq_rm [1947, end)
|
||||
1.48.607.854 I slot create_check: id 0 | task 122 | created context checkpoint 2 of 32 (pos_min = 923, pos_max = 1946, n_tokens = 1947, size = 21.006 MiB)
|
||||
1.50.295.724 I slot print_timing: id 0 | task 122 | prompt processing, n_tokens = 2459, progress = 1.00, t = 7.81 s / 314.67 tokens per second
|
||||
1.50.295.766 I slot operator(): id 0 | task 122 | cached n_tokens = 2459, memory_seq_rm [2459, end)
|
||||
1.50.296.108 I slot init_sampler: id 0 | task 122 | init sampler, took 0.31 ms, tokens: text = 2463, total = 2463
|
||||
1.50.306.876 I slot create_check: id 0 | task 122 | created context checkpoint 3 of 32 (pos_min = 1435, pos_max = 2458, n_tokens = 2459, size = 21.006 MiB)
|
||||
1.50.438.090 I slot print_timing: id 0 | task 122 | prompt eval time = 7859.48 ms / 2463 tokens ( 3.19 ms per token, 313.38 tokens per second)
|
||||
1.50.438.121 I slot print_timing: id 0 | task 122 | eval time = 97.44 ms / 5 tokens ( 24.36 ms per token, 41.05 tokens per second)
|
||||
1.50.438.122 I slot print_timing: id 0 | task 122 | total time = 7956.92 ms / 2468 tokens
|
||||
1.50.438.124 I slot print_timing: id 0 | task 122 | graphs reused = 43
|
||||
1.50.438.195 I slot release: id 0 | task 122 | stop processing: n_tokens = 2467, truncated = 0
|
||||
1.50.438.210 I srv update_slots: all slots are idle
|
||||
1.50.451.614 I srv server_strea: conv_id= (empty=1)
|
||||
1.50.453.716 I srv operator(): chat format: peg-native
|
||||
1.50.453.867 I slot get_availabl: id 0 | task -1 | - checking sim = 1.000 (2463/2463) > 0.100
|
||||
1.50.453.888 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, f_sim_best = 1.000 (> 0.100 thold), f_keep = 0.998
|
||||
1.50.453.922 I slot launch_slot_: id 0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> ?top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
|
||||
1.50.453.946 I slot launch_slot_: id 0 | task -1 | sampler params:
|
||||
repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
|
||||
dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 64
|
||||
top_k = 0, top_p = 0.950, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.000
|
||||
mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000, adaptive_target = -1.000, adaptive_decay = 0.900
|
||||
1.50.453.947 I slot launch_slot_: id 0 | task 131 | processing task, is_child = 0
|
||||
1.50.453.953 I slot operator(): id 0 | task 131 | new prompt, n_ctx_slot = 4096, n_keep = 0, task.n_tokens = 2463
|
||||
1.50.453.954 I slot operator(): id 0 | task 131 | erased invalidated context checkpoint (pos_min = 0, pos_max = 9, n_tokens = 10, n_swa = 512, pos_next = 0, size = 0.411 MiB)
|
||||
1.50.453.956 I slot operator(): id 0 | task 131 | erased invalidated context checkpoint (pos_min = 923, pos_max = 1946, n_tokens = 1947, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
1.50.453.957 I slot operator(): id 0 | task 131 | erased invalidated context checkpoint (pos_min = 1435, pos_max = 2458, n_tokens = 2459, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
1.50.456.681 I slot operator(): id 0 | task 131 | cached n_tokens = 0, memory_seq_rm [0, end)
|
||||
1.50.499.193 I slot operator(): id 0 | task 131 | cached n_tokens = 10, memory_seq_rm [10, end)
|
||||
1.50.499.340 I slot create_check: id 0 | task 131 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 9, n_tokens = 10, size = 0.411 MiB)
|
||||
1.56.514.920 I slot print_timing: id 0 | task 131 | prompt processing, n_tokens = 1947, progress = 0.79, t = 6.06 s / 321.24 tokens per second
|
||||
1.56.514.960 I slot operator(): id 0 | task 131 | cached n_tokens = 1947, memory_seq_rm [1947, end)
|
||||
1.56.526.117 I slot create_check: id 0 | task 131 | created context checkpoint 2 of 32 (pos_min = 923, pos_max = 1946, n_tokens = 1947, size = 21.006 MiB)
|
||||
1.58.266.037 I slot print_timing: id 0 | task 131 | prompt processing, n_tokens = 2459, progress = 1.00, t = 7.81 s / 314.77 tokens per second
|
||||
1.58.266.076 I slot operator(): id 0 | task 131 | cached n_tokens = 2459, memory_seq_rm [2459, end)
|
||||
1.58.266.449 I slot init_sampler: id 0 | task 131 | init sampler, took 0.34 ms, tokens: text = 2463, total = 2463
|
||||
1.58.277.170 I slot create_check: id 0 | task 131 | created context checkpoint 3 of 32 (pos_min = 1435, pos_max = 2458, n_tokens = 2459, size = 21.006 MiB)
|
||||
1.58.408.664 I slot print_timing: id 0 | task 131 | prompt eval time = 7857.15 ms / 2463 tokens ( 3.19 ms per token, 313.47 tokens per second)
|
||||
1.58.408.708 I slot print_timing: id 0 | task 131 | eval time = 97.53 ms / 5 tokens ( 24.38 ms per token, 41.01 tokens per second)
|
||||
1.58.408.709 I slot print_timing: id 0 | task 131 | total time = 7954.68 ms / 2468 tokens
|
||||
1.58.408.710 I slot print_timing: id 0 | task 131 | graphs reused = 46
|
||||
1.58.408.803 I slot release: id 0 | task 131 | stop processing: n_tokens = 2467, truncated = 0
|
||||
1.58.408.847 I srv update_slots: all slots are idle
|
||||
1.58.422.374 I srv server_strea: conv_id= (empty=1)
|
||||
1.58.424.350 I srv operator(): chat format: peg-native
|
||||
1.58.424.504 I slot get_availabl: id 0 | task -1 | - checking sim = 1.000 (2463/2463) > 0.100
|
||||
1.58.424.525 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, f_sim_best = 1.000 (> 0.100 thold), f_keep = 0.998
|
||||
1.58.424.562 I slot launch_slot_: id 0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> ?top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
|
||||
1.58.424.586 I slot launch_slot_: id 0 | task -1 | sampler params:
|
||||
repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
|
||||
dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 64
|
||||
top_k = 0, top_p = 0.950, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.000
|
||||
mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000, adaptive_target = -1.000, adaptive_decay = 0.900
|
||||
1.58.424.588 I slot launch_slot_: id 0 | task 140 | processing task, is_child = 0
|
||||
1.58.424.593 I slot operator(): id 0 | task 140 | new prompt, n_ctx_slot = 4096, n_keep = 0, task.n_tokens = 2463
|
||||
1.58.424.595 I slot operator(): id 0 | task 140 | erased invalidated context checkpoint (pos_min = 0, pos_max = 9, n_tokens = 10, n_swa = 512, pos_next = 0, size = 0.411 MiB)
|
||||
1.58.424.597 I slot operator(): id 0 | task 140 | erased invalidated context checkpoint (pos_min = 923, pos_max = 1946, n_tokens = 1947, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
1.58.424.599 I slot operator(): id 0 | task 140 | erased invalidated context checkpoint (pos_min = 1435, pos_max = 2458, n_tokens = 2459, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
1.58.427.438 I slot operator(): id 0 | task 140 | cached n_tokens = 0, memory_seq_rm [0, end)
|
||||
1.58.464.993 I slot operator(): id 0 | task 140 | cached n_tokens = 10, memory_seq_rm [10, end)
|
||||
1.58.465.133 I slot create_check: id 0 | task 140 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 9, n_tokens = 10, size = 0.411 MiB)
|
||||
2.04.573.806 I slot print_timing: id 0 | task 140 | prompt processing, n_tokens = 1947, progress = 0.79, t = 6.15 s / 316.63 tokens per second
|
||||
2.04.573.834 I slot operator(): id 0 | task 140 | cached n_tokens = 1947, memory_seq_rm [1947, end)
|
||||
2.04.582.157 I slot create_check: id 0 | task 140 | created context checkpoint 2 of 32 (pos_min = 923, pos_max = 1946, n_tokens = 1947, size = 21.006 MiB)
|
||||
2.06.277.842 I slot print_timing: id 0 | task 140 | prompt processing, n_tokens = 2459, progress = 1.00, t = 7.85 s / 313.12 tokens per second
|
||||
2.06.277.872 I slot operator(): id 0 | task 140 | cached n_tokens = 2459, memory_seq_rm [2459, end)
|
||||
2.06.278.177 I slot init_sampler: id 0 | task 140 | init sampler, took 0.28 ms, tokens: text = 2463, total = 2463
|
||||
2.06.287.879 I slot create_check: id 0 | task 140 | created context checkpoint 3 of 32 (pos_min = 1435, pos_max = 2458, n_tokens = 2459, size = 21.006 MiB)
|
||||
2.06.421.507 I slot print_timing: id 0 | task 140 | prompt eval time = 7897.30 ms / 2463 tokens ( 3.21 ms per token, 311.88 tokens per second)
|
||||
2.06.421.551 I slot print_timing: id 0 | task 140 | eval time = 99.59 ms / 5 tokens ( 24.90 ms per token, 40.16 tokens per second)
|
||||
2.06.421.552 I slot print_timing: id 0 | task 140 | total time = 7996.89 ms / 2468 tokens
|
||||
2.06.421.553 I slot print_timing: id 0 | task 140 | graphs reused = 49
|
||||
2.06.421.644 I slot release: id 0 | task 140 | stop processing: n_tokens = 2467, truncated = 0
|
||||
2.06.421.689 I srv update_slots: all slots are idle
|
||||
2.06.435.996 I srv server_strea: conv_id= (empty=1)
|
||||
2.06.437.968 I srv operator(): chat format: peg-native
|
||||
2.06.438.123 I slot get_availabl: id 0 | task -1 | - checking sim = 1.000 (2463/2463) > 0.100
|
||||
2.06.438.145 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, f_sim_best = 1.000 (> 0.100 thold), f_keep = 0.998
|
||||
2.06.438.183 I slot launch_slot_: id 0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> ?top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
|
||||
2.06.438.207 I slot launch_slot_: id 0 | task -1 | sampler params:
|
||||
repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
|
||||
dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 64
|
||||
top_k = 0, top_p = 0.950, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.000
|
||||
mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000, adaptive_target = -1.000, adaptive_decay = 0.900
|
||||
2.06.438.208 I slot launch_slot_: id 0 | task 149 | processing task, is_child = 0
|
||||
2.06.438.240 I slot operator(): id 0 | task 149 | new prompt, n_ctx_slot = 4096, n_keep = 0, task.n_tokens = 2463
|
||||
2.06.438.261 I slot operator(): id 0 | task 149 | erased invalidated context checkpoint (pos_min = 0, pos_max = 9, n_tokens = 10, n_swa = 512, pos_next = 0, size = 0.411 MiB)
|
||||
2.06.438.263 I slot operator(): id 0 | task 149 | erased invalidated context checkpoint (pos_min = 923, pos_max = 1946, n_tokens = 1947, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
2.06.438.265 I slot operator(): id 0 | task 149 | erased invalidated context checkpoint (pos_min = 1435, pos_max = 2458, n_tokens = 2459, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
2.06.441.182 I slot operator(): id 0 | task 149 | cached n_tokens = 0, memory_seq_rm [0, end)
|
||||
2.06.479.855 I slot operator(): id 0 | task 149 | cached n_tokens = 10, memory_seq_rm [10, end)
|
||||
2.06.480.018 I slot create_check: id 0 | task 149 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 9, n_tokens = 10, size = 0.411 MiB)
|
||||
2.12.502.363 I slot print_timing: id 0 | task 149 | prompt processing, n_tokens = 1947, progress = 0.79, t = 6.06 s / 321.07 tokens per second
|
||||
2.12.502.397 I slot operator(): id 0 | task 149 | cached n_tokens = 1947, memory_seq_rm [1947, end)
|
||||
2.12.511.803 I slot create_check: id 0 | task 149 | created context checkpoint 2 of 32 (pos_min = 923, pos_max = 1946, n_tokens = 1947, size = 21.006 MiB)
|
||||
2.14.172.396 I slot print_timing: id 0 | task 149 | prompt processing, n_tokens = 2459, progress = 1.00, t = 7.73 s / 317.94 tokens per second
|
||||
2.14.172.435 I slot operator(): id 0 | task 149 | cached n_tokens = 2459, memory_seq_rm [2459, end)
|
||||
2.14.172.781 I slot init_sampler: id 0 | task 149 | init sampler, took 0.32 ms, tokens: text = 2463, total = 2463
|
||||
2.14.188.349 I slot create_check: id 0 | task 149 | created context checkpoint 3 of 32 (pos_min = 1435, pos_max = 2458, n_tokens = 2459, size = 21.006 MiB)
|
||||
2.14.322.610 I slot print_timing: id 0 | task 149 | prompt eval time = 7784.85 ms / 2463 tokens ( 3.16 ms per token, 316.38 tokens per second)
|
||||
2.14.322.648 I slot print_timing: id 0 | task 149 | eval time = 99.49 ms / 5 tokens ( 24.87 ms per token, 40.20 tokens per second)
|
||||
2.14.322.649 I slot print_timing: id 0 | task 149 | total time = 7884.34 ms / 2468 tokens
|
||||
2.14.322.652 I slot print_timing: id 0 | task 149 | graphs reused = 52
|
||||
2.14.322.825 I slot release: id 0 | task 149 | stop processing: n_tokens = 2467, truncated = 0
|
||||
2.14.322.882 I srv update_slots: all slots are idle
|
||||
2.14.336.213 I srv server_strea: conv_id= (empty=1)
|
||||
2.14.338.345 I srv operator(): chat format: peg-native
|
||||
2.14.338.508 I slot get_availabl: id 0 | task -1 | - checking sim = 1.000 (2463/2463) > 0.100
|
||||
2.14.338.530 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, f_sim_best = 1.000 (> 0.100 thold), f_keep = 0.998
|
||||
2.14.338.570 I slot launch_slot_: id 0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> ?top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
|
||||
2.14.338.595 I slot launch_slot_: id 0 | task -1 | sampler params:
|
||||
repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
|
||||
dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 64
|
||||
top_k = 0, top_p = 0.950, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.000
|
||||
mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000, adaptive_target = -1.000, adaptive_decay = 0.900
|
||||
2.14.338.596 I slot launch_slot_: id 0 | task 158 | processing task, is_child = 0
|
||||
2.14.338.603 I slot operator(): id 0 | task 158 | new prompt, n_ctx_slot = 4096, n_keep = 0, task.n_tokens = 2463
|
||||
2.14.338.604 I slot operator(): id 0 | task 158 | erased invalidated context checkpoint (pos_min = 0, pos_max = 9, n_tokens = 10, n_swa = 512, pos_next = 0, size = 0.411 MiB)
|
||||
2.14.338.606 I slot operator(): id 0 | task 158 | erased invalidated context checkpoint (pos_min = 923, pos_max = 1946, n_tokens = 1947, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
2.14.338.608 I slot operator(): id 0 | task 158 | erased invalidated context checkpoint (pos_min = 1435, pos_max = 2458, n_tokens = 2459, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
2.14.342.006 I slot operator(): id 0 | task 158 | cached n_tokens = 0, memory_seq_rm [0, end)
|
||||
2.14.381.979 I slot operator(): id 0 | task 158 | cached n_tokens = 10, memory_seq_rm [10, end)
|
||||
2.14.382.169 I slot create_check: id 0 | task 158 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 9, n_tokens = 10, size = 0.411 MiB)
|
||||
2.20.338.510 I slot print_timing: id 0 | task 158 | prompt processing, n_tokens = 1947, progress = 0.79, t = 6.00 s / 324.51 tokens per second
|
||||
2.20.338.549 I slot operator(): id 0 | task 158 | cached n_tokens = 1947, memory_seq_rm [1947, end)
|
||||
2.20.348.601 I slot create_check: id 0 | task 158 | created context checkpoint 2 of 32 (pos_min = 923, pos_max = 1946, n_tokens = 1947, size = 21.006 MiB)
|
||||
2.22.006.356 I slot print_timing: id 0 | task 158 | prompt processing, n_tokens = 2459, progress = 1.00, t = 7.67 s / 320.70 tokens per second
|
||||
2.22.006.413 I slot operator(): id 0 | task 158 | cached n_tokens = 2459, memory_seq_rm [2459, end)
|
||||
2.22.006.966 I slot init_sampler: id 0 | task 158 | init sampler, took 0.52 ms, tokens: text = 2463, total = 2463
|
||||
2.22.017.502 I slot create_check: id 0 | task 158 | created context checkpoint 3 of 32 (pos_min = 1435, pos_max = 2458, n_tokens = 2459, size = 21.006 MiB)
|
||||
2.22.147.721 I slot print_timing: id 0 | task 158 | prompt eval time = 7711.88 ms / 2463 tokens ( 3.13 ms per token, 319.38 tokens per second)
|
||||
2.22.147.764 I slot print_timing: id 0 | task 158 | eval time = 97.20 ms / 5 tokens ( 24.30 ms per token, 41.15 tokens per second)
|
||||
2.22.147.765 I slot print_timing: id 0 | task 158 | total time = 7809.09 ms / 2468 tokens
|
||||
2.22.147.767 I slot print_timing: id 0 | task 158 | graphs reused = 55
|
||||
2.22.147.859 I slot release: id 0 | task 158 | stop processing: n_tokens = 2467, truncated = 0
|
||||
2.22.147.906 I srv update_slots: all slots are idle
|
||||
2.22.160.348 I srv server_strea: conv_id= (empty=1)
|
||||
2.22.162.402 I srv operator(): chat format: peg-native
|
||||
2.22.162.558 I slot get_availabl: id 0 | task -1 | - checking sim = 1.000 (2463/2463) > 0.100
|
||||
2.22.162.580 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, f_sim_best = 1.000 (> 0.100 thold), f_keep = 0.998
|
||||
2.22.162.621 I slot launch_slot_: id 0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> ?top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
|
||||
2.22.162.646 I slot launch_slot_: id 0 | task -1 | sampler params:
|
||||
repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
|
||||
dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 64
|
||||
top_k = 0, top_p = 0.950, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.000
|
||||
mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000, adaptive_target = -1.000, adaptive_decay = 0.900
|
||||
2.22.162.647 I slot launch_slot_: id 0 | task 167 | processing task, is_child = 0
|
||||
2.22.162.653 I slot operator(): id 0 | task 167 | new prompt, n_ctx_slot = 4096, n_keep = 0, task.n_tokens = 2463
|
||||
2.22.162.655 I slot operator(): id 0 | task 167 | erased invalidated context checkpoint (pos_min = 0, pos_max = 9, n_tokens = 10, n_swa = 512, pos_next = 0, size = 0.411 MiB)
|
||||
2.22.162.656 I slot operator(): id 0 | task 167 | erased invalidated context checkpoint (pos_min = 923, pos_max = 1946, n_tokens = 1947, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
2.22.162.658 I slot operator(): id 0 | task 167 | erased invalidated context checkpoint (pos_min = 1435, pos_max = 2458, n_tokens = 2459, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
2.22.165.451 I slot operator(): id 0 | task 167 | cached n_tokens = 0, memory_seq_rm [0, end)
|
||||
2.22.205.719 I slot operator(): id 0 | task 167 | cached n_tokens = 10, memory_seq_rm [10, end)
|
||||
2.22.205.912 I slot create_check: id 0 | task 167 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 9, n_tokens = 10, size = 0.411 MiB)
|
||||
2.28.093.543 I slot print_timing: id 0 | task 167 | prompt processing, n_tokens = 1947, progress = 0.79, t = 5.93 s / 328.28 tokens per second
|
||||
2.28.093.580 I slot operator(): id 0 | task 167 | cached n_tokens = 1947, memory_seq_rm [1947, end)
|
||||
2.28.103.164 I slot create_check: id 0 | task 167 | created context checkpoint 2 of 32 (pos_min = 923, pos_max = 1946, n_tokens = 1947, size = 21.006 MiB)
|
||||
2.29.771.570 I slot print_timing: id 0 | task 167 | prompt processing, n_tokens = 2459, progress = 1.00, t = 7.61 s / 323.17 tokens per second
|
||||
2.29.771.609 I slot operator(): id 0 | task 167 | cached n_tokens = 2459, memory_seq_rm [2459, end)
|
||||
2.29.771.950 I slot init_sampler: id 0 | task 167 | init sampler, took 0.31 ms, tokens: text = 2463, total = 2463
|
||||
2.29.787.518 I slot create_check: id 0 | task 167 | created context checkpoint 3 of 32 (pos_min = 1435, pos_max = 2458, n_tokens = 2459, size = 21.006 MiB)
|
||||
2.29.923.537 I slot print_timing: id 0 | task 167 | prompt eval time = 7660.73 ms / 2463 tokens ( 3.11 ms per token, 321.51 tokens per second)
|
||||
2.29.923.568 I slot print_timing: id 0 | task 167 | eval time = 100.13 ms / 5 tokens ( 25.03 ms per token, 39.95 tokens per second)
|
||||
2.29.923.569 I slot print_timing: id 0 | task 167 | total time = 7760.86 ms / 2468 tokens
|
||||
2.29.923.570 I slot print_timing: id 0 | task 167 | graphs reused = 58
|
||||
2.29.923.640 I slot release: id 0 | task 167 | stop processing: n_tokens = 2467, truncated = 0
|
||||
2.29.923.656 I srv update_slots: all slots are idle
|
||||
2.29.937.039 I srv server_strea: conv_id= (empty=1)
|
||||
2.29.939.019 I srv operator(): chat format: peg-native
|
||||
2.29.939.169 I slot get_availabl: id 0 | task -1 | - checking sim = 1.000 (2463/2463) > 0.100
|
||||
2.29.939.190 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, f_sim_best = 1.000 (> 0.100 thold), f_keep = 0.998
|
||||
2.29.939.219 I slot launch_slot_: id 0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> ?top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
|
||||
2.29.939.243 I slot launch_slot_: id 0 | task -1 | sampler params:
|
||||
repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
|
||||
dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 64
|
||||
top_k = 0, top_p = 0.950, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.000
|
||||
mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000, adaptive_target = -1.000, adaptive_decay = 0.900
|
||||
2.29.939.244 I slot launch_slot_: id 0 | task 176 | processing task, is_child = 0
|
||||
2.29.939.248 I slot operator(): id 0 | task 176 | new prompt, n_ctx_slot = 4096, n_keep = 0, task.n_tokens = 2463
|
||||
2.29.939.250 I slot operator(): id 0 | task 176 | erased invalidated context checkpoint (pos_min = 0, pos_max = 9, n_tokens = 10, n_swa = 512, pos_next = 0, size = 0.411 MiB)
|
||||
2.29.939.251 I slot operator(): id 0 | task 176 | erased invalidated context checkpoint (pos_min = 923, pos_max = 1946, n_tokens = 1947, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
2.29.939.253 I slot operator(): id 0 | task 176 | erased invalidated context checkpoint (pos_min = 1435, pos_max = 2458, n_tokens = 2459, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
2.29.942.121 I slot operator(): id 0 | task 176 | cached n_tokens = 0, memory_seq_rm [0, end)
|
||||
2.29.982.668 I slot operator(): id 0 | task 176 | cached n_tokens = 10, memory_seq_rm [10, end)
|
||||
2.29.982.803 I slot create_check: id 0 | task 176 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 9, n_tokens = 10, size = 0.411 MiB)
|
||||
2.35.895.620 I slot print_timing: id 0 | task 176 | prompt processing, n_tokens = 1947, progress = 0.79, t = 5.96 s / 326.88 tokens per second
|
||||
2.35.895.653 I slot operator(): id 0 | task 176 | cached n_tokens = 1947, memory_seq_rm [1947, end)
|
||||
2.35.904.453 I slot create_check: id 0 | task 176 | created context checkpoint 2 of 32 (pos_min = 923, pos_max = 1946, n_tokens = 1947, size = 21.006 MiB)
|
||||
2.37.560.896 I slot print_timing: id 0 | task 176 | prompt processing, n_tokens = 2459, progress = 1.00, t = 7.62 s / 322.64 tokens per second
|
||||
2.37.560.923 I slot operator(): id 0 | task 176 | cached n_tokens = 2459, memory_seq_rm [2459, end)
|
||||
2.37.561.179 I slot init_sampler: id 0 | task 176 | init sampler, took 0.23 ms, tokens: text = 2463, total = 2463
|
||||
2.37.571.042 I slot create_check: id 0 | task 176 | created context checkpoint 3 of 32 (pos_min = 1435, pos_max = 2458, n_tokens = 2459, size = 21.006 MiB)
|
||||
2.37.704.659 I slot print_timing: id 0 | task 176 | prompt eval time = 7665.90 ms / 2463 tokens ( 3.11 ms per token, 321.29 tokens per second)
|
||||
2.37.704.694 I slot print_timing: id 0 | task 176 | eval time = 99.48 ms / 5 tokens ( 24.87 ms per token, 40.21 tokens per second)
|
||||
2.37.704.695 I slot print_timing: id 0 | task 176 | total time = 7765.38 ms / 2468 tokens
|
||||
2.37.704.697 I slot print_timing: id 0 | task 176 | graphs reused = 61
|
||||
2.37.704.796 I slot release: id 0 | task 176 | stop processing: n_tokens = 2467, truncated = 0
|
||||
2.37.704.835 I srv update_slots: all slots are idle
|
||||
2.37.718.631 I srv server_strea: conv_id= (empty=1)
|
||||
2.37.720.658 I srv operator(): chat format: peg-native
|
||||
2.37.720.800 I slot get_availabl: id 0 | task -1 | - checking sim = 1.000 (2463/2463) > 0.100
|
||||
2.37.720.821 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, f_sim_best = 1.000 (> 0.100 thold), f_keep = 0.998
|
||||
2.37.720.853 I slot launch_slot_: id 0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> ?top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
|
||||
2.37.720.877 I slot launch_slot_: id 0 | task -1 | sampler params:
|
||||
repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
|
||||
dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 64
|
||||
top_k = 0, top_p = 0.950, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.000
|
||||
mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000, adaptive_target = -1.000, adaptive_decay = 0.900
|
||||
2.37.720.878 I slot launch_slot_: id 0 | task 185 | processing task, is_child = 0
|
||||
2.37.720.883 I slot operator(): id 0 | task 185 | new prompt, n_ctx_slot = 4096, n_keep = 0, task.n_tokens = 2463
|
||||
2.37.720.884 I slot operator(): id 0 | task 185 | erased invalidated context checkpoint (pos_min = 0, pos_max = 9, n_tokens = 10, n_swa = 512, pos_next = 0, size = 0.411 MiB)
|
||||
2.37.720.886 I slot operator(): id 0 | task 185 | erased invalidated context checkpoint (pos_min = 923, pos_max = 1946, n_tokens = 1947, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
2.37.720.887 I slot operator(): id 0 | task 185 | erased invalidated context checkpoint (pos_min = 1435, pos_max = 2458, n_tokens = 2459, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
2.37.724.234 I slot operator(): id 0 | task 185 | cached n_tokens = 0, memory_seq_rm [0, end)
|
||||
2.37.763.006 I slot operator(): id 0 | task 185 | cached n_tokens = 10, memory_seq_rm [10, end)
|
||||
2.37.763.162 I slot create_check: id 0 | task 185 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 9, n_tokens = 10, size = 0.411 MiB)
|
||||
2.43.739.683 I slot print_timing: id 0 | task 185 | prompt processing, n_tokens = 1947, progress = 0.79, t = 6.02 s / 323.49 tokens per second
|
||||
2.43.739.720 I slot operator(): id 0 | task 185 | cached n_tokens = 1947, memory_seq_rm [1947, end)
|
||||
2.43.750.150 I slot create_check: id 0 | task 185 | created context checkpoint 2 of 32 (pos_min = 923, pos_max = 1946, n_tokens = 1947, size = 21.006 MiB)
|
||||
2.45.402.302 I slot print_timing: id 0 | task 185 | prompt processing, n_tokens = 2459, progress = 1.00, t = 7.68 s / 320.13 tokens per second
|
||||
2.45.402.359 I slot operator(): id 0 | task 185 | cached n_tokens = 2459, memory_seq_rm [2459, end)
|
||||
2.45.402.912 I slot init_sampler: id 0 | task 185 | init sampler, took 0.52 ms, tokens: text = 2463, total = 2463
|
||||
2.45.413.993 I slot create_check: id 0 | task 185 | created context checkpoint 3 of 32 (pos_min = 1435, pos_max = 2458, n_tokens = 2459, size = 21.006 MiB)
|
||||
2.45.546.799 I slot print_timing: id 0 | task 185 | prompt eval time = 7726.31 ms / 2463 tokens ( 3.14 ms per token, 318.78 tokens per second)
|
||||
2.45.546.841 I slot print_timing: id 0 | task 185 | eval time = 99.58 ms / 5 tokens ( 24.89 ms per token, 40.17 tokens per second)
|
||||
2.45.546.842 I slot print_timing: id 0 | task 185 | total time = 7825.89 ms / 2468 tokens
|
||||
2.45.546.844 I slot print_timing: id 0 | task 185 | graphs reused = 64
|
||||
2.45.546.939 I slot release: id 0 | task 185 | stop processing: n_tokens = 2467, truncated = 0
|
||||
2.45.546.985 I srv update_slots: all slots are idle
|
||||
2.45.558.480 I srv server_strea: conv_id= (empty=1)
|
||||
2.45.560.410 I srv operator(): chat format: peg-native
|
||||
2.45.560.609 I slot get_availabl: id 0 | task -1 | - checking sim = 1.000 (2463/2463) > 0.100
|
||||
2.45.560.670 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, f_sim_best = 1.000 (> 0.100 thold), f_keep = 0.998
|
||||
2.45.560.793 I slot launch_slot_: id 0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> ?top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
|
||||
2.45.560.912 I slot launch_slot_: id 0 | task -1 | sampler params:
|
||||
repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
|
||||
dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 64
|
||||
top_k = 0, top_p = 0.950, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.000
|
||||
mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000, adaptive_target = -1.000, adaptive_decay = 0.900
|
||||
2.45.561.019 I slot launch_slot_: id 0 | task 194 | processing task, is_child = 0
|
||||
2.45.561.085 I slot operator(): id 0 | task 194 | new prompt, n_ctx_slot = 4096, n_keep = 0, task.n_tokens = 2463
|
||||
2.45.561.195 I slot operator(): id 0 | task 194 | erased invalidated context checkpoint (pos_min = 0, pos_max = 9, n_tokens = 10, n_swa = 512, pos_next = 0, size = 0.411 MiB)
|
||||
2.45.561.255 I slot operator(): id 0 | task 194 | erased invalidated context checkpoint (pos_min = 923, pos_max = 1946, n_tokens = 1947, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
2.45.561.315 I slot operator(): id 0 | task 194 | erased invalidated context checkpoint (pos_min = 1435, pos_max = 2458, n_tokens = 2459, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
2.45.564.249 I slot operator(): id 0 | task 194 | cached n_tokens = 0, memory_seq_rm [0, end)
|
||||
2.45.602.404 I slot operator(): id 0 | task 194 | cached n_tokens = 10, memory_seq_rm [10, end)
|
||||
2.45.602.591 I slot create_check: id 0 | task 194 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 9, n_tokens = 10, size = 0.411 MiB)
|
||||
2.51.473.855 I slot print_timing: id 0 | task 194 | prompt processing, n_tokens = 1947, progress = 0.79, t = 5.91 s / 329.29 tokens per second
|
||||
2.51.473.897 I slot operator(): id 0 | task 194 | cached n_tokens = 1947, memory_seq_rm [1947, end)
|
||||
2.51.483.840 I slot create_check: id 0 | task 194 | created context checkpoint 2 of 32 (pos_min = 923, pos_max = 1946, n_tokens = 1947, size = 21.006 MiB)
|
||||
2.53.119.479 I slot print_timing: id 0 | task 194 | prompt processing, n_tokens = 2459, progress = 1.00, t = 7.56 s / 325.34 tokens per second
|
||||
2.53.119.536 I slot operator(): id 0 | task 194 | cached n_tokens = 2459, memory_seq_rm [2459, end)
|
||||
2.53.120.086 I slot init_sampler: id 0 | task 194 | init sampler, took 0.52 ms, tokens: text = 2463, total = 2463
|
||||
2.53.132.150 I slot create_check: id 0 | task 194 | created context checkpoint 3 of 32 (pos_min = 1435, pos_max = 2458, n_tokens = 2459, size = 21.006 MiB)
|
||||
2.53.268.585 I slot print_timing: id 0 | task 194 | prompt eval time = 7605.86 ms / 2463 tokens ( 3.09 ms per token, 323.83 tokens per second)
|
||||
2.53.268.633 I slot print_timing: id 0 | task 194 | eval time = 101.61 ms / 5 tokens ( 25.40 ms per token, 39.37 tokens per second)
|
||||
2.53.268.634 I slot print_timing: id 0 | task 194 | total time = 7707.47 ms / 2468 tokens
|
||||
2.53.268.636 I slot print_timing: id 0 | task 194 | graphs reused = 67
|
||||
2.53.268.729 I slot release: id 0 | task 194 | stop processing: n_tokens = 2467, truncated = 0
|
||||
2.53.268.758 I srv update_slots: all slots are idle
|
||||
2.53.296.443 I srv server_strea: conv_id= (empty=1)
|
||||
2.53.296.752 I srv operator(): chat format: peg-native
|
||||
2.53.296.955 I slot get_availabl: id 0 | task -1 | - checking sim = 0.429 (12/28) > 0.100
|
||||
2.53.297.013 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, f_sim_best = 0.429 (> 0.100 thold), f_keep = 0.005
|
||||
2.53.297.131 I slot launch_slot_: id 0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> ?top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
|
||||
2.53.297.163 I slot launch_slot_: id 0 | task -1 | sampler params:
|
||||
repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
|
||||
dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 64
|
||||
top_k = 0, top_p = 0.950, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.000
|
||||
mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000, adaptive_target = -1.000, adaptive_decay = 0.900
|
||||
2.53.297.164 I slot launch_slot_: id 0 | task 203 | processing task, is_child = 0
|
||||
2.53.297.170 I slot operator(): id 0 | task 203 | new prompt, n_ctx_slot = 4096, n_keep = 0, task.n_tokens = 28
|
||||
2.53.297.171 I slot operator(): id 0 | task 203 | erased invalidated context checkpoint (pos_min = 0, pos_max = 9, n_tokens = 10, n_swa = 512, pos_next = 0, size = 0.411 MiB)
|
||||
2.53.297.173 I slot operator(): id 0 | task 203 | erased invalidated context checkpoint (pos_min = 923, pos_max = 1946, n_tokens = 1947, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
2.53.297.175 I slot operator(): id 0 | task 203 | erased invalidated context checkpoint (pos_min = 1435, pos_max = 2458, n_tokens = 2459, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
2.53.299.939 I slot operator(): id 0 | task 203 | cached n_tokens = 0, memory_seq_rm [0, end)
|
||||
2.53.336.981 I slot operator(): id 0 | task 203 | cached n_tokens = 10, memory_seq_rm [10, end)
|
||||
2.53.337.138 I slot create_check: id 0 | task 203 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 9, n_tokens = 10, size = 0.411 MiB)
|
||||
2.53.387.138 I slot operator(): id 0 | task 203 | cached n_tokens = 24, memory_seq_rm [24, end)
|
||||
2.53.387.205 I slot init_sampler: id 0 | task 203 | init sampler, took 0.01 ms, tokens: text = 28, total = 28
|
||||
2.53.387.353 I slot create_check: id 0 | task 203 | created context checkpoint 2 of 32 (pos_min = 0, pos_max = 23, n_tokens = 24, size = 0.985 MiB)
|
||||
2.53.432.365 I slot print_timing: id 0 | task 203 | prompt eval time = 114.85 ms / 28 tokens ( 4.10 ms per token, 243.80 tokens per second)
|
||||
2.53.432.400 I slot print_timing: id 0 | task 203 | eval time = 20.32 ms / 2 tokens ( 20.32 ms per token, 49.22 tokens per second)
|
||||
2.53.432.401 I slot print_timing: id 0 | task 203 | total time = 135.17 ms / 30 tokens
|
||||
2.53.432.402 I slot print_timing: id 0 | task 203 | graphs reused = 67
|
||||
2.53.432.478 I slot release: id 0 | task 203 | stop processing: n_tokens = 29, truncated = 0
|
||||
2.53.432.498 I srv update_slots: all slots are idle
|
||||
2.53.443.129 I srv server_strea: conv_id= (empty=1)
|
||||
2.53.443.434 I srv operator(): chat format: peg-native
|
||||
2.53.443.575 I slot get_availabl: id 0 | task -1 | - checking sim = 0.444 (12/27) > 0.100
|
||||
2.53.443.595 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, f_sim_best = 0.444 (> 0.100 thold), f_keep = 0.414
|
||||
2.53.443.625 I slot launch_slot_: id 0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> ?top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
|
||||
2.53.443.653 I slot launch_slot_: id 0 | task -1 | sampler params:
|
||||
repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
|
||||
dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 64
|
||||
top_k = 0, top_p = 0.950, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.000
|
||||
mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000, adaptive_target = -1.000, adaptive_decay = 0.900
|
||||
2.53.443.655 I slot launch_slot_: id 0 | task 208 | processing task, is_child = 0
|
||||
2.53.443.660 I slot operator(): id 0 | task 208 | new prompt, n_ctx_slot = 4096, n_keep = 0, task.n_tokens = 27
|
||||
2.53.443.661 I slot operator(): id 0 | task 208 | erased invalidated context checkpoint (pos_min = 0, pos_max = 9, n_tokens = 10, n_swa = 512, pos_next = 0, size = 0.411 MiB)
|
||||
2.53.443.663 I slot operator(): id 0 | task 208 | erased invalidated context checkpoint (pos_min = 0, pos_max = 23, n_tokens = 24, n_swa = 512, pos_next = 0, size = 0.985 MiB)
|
||||
2.53.443.663 I slot operator(): id 0 | task 208 | cached n_tokens = 0, memory_seq_rm [0, end)
|
||||
2.53.481.803 I srv operator(): operator(): cleaning up before exit...
|
||||
+48
@@ -0,0 +1,48 @@
|
||||
INFO 09-24 05:00:07 [patch.py:259] NVFP4 W4A4 weight_scale NaN-clamp: installed.
|
||||
INFO 09-24 05:00:07 [patch.py:510] inductor factorable-divisibility patch: installed.
|
||||
INFO 09-24 05:00:07 [patch.py:589] [cumem-cuda] CuMemAllocator._python_free_callback patched: asleep guard extended to all platforms.
|
||||
INFO 09-24 05:00:08 [omni_base.py:216] [AsyncOmni] Initializing with model /home/zhout/project/edge_infer/models/Spark-X2.5-1.7B-BF16
|
||||
INFO 09-24 05:00:08 [async_omni_engine.py:172] [AsyncOmniEngine] Initializing with model /home/zhout/project/edge_infer/models/Spark-X2.5-1.7B-BF16
|
||||
ERROR 09-24 05:00:08 [repo_utils.py:119] Error parsing config for /home/zhout/project/edge_infer/models/Spark-X2.5-1.7B-BF16: The repository /home/zhout/project/edge_infer/models/Spark-X2.5-1.7B-BF16 contains custom code which must be executed to correctly load the model. You can inspect the repository content at /home/zhout/project/edge_infer/models/Spark-X2.5-1.7B-BF16 .
|
||||
ERROR 09-24 05:00:08 [repo_utils.py:119] You can inspect the repository content at https://hf.co//home/zhout/project/edge_infer/models/Spark-X2.5-1.7B-BF16.
|
||||
ERROR 09-24 05:00:08 [repo_utils.py:119] Please pass the argument `trust_remote_code=True` to allow custom code to be run., retrying 1 of 2
|
||||
ERROR 09-24 05:00:10 [repo_utils.py:117] Error parsing config for /home/zhout/project/edge_infer/models/Spark-X2.5-1.7B-BF16: The repository /home/zhout/project/edge_infer/models/Spark-X2.5-1.7B-BF16 contains custom code which must be executed to correctly load the model. You can inspect the repository content at /home/zhout/project/edge_infer/models/Spark-X2.5-1.7B-BF16 .
|
||||
ERROR 09-24 05:00:10 [repo_utils.py:117] You can inspect the repository content at https://hf.co//home/zhout/project/edge_infer/models/Spark-X2.5-1.7B-BF16.
|
||||
ERROR 09-24 05:00:10 [repo_utils.py:117] Please pass the argument `trust_remote_code=True` to allow custom code to be run.
|
||||
INFO 09-24 05:00:10 [async_omni_engine.py:290] [AsyncOmniEngine] Launching Orchestrator thread with 1 stages
|
||||
INFO 09-24 05:00:10 [stage_init_utils.py:768] [stage_init] Set VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
INFO 09-24 05:00:10 [initialization.py:356] Loaded OmniTransferConfig with 0 connector configurations
|
||||
INFO 09-24 05:00:12 [orchestrator.py:604] [Orchestrator] Starting event loop
|
||||
INFO 09-24 05:00:12 [async_omni_engine.py:323] [AsyncOmniEngine] Orchestrator ready with 1 stages
|
||||
INFO 09-24 05:00:12 [omni_base.py:235] [AsyncOmni] AsyncOmniEngine initialized in 3.40 seconds
|
||||
INFO 09-24 05:00:12 [omni_base.py:255] [AsyncOmni] Initialized with 1 stages for model /home/zhout/project/edge_infer/models/Spark-X2.5-1.7B-BF16
|
||||
INFO 09-24 05:00:12 [omni_base.py:781] [AsyncOmni] Shutting down
|
||||
INFO 09-24 05:00:12 [async_omni_engine.py:1807] [AsyncOmniEngine] Shutting down Orchestrator
|
||||
INFO 09-24 05:00:12 [orchestrator.py:705] [Orchestrator] Received shutdown signal
|
||||
INFO 09-24 05:00:12 [orchestrator.py:3092] [Orchestrator] Shutting down all 1 client(s)
|
||||
INFO 09-24 05:00:12 [stage_pool.py:1354] [StagePool] Stage 0 replica 0 shut down
|
||||
{
|
||||
"device": "cpu",
|
||||
"entrypoint": "AsyncOmni.generate",
|
||||
"startup_s": 3.3993976110359654,
|
||||
"request_wall_s": 0.14475277299061418,
|
||||
"outputs": [
|
||||
{
|
||||
"request_id": "public-spark-1",
|
||||
"text": "Paris",
|
||||
"error": null,
|
||||
"stage_event": {
|
||||
"request_id": "public-spark-1-8212fa08",
|
||||
"stage_id": 0,
|
||||
"epoch": 1,
|
||||
"seq": 1,
|
||||
"kind": "text",
|
||||
"worker_generation": "c63917cdd3ac4552b6d2756c8bc7b8fd",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
}
|
||||
}
|
||||
],
|
||||
"status": "passed"
|
||||
}
|
||||
+25
@@ -0,0 +1,25 @@
|
||||
{
|
||||
"device": "cpu",
|
||||
"entrypoint": "AsyncOmni.generate",
|
||||
"startup_s": 3.3993976110359654,
|
||||
"request_wall_s": 0.14475277299061418,
|
||||
"outputs": [
|
||||
{
|
||||
"request_id": "public-spark-1",
|
||||
"text": "Paris",
|
||||
"error": null,
|
||||
"stage_event": {
|
||||
"request_id": "public-spark-1-8212fa08",
|
||||
"stage_id": 0,
|
||||
"epoch": 1,
|
||||
"seq": 1,
|
||||
"kind": "text",
|
||||
"worker_generation": "c63917cdd3ac4552b6d2756c8bc7b8fd",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
}
|
||||
}
|
||||
],
|
||||
"status": "passed"
|
||||
}
|
||||
+220
@@ -0,0 +1,220 @@
|
||||
warning: no usable GPU found, --gpu-layers option will be ignored
|
||||
warning: one possible reason is that llama.cpp was compiled without GPU support
|
||||
warning: consult docs/build.md for compilation instructions
|
||||
0.00.001.987 I cmn common_param: common_params_print_info: build 10849 (e71b80510) with GNU 11.4.0 for Linux x86_64
|
||||
0.00.002.008 I cmn common_param: common_params_print_info: verbosity = 4 (adjust with the `-lv N` CLI arg)
|
||||
0.00.002.009 I cmn common_param: device_info:
|
||||
0.00.002.020 I cmn common_param: - CPU : AMD Ryzen AI 9 HX 370 w/ Radeon 890M (31647 MiB, 31647 MiB free)
|
||||
0.00.002.083 I cmn common_param: system_info: n_threads = 12 (n_threads_batch = 12) / 24 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX_VNNI = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | AMX_INT8 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
|
||||
0.00.002.170 I srv init: using 23 threads for HTTP server
|
||||
0.00.002.199 I srv init: The UI is disabled
|
||||
0.00.002.200 I srv init: Use --ui/--no-ui (or deprecated --webui/--no-webui) to enable/disable
|
||||
0.00.002.249 W srv llama_server: -----------------
|
||||
0.00.002.266 W srv llama_server: CORS is set to allow all origins ('*') and no API key is set
|
||||
0.00.002.267 W srv llama_server: this can be a security risk (cross-origin attacks)
|
||||
0.00.002.267 W srv llama_server: more info: https://github.com/ggml-org/llama.cpp/pull/25655
|
||||
0.00.002.267 W srv llama_server: -----------------
|
||||
0.00.002.285 I srv start: binding port with default address family
|
||||
0.00.003.811 I srv load_model: loading model '/home/zhout/project/edge_infer/models/Spark-X2.5-1.7B-GGUF/Spark-X2.5-1.7B-Q4_K_M.gguf'
|
||||
0.00.003.869 I srv load_model: local path '/home/zhout/project/edge_infer/models/Spark-X2.5-1.7B-GGUF/Spark-X2.5-1.7B-Q4_K_M.gguf'
|
||||
0.00.023.716 I llama_model_loader: loaded meta data with 37 key-value pairs and 226 tensors from /home/zhout/project/edge_infer/models/Spark-X2.5-1.7B-GGUF/Spark-X2.5-1.7B-Q4_K_M.gguf (version GGUF V3 (latest))
|
||||
0.00.023.751 I llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
0.00.023.758 I llama_model_loader: - kv 0: general.architecture str = spark2_5
|
||||
0.00.023.759 I llama_model_loader: - kv 1: general.type str = model
|
||||
0.00.023.761 I llama_model_loader: - kv 2: general.sampling.top_k i32 = 0
|
||||
0.00.023.766 I llama_model_loader: - kv 3: general.sampling.top_p f32 = 0.950000
|
||||
0.00.023.767 I llama_model_loader: - kv 4: general.sampling.temp f32 = 1.000000
|
||||
0.00.023.767 I llama_model_loader: - kv 5: general.name str = Hf_Format
|
||||
0.00.023.767 I llama_model_loader: - kv 6: general.size_label str = 1.7B
|
||||
0.00.023.768 I llama_model_loader: - kv 7: spark2_5.block_count u32 = 28
|
||||
0.00.023.768 I llama_model_loader: - kv 8: spark2_5.context_length u32 = 1048576
|
||||
0.00.023.769 I llama_model_loader: - kv 9: spark2_5.embedding_length u32 = 2048
|
||||
0.00.023.769 I llama_model_loader: - kv 10: spark2_5.feed_forward_length u32 = 6656
|
||||
0.00.023.769 I llama_model_loader: - kv 11: spark2_5.attention.head_count u32 = 8
|
||||
0.00.023.769 I llama_model_loader: - kv 12: spark2_5.attention.head_count_kv u32 = 2
|
||||
0.00.023.771 I llama_model_loader: - kv 13: spark2_5.rope.freq_base f32 = 5000000.000000
|
||||
0.00.023.771 I llama_model_loader: - kv 14: spark2_5.rope.freq_base_swa f32 = 10000.000000
|
||||
0.00.023.772 I llama_model_loader: - kv 15: spark2_5.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
0.00.023.772 I llama_model_loader: - kv 16: spark2_5.attention.key_length u32 = 256
|
||||
0.00.023.772 I llama_model_loader: - kv 17: spark2_5.attention.value_length u32 = 256
|
||||
0.00.023.773 I llama_model_loader: - kv 18: spark2_5.vocab_size u32 = 131072
|
||||
0.00.023.773 I llama_model_loader: - kv 19: spark2_5.attention.sliding_window u32 = 512
|
||||
0.00.023.789 I llama_model_loader: - kv 20: spark2_5.attention.sliding_window_pattern arr[bool,28] = [true, true, true, false, true, true,...
|
||||
0.00.023.789 I llama_model_loader: - kv 21: spark2_5.rope.dimension_count u32 = 64
|
||||
0.00.023.789 I llama_model_loader: - kv 22: spark2_5.rope.dimension_count_swa u32 = 256
|
||||
0.00.023.789 I llama_model_loader: - kv 23: tokenizer.ggml.model str = gpt2
|
||||
0.00.023.790 I llama_model_loader: - kv 24: tokenizer.ggml.pre str = spark2_5
|
||||
0.00.031.446 I llama_model_loader: - kv 25: tokenizer.ggml.tokens arr[str,131072] = ["<｜startâ–�ofâ–�sentence｜>", "<ï...
|
||||
0.00.033.132 I llama_model_loader: - kv 26: tokenizer.ggml.token_type arr[i32,131072] = [3, 3, 3, 4, 4, 3, 3, 3, 4, 4, 4, 3, ...
|
||||
0.00.040.276 I llama_model_loader: - kv 27: tokenizer.ggml.merges arr[str,130716] = ["Ä Ä ", "Ä t", "i n", "Ä a", "Ä Ä...
|
||||
0.00.040.299 I llama_model_loader: - kv 28: tokenizer.ggml.bos_token_id u32 = 0
|
||||
0.00.040.299 I llama_model_loader: - kv 29: tokenizer.ggml.eos_token_id u32 = 1
|
||||
0.00.040.300 I llama_model_loader: - kv 30: tokenizer.ggml.unknown_token_id u32 = 5
|
||||
0.00.040.300 I llama_model_loader: - kv 31: tokenizer.ggml.padding_token_id u32 = 2
|
||||
0.00.040.300 I llama_model_loader: - kv 32: tokenizer.ggml.add_bos_token bool = false
|
||||
0.00.040.301 I llama_model_loader: - kv 33: tokenizer.ggml.add_eos_token bool = false
|
||||
0.00.040.303 I llama_model_loader: - kv 34: tokenizer.chat_template str = {%- if not messages %}\n {{- raise_...
|
||||
0.00.040.303 I llama_model_loader: - kv 35: general.quantization_version u32 = 2
|
||||
0.00.040.304 I llama_model_loader: - kv 36: general.file_type u32 = 15
|
||||
0.00.040.307 I llama_model_loader: - type f32: 57 tensors
|
||||
0.00.040.308 I llama_model_loader: - type q4_K: 140 tensors
|
||||
0.00.040.308 I llama_model_loader: - type q6_K: 29 tensors
|
||||
0.00.040.311 I print_info: file format = GGUF V3 (latest)
|
||||
0.00.040.313 I print_info: file type = Q4_K - Medium
|
||||
0.00.040.315 I print_info: file size = 1.03 GiB (5.16 BPW)
|
||||
0.00.092.297 I load: 86 unused tokens
|
||||
0.00.109.781 I load: printing all EOG tokens:
|
||||
0.00.109.806 I load: - 1 ('<｜end�of�sentence｜>')
|
||||
0.00.109.916 I load: special tokens cache size = 113
|
||||
0.00.132.423 I load: token to piece cache size = 0.7975 MB
|
||||
0.00.132.473 I print_info: arch = spark2_5
|
||||
0.00.132.473 I print_info: vocab_only = 0
|
||||
0.00.132.474 I print_info: no_alloc = 0
|
||||
0.00.132.474 I print_info: n_ctx_train = 1048576
|
||||
0.00.132.476 I print_info: n_embd_inp = 2048
|
||||
0.00.132.476 I print_info: n_embd = 2048
|
||||
0.00.132.477 I print_info: n_embd_out = 2048
|
||||
0.00.132.477 I print_info: n_layer = 28
|
||||
0.00.132.477 I print_info: n_layer_all = 28
|
||||
0.00.132.492 I print_info: n_head = 8
|
||||
0.00.132.494 I print_info: n_head_kv = 2
|
||||
0.00.132.494 I print_info: n_rot = 64
|
||||
0.00.132.494 I print_info: n_swa = 512
|
||||
0.00.132.496 I print_info: is_swa_any = 1
|
||||
0.00.132.496 I print_info: non_causal_type = 0
|
||||
0.00.132.496 I print_info: n_embd_head_k = 256
|
||||
0.00.132.496 I print_info: n_embd_head_v = 256
|
||||
0.00.132.498 I print_info: n_gqa = 4
|
||||
0.00.132.502 I print_info: n_embd_k_gqa = 512
|
||||
0.00.132.505 I print_info: n_embd_v_gqa = 512
|
||||
0.00.132.506 I print_info: f_norm_eps = 0.0e+00
|
||||
0.00.132.507 I print_info: f_norm_rms_eps = 1.0e-06
|
||||
0.00.132.507 I print_info: f_clamp_kqv = 0.0e+00
|
||||
0.00.132.507 I print_info: f_max_alibi_bias = 0.0e+00
|
||||
0.00.132.507 I print_info: f_logit_scale = 0.0e+00
|
||||
0.00.132.507 I print_info: f_attn_scale = 0.0e+00
|
||||
0.00.132.508 I print_info: f_attn_value_scale = 0.0000
|
||||
0.00.132.509 I print_info: n_ff = 6656
|
||||
0.00.132.509 I print_info: n_expert = 0
|
||||
0.00.132.510 I print_info: n_expert_used = 0
|
||||
0.00.132.510 I print_info: n_expert_groups = 0
|
||||
0.00.132.510 I print_info: n_group_used = 0
|
||||
0.00.132.510 I print_info: causal attn = 1
|
||||
0.00.132.510 I print_info: pooling type = -1
|
||||
0.00.132.510 I print_info: rope type = 2
|
||||
0.00.132.511 I print_info: rope scaling = linear
|
||||
0.00.132.512 AMX is not ready to be used!
|
||||
I print_info: freq_base_train = 5000000.0
|
||||
0.00.132.512 I print_info: freq_scale_train = 1
|
||||
0.00.132.512 I print_info: freq_base_swa = 10000.0
|
||||
0.00.132.512 I print_info: freq_scale_swa = 1
|
||||
0.00.132.513 I print_info: n_embd_head_k_swa = 256
|
||||
0.00.132.513 I print_info: n_embd_head_v_swa = 256
|
||||
0.00.132.513 I print_info: n_rot_swa = 256
|
||||
0.00.132.513 I print_info: n_ctx_orig_yarn = 1048576
|
||||
0.00.132.513 I print_info: rope_yarn_log_mul = 0.0000
|
||||
0.00.132.513 I print_info: rope_finetuned = unknown
|
||||
0.00.132.514 I print_info: model type = 1.7B
|
||||
0.00.132.515 I print_info: model params = 1.71 B
|
||||
0.00.132.515 I print_info: general.name = Hf_Format
|
||||
0.00.132.519 I print_info: vocab type = BPE
|
||||
0.00.132.520 I print_info: n_vocab = 131072
|
||||
0.00.132.521 I print_info: n_merges = 130716
|
||||
0.00.132.521 I print_info: BOS token = 0 '<｜start�of�sentence｜>'
|
||||
0.00.132.521 I print_info: EOS token = 1 '<｜end�of�sentence｜>'
|
||||
0.00.132.522 I print_info: EOT token = 1 '<｜end�of�sentence｜>'
|
||||
0.00.132.522 I print_info: UNK token = 5 '<unk>'
|
||||
0.00.132.522 I print_info: PAD token = 2 '<｜�pad�｜>'
|
||||
0.00.132.522 I print_info: LF token = 198 'ÄŠ'
|
||||
0.00.132.522 I print_info: FIM PRE token = 11 '<｜fim�begin｜>'
|
||||
0.00.132.523 I print_info: FIM SUF token = 12 '<｜fim�hole｜>'
|
||||
0.00.132.523 I print_info: FIM MID token = 13 '<｜fim�end｜>'
|
||||
0.00.132.523 I print_info: EOG token = 1 '<｜end�of�sentence｜>'
|
||||
0.00.132.523 I print_info: max token length = 210
|
||||
0.00.132.527 I load_tensors: loading model tensors, this can take a while... (load_mode = mmap)
|
||||
0.00.136.999 I load_tensors: CPU_Mapped model buffer size = 1043.71 MiB
|
||||
0.00.137.021 I load_tensors: CPU_REPACK model buffer size = 622.37 MiB
|
||||
0.00.660.098 I cmn common_init_: added <｜end�of�sentence｜> logit bias = -inf
|
||||
0.00.660.641 I llama_context: constructing llama_context
|
||||
0.00.660.663 I llama_context: n_seq_max = 1
|
||||
0.00.660.664 I llama_context: n_ctx = 4096
|
||||
0.00.660.664 I llama_context: n_ctx_seq = 4096
|
||||
0.00.660.664 I llama_context: n_batch = 2048
|
||||
0.00.660.664 I llama_context: n_ubatch = 512
|
||||
0.00.660.665 I llama_context: causal_attn = 1
|
||||
0.00.660.667 I llama_context: flash_attn = auto
|
||||
0.00.660.667 I llama_context: kv_unified = false
|
||||
0.00.660.671 I llama_context: freq_base = 5000000.0
|
||||
0.00.660.671 I llama_context: freq_scale = 1
|
||||
0.00.660.671 I llama_context: n_rs_seq = 0
|
||||
0.00.660.671 I llama_context: n_outputs_max = 1
|
||||
0.00.660.671 I llama_context: n_outputs_max_per_seq = 1
|
||||
0.00.660.672 I llama_context: n_ctx_seq (4096) < n_ctx_train (1048576) -- the full capacity of the model will not be utilized
|
||||
0.00.660.720 I llama_context: CPU output buffer size = 0.50 MiB
|
||||
0.00.660.732 I llama_kv_cache_iswa: creating non-SWA KV cache, size = 4096 cells
|
||||
0.00.660.830 I llama_kv_cache: CPU KV buffer size = 56.00 MiB
|
||||
0.00.698.835 I llama_kv_cache: size = 56.00 MiB ( 4096 cells, 7 layers, 1/1 seqs), K (f16): 28.00 MiB, V (f16): 28.00 MiB
|
||||
0.00.698.866 I llama_kv_cache: attn_rot_k = 0, n_embd_head_k_all = 256
|
||||
0.00.698.866 I llama_kv_cache: attn_rot_v = 0, n_embd_head_k_all = 256
|
||||
0.00.698.870 I llama_kv_cache_iswa: creating SWA KV cache, size = 1024 cells
|
||||
0.00.698.952 I llama_kv_cache: CPU KV buffer size = 42.00 MiB
|
||||
0.00.728.529 I llama_kv_cache: size = 42.00 MiB ( 1024 cells, 21 layers, 1/1 seqs), K (f16): 21.00 MiB, V (f16): 21.00 MiB
|
||||
0.00.728.553 I llama_kv_cache: attn_rot_k = 0, n_embd_head_k_all = 256
|
||||
0.00.728.553 I llama_kv_cache: attn_rot_v = 0, n_embd_head_k_all = 256
|
||||
0.00.728.574 I sched_reserve: reserving ...
|
||||
0.00.729.229 I resolve_fused_ops: Flash Attention enabled
|
||||
0.00.729.251 I resolve_fused_ops: resolving fused DeepSeek V4 HC support:
|
||||
0.00.729.411 I resolve_fused_ops: fused DeepSeek V4 HC pre enabled
|
||||
0.00.729.580 I resolve_fused_ops: fused DeepSeek V4 HC comb enabled
|
||||
0.00.729.736 I resolve_fused_ops: fused DeepSeek V4 HC post enabled
|
||||
0.00.730.328 I sched_reserve: CPU compute buffer size = 60.02 MiB
|
||||
0.00.730.349 I sched_reserve: graph nodes = 986
|
||||
0.00.730.349 I sched_reserve: graph splits = 1
|
||||
0.00.730.351 I sched_reserve: reserve took 1.77 ms, sched copies = 1
|
||||
0.00.730.391 I cmn init: llama threadpool init, n_threads = 12
|
||||
0.00.730.437 I cmn common_init_: warming up the model with an empty run - please wait ... (--no-warmup to disable)
|
||||
0.00.787.044 I srv load_model: initializing, n_slots = 1, n_ctx_slot = 4096, kv_unified = 'false'
|
||||
0.00.787.095 I spec common_specu: no implementations specified for speculative decoding
|
||||
0.00.787.099 I slot load_model: id 0 | task -1 | new slot, n_ctx = 4096
|
||||
0.00.787.169 I srv load_model: prompt cache is disabled - use `--cache-ram N` to enable it
|
||||
0.00.787.193 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
|
||||
0.00.787.194 I srv load_model: context checkpoints enabled, max = 32, min spacing = 8192
|
||||
0.00.787.217 W srv init: --cache-idle-slots requires --cache-ram, disabling
|
||||
0.00.795.786 I srv init: init: chat template, example_format: '<｜start�of�sentence｜><|System|>
|
||||
you are a helpful assistant.
|
||||
|
||||
You are a helpful assistant<｜end�of�sentence｜><｜start�of�sentence｜><|User|>Hello<｜end�of�sentence｜><｜start�of�sentence｜><|Bot|></think>Hi there<｜end�of�sentence｜><｜start�of�sentence｜><|User|>How are you?<｜end�of�sentence｜><｜start�of�sentence｜><|Bot|></think>'
|
||||
0.00.801.190 I srv init: init: chat template, thinking = 0
|
||||
0.00.801.219 I srv init: preserve_reasoning kwarg: true
|
||||
0.00.801.220 W srv init: chat template supports preserving reasoning, it is enabled by default (may use more tokens, disable via --no-reasoning-preserve)
|
||||
0.00.801.260 I srv llama_server: model loaded
|
||||
0.00.801.264 I srv llama_server: listening on http://127.0.0.1:46731
|
||||
0.00.801.466 I srv update_slots: all slots are idle
|
||||
0.00.814.366 I srv server_strea: conv_id= (empty=1)
|
||||
0.00.814.715 I srv operator(): chat format: peg-native
|
||||
0.00.814.865 I slot get_availabl: id 0 | task -1 | - skipping, slot is empty
|
||||
0.00.814.884 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = -1
|
||||
0.00.814.919 I slot launch_slot_: id 0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> ?top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
|
||||
0.00.814.948 I slot launch_slot_: id 0 | task -1 | sampler params:
|
||||
repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
|
||||
dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 64
|
||||
top_k = 0, top_p = 0.950, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.000
|
||||
mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000, adaptive_target = -1.000, adaptive_decay = 0.900
|
||||
0.00.814.952 I slot launch_slot_: id 0 | task 0 | processing task, is_child = 0
|
||||
0.00.814.962 I slot operator(): id 0 | task 0 | new prompt, n_ctx_slot = 4096, n_keep = 0, task.n_tokens = 28
|
||||
0.00.814.965 I slot operator(): id 0 | task 0 | cached n_tokens = 0, memory_seq_rm [0, end)
|
||||
0.00.851.888 I slot operator(): id 0 | task 0 | cached n_tokens = 10, memory_seq_rm [10, end)
|
||||
0.00.852.066 I slot create_check: id 0 | task 0 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 9, n_tokens = 10, size = 0.411 MiB)
|
||||
0.00.902.656 I slot operator(): id 0 | task 0 | cached n_tokens = 24, memory_seq_rm [24, end)
|
||||
0.00.902.753 I slot init_sampler: id 0 | task 0 | init sampler, took 0.02 ms, tokens: text = 28, total = 28
|
||||
0.00.903.382 I slot create_check: id 0 | task 0 | created context checkpoint 2 of 32 (pos_min = 0, pos_max = 23, n_tokens = 24, size = 0.985 MiB)
|
||||
0.00.948.341 I slot print_timing: id 0 | task 0 | prompt eval time = 113.24 ms / 28 tokens ( 4.04 ms per token, 247.26 tokens per second)
|
||||
0.00.948.370 I slot print_timing: id 0 | task 0 | eval time = 20.11 ms / 2 tokens ( 20.11 ms per token, 49.73 tokens per second)
|
||||
0.00.948.371 I slot print_timing: id 0 | task 0 | total time = 133.35 ms / 30 tokens
|
||||
0.00.948.378 I slot print_timing: id 0 | task 0 | graphs reused = 1
|
||||
0.00.948.446 I slot release: id 0 | task 0 | stop processing: n_tokens = 29, truncated = 0
|
||||
0.00.948.458 I srv update_slots: all slots are idle
|
||||
0.00.954.091 I srv operator(): operator(): cleaning up before exit...
|
||||
0.00.955.233 I common_memory_breakdown_print: | memory breakdown [MiB] | total free self model context compute unaccounted |
|
||||
0.00.955.254 I common_memory_breakdown_print: | - Host | 1201 = 1043 + 98 + 60 |
|
||||
0.00.955.255 I common_memory_breakdown_print: | - CPU_REPACK | 622 = 622 + 0 + 0 |
|
||||
+172
@@ -0,0 +1,172 @@
|
||||
INFO 09-24 04:55:38 [patch.py:259] NVFP4 W4A4 weight_scale NaN-clamp: installed.
|
||||
INFO 09-24 04:55:38 [patch.py:510] inductor factorable-divisibility patch: installed.
|
||||
INFO 09-24 04:55:38 [patch.py:589] [cumem-cuda] CuMemAllocator._python_free_callback patched: asleep guard extended to all platforms.
|
||||
INFO 09-24 04:55:40 [stage_init_utils.py:768] [stage_init] Set VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
WARNING 09-24 04:55:40 [stage_init_utils.py:1887] [stage_init] Failed to load transfer config: [Errno 21] Is a directory: '/home/zhout/project/edge_infer/vllm-omni-edge'
|
||||
{
|
||||
"scope": "complete Spark GGUF text requests through Omni StageRuntime/StagePool and external llama.cpp backend",
|
||||
"model_file": "/home/zhout/project/edge_infer/models/Spark-X2.5-1.7B-GGUF/Spark-X2.5-1.7B-Q4_K_M.gguf",
|
||||
"device": "cpu",
|
||||
"memory_budget": {
|
||||
"capacities": {
|
||||
"host_ram": 12884901888
|
||||
},
|
||||
"demands": {
|
||||
"host_ram": 4294967296
|
||||
}
|
||||
},
|
||||
"warmup_count": 0,
|
||||
"measured_count": 1,
|
||||
"startup_s": 2.108074760006275,
|
||||
"execution_plan": {
|
||||
"backend": "external.llamacpp.text.v1",
|
||||
"stage_id": 0,
|
||||
"worker_generation": "b7dac296a8c84e998547b45c4478a311",
|
||||
"model_sha256": "902bde2522394954ac17821b3e5fd0df02defbc6944f122253f2580acf0503f4",
|
||||
"server_sha256": "0d6bb6ade8da8f3231917ce23261da5331333df0007dc4bda74c00e0774e092c",
|
||||
"model_alias": "/home/zhout/project/edge_infer/models/Spark-X2.5-1.7B-GGUF/Spark-X2.5-1.7B-Q4_K_M.gguf",
|
||||
"requested_device": "cpu",
|
||||
"expected_device_name": null,
|
||||
"layer_assignments": 0,
|
||||
"offloaded_layers": null,
|
||||
"model_buffer_devices": [],
|
||||
"all_model_buffers": [
|
||||
"CPU_Mapped",
|
||||
"CPU_REPACK"
|
||||
],
|
||||
"cpu_only_layout": true,
|
||||
"worker_pid": 2121572,
|
||||
"loaded_rss_bytes": 1914925056,
|
||||
"reserved_bytes": {
|
||||
"host_ram": 4294967296
|
||||
},
|
||||
"memory_pool": "host_ram",
|
||||
"memory_overhead_bytes": 2147483648,
|
||||
"context_tokens": 4096,
|
||||
"max_new_tokens": 96,
|
||||
"max_io_bytes": 1048576,
|
||||
"request_capacity": 1,
|
||||
"stateful_session": "llama.cpp-owned; reset for each complete request",
|
||||
"evidence": "B"
|
||||
},
|
||||
"checks": [
|
||||
{
|
||||
"case": "short",
|
||||
"answer": "Paris",
|
||||
"expected": "Paris",
|
||||
"correct": true,
|
||||
"wall_s": 0.1604909870075062,
|
||||
"stage_event": {
|
||||
"request_id": "short-450962718247056",
|
||||
"stage_id": 0,
|
||||
"epoch": 1,
|
||||
"seq": 1,
|
||||
"kind": "text",
|
||||
"worker_generation": "b7dac296a8c84e998547b45c4478a311",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"llamacpp_wall_s": 0.15617399098118767,
|
||||
"usage": {
|
||||
"completion_tokens": 2,
|
||||
"prompt_tokens": 28,
|
||||
"total_tokens": 30,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"case": "inventory_120",
|
||||
"answer": "1511",
|
||||
"expected": "1511",
|
||||
"correct": true,
|
||||
"wall_s": 7.575511201983318,
|
||||
"stage_event": {
|
||||
"request_id": "inventory_120-450962878826651",
|
||||
"stage_id": 0,
|
||||
"epoch": 2,
|
||||
"seq": 1,
|
||||
"kind": "text",
|
||||
"worker_generation": "b7dac296a8c84e998547b45c4478a311",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"llamacpp_wall_s": 7.569192337978166,
|
||||
"usage": {
|
||||
"completion_tokens": 5,
|
||||
"prompt_tokens": 2463,
|
||||
"total_tokens": 2468,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"nearest_rank_p50_wall_s": 7.60705011402024,
|
||||
"nearest_rank_p95_wall_s": 7.60705011402024,
|
||||
"context_check": {
|
||||
"rejected": true,
|
||||
"error": "ResourceUnavailable: Spark prompt plus generation bound exceeds context; refusing truncation",
|
||||
"recovery": {
|
||||
"case": "short",
|
||||
"answer": "Paris",
|
||||
"expected": "Paris",
|
||||
"correct": true,
|
||||
"wall_s": 0.14723540598060936,
|
||||
"stage_event": {
|
||||
"request_id": "short-450978078478676",
|
||||
"stage_id": 0,
|
||||
"epoch": 4,
|
||||
"seq": 1,
|
||||
"kind": "text",
|
||||
"worker_generation": "b7dac296a8c84e998547b45c4478a311",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"llamacpp_wall_s": 0.14451628603274003,
|
||||
"usage": {
|
||||
"completion_tokens": 2,
|
||||
"prompt_tokens": 28,
|
||||
"total_tokens": 30,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"abort_check": {
|
||||
"stale_output": false,
|
||||
"worker_exited": true,
|
||||
"ledger_after_abort": {
|
||||
"capacities": {
|
||||
"host_ram": 12884901888
|
||||
},
|
||||
"reserved": {
|
||||
"host_ram": 0
|
||||
},
|
||||
"owners": [],
|
||||
"quarantined": []
|
||||
}
|
||||
},
|
||||
"status": "passed",
|
||||
"ledger_after_shutdown": {
|
||||
"capacities": {
|
||||
"host_ram": 12884901888
|
||||
},
|
||||
"reserved": {
|
||||
"host_ram": 0
|
||||
},
|
||||
"owners": [],
|
||||
"quarantined": []
|
||||
}
|
||||
}
|
||||
+199
@@ -0,0 +1,199 @@
|
||||
{
|
||||
"scope": "complete Spark GGUF text requests through Omni StageRuntime/StagePool and external llama.cpp backend",
|
||||
"model_file": "/home/zhout/project/edge_infer/models/Spark-X2.5-1.7B-GGUF/Spark-X2.5-1.7B-Q4_K_M.gguf",
|
||||
"device": "cpu",
|
||||
"memory_budget": {
|
||||
"capacities": {
|
||||
"host_ram": 12884901888
|
||||
},
|
||||
"demands": {
|
||||
"host_ram": 4294967296
|
||||
}
|
||||
},
|
||||
"warmup_count": 0,
|
||||
"measured_count": 1,
|
||||
"startup_s": 2.108074760006275,
|
||||
"execution_plan": {
|
||||
"backend": "external.llamacpp.text.v1",
|
||||
"stage_id": 0,
|
||||
"worker_generation": "b7dac296a8c84e998547b45c4478a311",
|
||||
"model_sha256": "902bde2522394954ac17821b3e5fd0df02defbc6944f122253f2580acf0503f4",
|
||||
"server_sha256": "0d6bb6ade8da8f3231917ce23261da5331333df0007dc4bda74c00e0774e092c",
|
||||
"model_alias": "/home/zhout/project/edge_infer/models/Spark-X2.5-1.7B-GGUF/Spark-X2.5-1.7B-Q4_K_M.gguf",
|
||||
"requested_device": "cpu",
|
||||
"expected_device_name": null,
|
||||
"layer_assignments": 0,
|
||||
"offloaded_layers": null,
|
||||
"model_buffer_devices": [],
|
||||
"all_model_buffers": [
|
||||
"CPU_Mapped",
|
||||
"CPU_REPACK"
|
||||
],
|
||||
"cpu_only_layout": true,
|
||||
"worker_pid": 2121572,
|
||||
"loaded_rss_bytes": 1914925056,
|
||||
"reserved_bytes": {
|
||||
"host_ram": 4294967296
|
||||
},
|
||||
"memory_pool": "host_ram",
|
||||
"memory_overhead_bytes": 2147483648,
|
||||
"context_tokens": 4096,
|
||||
"max_new_tokens": 96,
|
||||
"max_io_bytes": 1048576,
|
||||
"request_capacity": 1,
|
||||
"stateful_session": "llama.cpp-owned; reset for each complete request",
|
||||
"evidence": "B"
|
||||
},
|
||||
"checks": [
|
||||
{
|
||||
"case": "short",
|
||||
"answer": "Paris",
|
||||
"expected": "Paris",
|
||||
"correct": true,
|
||||
"wall_s": 0.1604909870075062,
|
||||
"stage_event": {
|
||||
"request_id": "short-450962718247056",
|
||||
"stage_id": 0,
|
||||
"epoch": 1,
|
||||
"seq": 1,
|
||||
"kind": "text",
|
||||
"worker_generation": "b7dac296a8c84e998547b45c4478a311",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"llamacpp_wall_s": 0.15617399098118767,
|
||||
"usage": {
|
||||
"completion_tokens": 2,
|
||||
"prompt_tokens": 28,
|
||||
"total_tokens": 30,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"case": "inventory_120",
|
||||
"answer": "1511",
|
||||
"expected": "1511",
|
||||
"correct": true,
|
||||
"wall_s": 7.575511201983318,
|
||||
"stage_event": {
|
||||
"request_id": "inventory_120-450962878826651",
|
||||
"stage_id": 0,
|
||||
"epoch": 2,
|
||||
"seq": 1,
|
||||
"kind": "text",
|
||||
"worker_generation": "b7dac296a8c84e998547b45c4478a311",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"llamacpp_wall_s": 7.569192337978166,
|
||||
"usage": {
|
||||
"completion_tokens": 5,
|
||||
"prompt_tokens": 2463,
|
||||
"total_tokens": 2468,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"warmups": [],
|
||||
"measured": [
|
||||
{
|
||||
"case": "inventory_120",
|
||||
"answer": "1511",
|
||||
"expected": "1511",
|
||||
"correct": true,
|
||||
"wall_s": 7.60705011402024,
|
||||
"stage_event": {
|
||||
"request_id": "inventory_120-450970454411312",
|
||||
"stage_id": 0,
|
||||
"epoch": 3,
|
||||
"seq": 1,
|
||||
"kind": "text",
|
||||
"worker_generation": "b7dac296a8c84e998547b45c4478a311",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"llamacpp_wall_s": 7.600058256997727,
|
||||
"usage": {
|
||||
"completion_tokens": 5,
|
||||
"prompt_tokens": 2463,
|
||||
"total_tokens": 2468,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"nearest_rank_p50_wall_s": 7.60705011402024,
|
||||
"nearest_rank_p95_wall_s": 7.60705011402024,
|
||||
"context_check": {
|
||||
"rejected": true,
|
||||
"error": "ResourceUnavailable: Spark prompt plus generation bound exceeds context; refusing truncation",
|
||||
"recovery": {
|
||||
"case": "short",
|
||||
"answer": "Paris",
|
||||
"expected": "Paris",
|
||||
"correct": true,
|
||||
"wall_s": 0.14723540598060936,
|
||||
"stage_event": {
|
||||
"request_id": "short-450978078478676",
|
||||
"stage_id": 0,
|
||||
"epoch": 4,
|
||||
"seq": 1,
|
||||
"kind": "text",
|
||||
"worker_generation": "b7dac296a8c84e998547b45c4478a311",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"llamacpp_wall_s": 0.14451628603274003,
|
||||
"usage": {
|
||||
"completion_tokens": 2,
|
||||
"prompt_tokens": 28,
|
||||
"total_tokens": 30,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"abort_check": {
|
||||
"stale_output": false,
|
||||
"worker_exited": true,
|
||||
"ledger_after_abort": {
|
||||
"capacities": {
|
||||
"host_ram": 12884901888
|
||||
},
|
||||
"reserved": {
|
||||
"host_ram": 0
|
||||
},
|
||||
"owners": [],
|
||||
"quarantined": []
|
||||
}
|
||||
},
|
||||
"status": "passed",
|
||||
"ledger_after_shutdown": {
|
||||
"capacities": {
|
||||
"host_ram": 12884901888
|
||||
},
|
||||
"reserved": {
|
||||
"host_ram": 0
|
||||
},
|
||||
"owners": [],
|
||||
"quarantined": []
|
||||
}
|
||||
}
|
||||
+320
@@ -0,0 +1,320 @@
|
||||
warning: no usable GPU found, --gpu-layers option will be ignored
|
||||
warning: one possible reason is that llama.cpp was compiled without GPU support
|
||||
warning: consult docs/build.md for compilation instructions
|
||||
0.00.002.621 I cmn common_param: common_params_print_info: build 10849 (e71b80510) with GNU 11.4.0 for Linux x86_64
|
||||
0.00.002.643 I cmn common_param: common_params_print_info: verbosity = 4 (adjust with the `-lv N` CLI arg)
|
||||
0.00.002.644 I cmn common_param: device_info:
|
||||
0.00.002.657 I cmn common_param: - CPU : AMD Ryzen AI 9 HX 370 w/ Radeon 890M (31647 MiB, 31647 MiB free)
|
||||
0.00.002.715 I cmn common_param: system_info: n_threads = 12 (n_threads_batch = 12) / 24 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX_VNNI = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | AMX_INT8 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
|
||||
0.00.002.816 I srv init: using 23 threads for HTTP server
|
||||
0.00.002.834 I srv init: The UI is disabled
|
||||
0.00.002.835 I srv init: Use --ui/--no-ui (or deprecated --webui/--no-webui) to enable/disable
|
||||
0.00.002.889 W srv llama_server: -----------------
|
||||
0.00.002.907 W srv llama_server: CORS is set to allow all origins ('*') and no API key is set
|
||||
0.00.002.908 W srv llama_server: this can be a security risk (cross-origin attacks)
|
||||
0.00.002.908 W srv llama_server: more info: https://github.com/ggml-org/llama.cpp/pull/25655
|
||||
0.00.002.908 W srv llama_server: -----------------
|
||||
0.00.002.927 I srv start: binding port with default address family
|
||||
0.00.004.380 I srv load_model: loading model '/home/zhout/project/edge_infer/models/Spark-X2.5-1.7B-GGUF/Spark-X2.5-1.7B-Q4_K_M.gguf'
|
||||
0.00.004.400 I srv load_model: local path '/home/zhout/project/edge_infer/models/Spark-X2.5-1.7B-GGUF/Spark-X2.5-1.7B-Q4_K_M.gguf'
|
||||
0.00.030.351 I llama_model_loader: loaded meta data with 37 key-value pairs and 226 tensors from /home/zhout/project/edge_infer/models/Spark-X2.5-1.7B-GGUF/Spark-X2.5-1.7B-Q4_K_M.gguf (version GGUF V3 (latest))
|
||||
0.00.030.391 I llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
0.00.030.401 I llama_model_loader: - kv 0: general.architecture str = spark2_5
|
||||
0.00.030.402 I llama_model_loader: - kv 1: general.type str = model
|
||||
0.00.030.405 I llama_model_loader: - kv 2: general.sampling.top_k i32 = 0
|
||||
0.00.030.410 I llama_model_loader: - kv 3: general.sampling.top_p f32 = 0.950000
|
||||
0.00.030.411 I llama_model_loader: - kv 4: general.sampling.temp f32 = 1.000000
|
||||
0.00.030.411 I llama_model_loader: - kv 5: general.name str = Hf_Format
|
||||
0.00.030.412 I llama_model_loader: - kv 6: general.size_label str = 1.7B
|
||||
0.00.030.413 I llama_model_loader: - kv 7: spark2_5.block_count u32 = 28
|
||||
0.00.030.413 I llama_model_loader: - kv 8: spark2_5.context_length u32 = 1048576
|
||||
0.00.030.413 I llama_model_loader: - kv 9: spark2_5.embedding_length u32 = 2048
|
||||
0.00.030.414 I llama_model_loader: - kv 10: spark2_5.feed_forward_length u32 = 6656
|
||||
0.00.030.414 I llama_model_loader: - kv 11: spark2_5.attention.head_count u32 = 8
|
||||
0.00.030.414 I llama_model_loader: - kv 12: spark2_5.attention.head_count_kv u32 = 2
|
||||
0.00.030.416 I llama_model_loader: - kv 13: spark2_5.rope.freq_base f32 = 5000000.000000
|
||||
0.00.030.416 I llama_model_loader: - kv 14: spark2_5.rope.freq_base_swa f32 = 10000.000000
|
||||
0.00.030.417 I llama_model_loader: - kv 15: spark2_5.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
0.00.030.417 I llama_model_loader: - kv 16: spark2_5.attention.key_length u32 = 256
|
||||
0.00.030.417 I llama_model_loader: - kv 17: spark2_5.attention.value_length u32 = 256
|
||||
0.00.030.417 I llama_model_loader: - kv 18: spark2_5.vocab_size u32 = 131072
|
||||
0.00.030.418 I llama_model_loader: - kv 19: spark2_5.attention.sliding_window u32 = 512
|
||||
0.00.030.439 I llama_model_loader: - kv 20: spark2_5.attention.sliding_window_pattern arr[bool,28] = [true, true, true, false, true, true,...
|
||||
0.00.030.439 I llama_model_loader: - kv 21: spark2_5.rope.dimension_count u32 = 64
|
||||
0.00.030.439 I llama_model_loader: - kv 22: spark2_5.rope.dimension_count_swa u32 = 256
|
||||
0.00.030.440 I llama_model_loader: - kv 23: tokenizer.ggml.model str = gpt2
|
||||
0.00.030.440 I llama_model_loader: - kv 24: tokenizer.ggml.pre str = spark2_5
|
||||
0.00.038.822 I llama_model_loader: - kv 25: tokenizer.ggml.tokens arr[str,131072] = ["<｜startâ–�ofâ–�sentence｜>", "<ï...
|
||||
0.00.040.548 I llama_model_loader: - kv 26: tokenizer.ggml.token_type arr[i32,131072] = [3, 3, 3, 4, 4, 3, 3, 3, 4, 4, 4, 3, ...
|
||||
0.00.048.065 I llama_model_loader: - kv 27: tokenizer.ggml.merges arr[str,130716] = ["Ä Ä ", "Ä t", "i n", "Ä a", "Ä Ä...
|
||||
0.00.048.092 I llama_model_loader: - kv 28: tokenizer.ggml.bos_token_id u32 = 0
|
||||
0.00.048.093 I llama_model_loader: - kv 29: tokenizer.ggml.eos_token_id u32 = 1
|
||||
0.00.048.093 I llama_model_loader: - kv 30: tokenizer.ggml.unknown_token_id u32 = 5
|
||||
0.00.048.094 I llama_model_loader: - kv 31: tokenizer.ggml.padding_token_id u32 = 2
|
||||
0.00.048.095 I llama_model_loader: - kv 32: tokenizer.ggml.add_bos_token bool = false
|
||||
0.00.048.095 I llama_model_loader: - kv 33: tokenizer.ggml.add_eos_token bool = false
|
||||
0.00.048.098 I llama_model_loader: - kv 34: tokenizer.chat_template str = {%- if not messages %}\n {{- raise_...
|
||||
0.00.048.099 I llama_model_loader: - kv 35: general.quantization_version u32 = 2
|
||||
0.00.048.100 I llama_model_loader: - kv 36: general.file_type u32 = 15
|
||||
0.00.048.106 I llama_model_loader: - type f32: 57 tensors
|
||||
0.00.048.107 I llama_model_loader: - type q4_K: 140 tensors
|
||||
0.00.048.107 I llama_model_loader: - type q6_K: 29 tensors
|
||||
0.00.048.110 I print_info: file format = GGUF V3 (latest)
|
||||
0.00.048.112 I print_info: file type = Q4_K - Medium
|
||||
0.00.048.116 I print_info: file size = 1.03 GiB (5.16 BPW)
|
||||
0.00.117.411 I load: 86 unused tokens
|
||||
0.00.137.161 I load: printing all EOG tokens:
|
||||
0.00.137.189 I load: - 1 ('<｜end�of�sentence｜>')
|
||||
0.00.137.314 I load: special tokens cache size = 113
|
||||
0.00.162.217 I load: token to piece cache size = 0.7975 MB
|
||||
0.00.162.273 I print_info: arch = spark2_5
|
||||
0.00.162.273 I print_info: vocab_only = 0
|
||||
0.00.162.273 I print_info: no_alloc = 0
|
||||
0.00.162.274 I print_info: n_ctx_train = 1048576
|
||||
0.00.162.276 I print_info: n_embd_inp = 2048
|
||||
0.00.162.276 I print_info: n_embd = 2048
|
||||
0.00.162.278 I print_info: n_embd_out = 2048
|
||||
0.00.162.278 I print_info: n_layer = 28
|
||||
0.00.162.278 I print_info: n_layer_all = 28
|
||||
0.00.162.298 I print_info: n_head = 8
|
||||
0.00.162.300 I print_info: n_head_kv = 2
|
||||
0.00.162.301 I print_info: n_rot = 64
|
||||
0.00.162.301 I print_info: n_swa = 512
|
||||
0.00.162.302 I print_info: is_swa_any = 1
|
||||
0.00.162.302 I print_info: non_causal_type = 0
|
||||
0.00.162.302 I print_info: n_embd_head_k = 256
|
||||
0.00.162.302 I print_info: n_embd_head_v = 256
|
||||
0.00.162.304 I print_info: n_gqa = 4
|
||||
0.00.162.310 I print_info: n_embd_k_gqa = 512
|
||||
0.00.162.313 I print_info: n_embd_v_gqa = 512
|
||||
0.00.162.314 I print_info: f_norm_eps = 0.0e+00
|
||||
0.00.162.315 I print_info: f_norm_rms_eps = 1.0e-06
|
||||
0.00.162.315 I print_info: f_clamp_kqv = 0.0e+00
|
||||
0.00.162.315 I print_info: f_max_alibi_bias = 0.0e+00
|
||||
0.00.162.315 I print_info: f_logit_scale = 0.0e+00
|
||||
0.00.162.316 I print_info: f_attn_scale = 0.0e+00
|
||||
0.00.162.316 I print_info: f_attn_value_scale = 0.0000
|
||||
0.00.162.317 I print_info: n_ff = 6656
|
||||
0.00.162.317 I print_info: n_expert = 0
|
||||
0.00.162.318 I print_info: n_expert_used = 0
|
||||
0.00.162.318 I print_info: n_expert_groups = 0
|
||||
0.00.162.318 I print_info: n_group_used = 0
|
||||
0.00.162.318 I print_info: causal attn = 1
|
||||
0.00.162.318 AMX is not ready to be used!
|
||||
I print_info: pooling type = -1
|
||||
0.00.162.319 I print_info: rope type = 2
|
||||
0.00.162.319 I print_info: rope scaling = linear
|
||||
0.00.162.320 I print_info: freq_base_train = 5000000.0
|
||||
0.00.162.320 I print_info: freq_scale_train = 1
|
||||
0.00.162.320 I print_info: freq_base_swa = 10000.0
|
||||
0.00.162.321 I print_info: freq_scale_swa = 1
|
||||
0.00.162.321 I print_info: n_embd_head_k_swa = 256
|
||||
0.00.162.321 I print_info: n_embd_head_v_swa = 256
|
||||
0.00.162.321 I print_info: n_rot_swa = 256
|
||||
0.00.162.321 I print_info: n_ctx_orig_yarn = 1048576
|
||||
0.00.162.321 I print_info: rope_yarn_log_mul = 0.0000
|
||||
0.00.162.321 I print_info: rope_finetuned = unknown
|
||||
0.00.162.323 I print_info: model type = 1.7B
|
||||
0.00.162.323 I print_info: model params = 1.71 B
|
||||
0.00.162.323 I print_info: general.name = Hf_Format
|
||||
0.00.162.328 I print_info: vocab type = BPE
|
||||
0.00.162.330 I print_info: n_vocab = 131072
|
||||
0.00.162.330 I print_info: n_merges = 130716
|
||||
0.00.162.331 I print_info: BOS token = 0 '<｜start�of�sentence｜>'
|
||||
0.00.162.331 I print_info: EOS token = 1 '<｜end�of�sentence｜>'
|
||||
0.00.162.331 I print_info: EOT token = 1 '<｜end�of�sentence｜>'
|
||||
0.00.162.332 I print_info: UNK token = 5 '<unk>'
|
||||
0.00.162.332 I print_info: PAD token = 2 '<｜�pad�｜>'
|
||||
0.00.162.332 I print_info: LF token = 198 'ÄŠ'
|
||||
0.00.162.333 I print_info: FIM PRE token = 11 '<｜fim�begin｜>'
|
||||
0.00.162.333 I print_info: FIM SUF token = 12 '<｜fim�hole｜>'
|
||||
0.00.162.333 I print_info: FIM MID token = 13 '<｜fim�end｜>'
|
||||
0.00.162.333 I print_info: EOG token = 1 '<｜end�of�sentence｜>'
|
||||
0.00.162.334 I print_info: max token length = 210
|
||||
0.00.162.338 I load_tensors: loading model tensors, this can take a while... (load_mode = mmap)
|
||||
0.00.166.713 I load_tensors: CPU_Mapped model buffer size = 1043.71 MiB
|
||||
0.00.166.736 I load_tensors: CPU_REPACK model buffer size = 622.37 MiB
|
||||
0.00.704.470 I cmn common_init_: added <｜end�of�sentence｜> logit bias = -inf
|
||||
0.00.704.978 I llama_context: constructing llama_context
|
||||
0.00.705.002 I llama_context: n_seq_max = 1
|
||||
0.00.705.002 I llama_context: n_ctx = 4096
|
||||
0.00.705.002 I llama_context: n_ctx_seq = 4096
|
||||
0.00.705.003 I llama_context: n_batch = 2048
|
||||
0.00.705.003 I llama_context: n_ubatch = 512
|
||||
0.00.705.003 I llama_context: causal_attn = 1
|
||||
0.00.705.004 I llama_context: flash_attn = auto
|
||||
0.00.705.005 I llama_context: kv_unified = false
|
||||
0.00.705.008 I llama_context: freq_base = 5000000.0
|
||||
0.00.705.009 I llama_context: freq_scale = 1
|
||||
0.00.705.009 I llama_context: n_rs_seq = 0
|
||||
0.00.705.009 I llama_context: n_outputs_max = 1
|
||||
0.00.705.009 I llama_context: n_outputs_max_per_seq = 1
|
||||
0.00.705.009 I llama_context: n_ctx_seq (4096) < n_ctx_train (1048576) -- the full capacity of the model will not be utilized
|
||||
0.00.705.058 I llama_context: CPU output buffer size = 0.50 MiB
|
||||
0.00.705.093 I llama_kv_cache_iswa: creating non-SWA KV cache, size = 4096 cells
|
||||
0.00.705.191 I llama_kv_cache: CPU KV buffer size = 56.00 MiB
|
||||
0.00.743.866 I llama_kv_cache: size = 56.00 MiB ( 4096 cells, 7 layers, 1/1 seqs), K (f16): 28.00 MiB, V (f16): 28.00 MiB
|
||||
0.00.743.897 I llama_kv_cache: attn_rot_k = 0, n_embd_head_k_all = 256
|
||||
0.00.743.898 I llama_kv_cache: attn_rot_v = 0, n_embd_head_k_all = 256
|
||||
0.00.743.901 I llama_kv_cache_iswa: creating SWA KV cache, size = 1024 cells
|
||||
0.00.743.979 I llama_kv_cache: CPU KV buffer size = 42.00 MiB
|
||||
0.00.774.380 I llama_kv_cache: size = 42.00 MiB ( 1024 cells, 21 layers, 1/1 seqs), K (f16): 21.00 MiB, V (f16): 21.00 MiB
|
||||
0.00.774.410 I llama_kv_cache: attn_rot_k = 0, n_embd_head_k_all = 256
|
||||
0.00.774.410 I llama_kv_cache: attn_rot_v = 0, n_embd_head_k_all = 256
|
||||
0.00.774.439 I sched_reserve: reserving ...
|
||||
0.00.775.019 I resolve_fused_ops: Flash Attention enabled
|
||||
0.00.775.041 I resolve_fused_ops: resolving fused DeepSeek V4 HC support:
|
||||
0.00.775.193 I resolve_fused_ops: fused DeepSeek V4 HC pre enabled
|
||||
0.00.775.346 I resolve_fused_ops: fused DeepSeek V4 HC comb enabled
|
||||
0.00.775.501 I resolve_fused_ops: fused DeepSeek V4 HC post enabled
|
||||
0.00.776.101 I sched_reserve: CPU compute buffer size = 60.02 MiB
|
||||
0.00.776.121 I sched_reserve: graph nodes = 986
|
||||
0.00.776.122 I sched_reserve: graph splits = 1
|
||||
0.00.776.123 I sched_reserve: reserve took 1.68 ms, sched copies = 1
|
||||
0.00.776.153 I cmn init: llama threadpool init, n_threads = 12
|
||||
0.00.776.178 I cmn common_init_: warming up the model with an empty run - please wait ... (--no-warmup to disable)
|
||||
0.00.844.437 I srv load_model: initializing, n_slots = 1, n_ctx_slot = 4096, kv_unified = 'false'
|
||||
0.00.844.498 I spec common_specu: no implementations specified for speculative decoding
|
||||
0.00.844.503 I slot load_model: id 0 | task -1 | new slot, n_ctx = 4096
|
||||
0.00.844.573 I srv load_model: prompt cache is disabled - use `--cache-ram N` to enable it
|
||||
0.00.844.573 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
|
||||
0.00.844.574 I srv load_model: context checkpoints enabled, max = 32, min spacing = 8192
|
||||
0.00.844.599 W srv init: --cache-idle-slots requires --cache-ram, disabling
|
||||
0.00.853.650 I srv init: init: chat template, example_format: '<｜start�of�sentence｜><|System|>
|
||||
you are a helpful assistant.
|
||||
|
||||
You are a helpful assistant<｜end�of�sentence｜><｜start�of�sentence｜><|User|>Hello<｜end�of�sentence｜><｜start�of�sentence｜><|Bot|></think>Hi there<｜end�of�sentence｜><｜start�of�sentence｜><|User|>How are you?<｜end�of�sentence｜><｜start�of�sentence｜><|Bot|></think>'
|
||||
0.00.859.441 I srv init: init: chat template, thinking = 0
|
||||
0.00.859.470 I srv init: preserve_reasoning kwarg: true
|
||||
0.00.859.471 W srv init: chat template supports preserving reasoning, it is enabled by default (may use more tokens, disable via --no-reasoning-preserve)
|
||||
0.00.859.512 I srv llama_server: model loaded
|
||||
0.00.859.533 I srv llama_server: listening on http://127.0.0.1:54407
|
||||
0.00.859.739 I srv update_slots: all slots are idle
|
||||
0.00.992.512 I srv server_strea: conv_id= (empty=1)
|
||||
0.00.992.835 I srv operator(): chat format: peg-native
|
||||
0.00.992.983 I slot get_availabl: id 0 | task -1 | - skipping, slot is empty
|
||||
0.00.993.002 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = -1
|
||||
0.00.993.044 I slot launch_slot_: id 0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> ?top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
|
||||
0.00.993.073 I slot launch_slot_: id 0 | task -1 | sampler params:
|
||||
repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
|
||||
dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 64
|
||||
top_k = 0, top_p = 0.950, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.000
|
||||
mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000, adaptive_target = -1.000, adaptive_decay = 0.900
|
||||
0.00.993.074 I slot launch_slot_: id 0 | task 0 | processing task, is_child = 0
|
||||
0.00.993.085 I slot operator(): id 0 | task 0 | new prompt, n_ctx_slot = 4096, n_keep = 0, task.n_tokens = 28
|
||||
0.00.993.088 I slot operator(): id 0 | task 0 | cached n_tokens = 0, memory_seq_rm [0, end)
|
||||
0.01.032.919 I slot operator(): id 0 | task 0 | cached n_tokens = 10, memory_seq_rm [10, end)
|
||||
0.01.033.066 I slot create_check: id 0 | task 0 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 9, n_tokens = 10, size = 0.411 MiB)
|
||||
0.01.090.606 I slot operator(): id 0 | task 0 | cached n_tokens = 24, memory_seq_rm [24, end)
|
||||
0.01.090.694 I slot init_sampler: id 0 | task 0 | init sampler, took 0.02 ms, tokens: text = 28, total = 28
|
||||
0.01.091.528 I slot create_check: id 0 | task 0 | created context checkpoint 2 of 32 (pos_min = 0, pos_max = 23, n_tokens = 24, size = 0.985 MiB)
|
||||
0.01.140.503 I slot print_timing: id 0 | task 0 | prompt eval time = 124.62 ms / 28 tokens ( 4.45 ms per token, 224.68 tokens per second)
|
||||
0.01.140.540 I slot print_timing: id 0 | task 0 | eval time = 22.76 ms / 2 tokens ( 22.76 ms per token, 43.93 tokens per second)
|
||||
0.01.140.541 I slot print_timing: id 0 | task 0 | total time = 147.38 ms / 30 tokens
|
||||
0.01.140.548 I slot print_timing: id 0 | task 0 | graphs reused = 1
|
||||
0.01.140.601 I slot release: id 0 | task 0 | stop processing: n_tokens = 29, truncated = 0
|
||||
0.01.140.616 I srv update_slots: all slots are idle
|
||||
0.01.156.168 I srv server_strea: conv_id= (empty=1)
|
||||
0.01.158.315 I srv operator(): chat format: peg-native
|
||||
0.01.158.472 I slot get_availabl: id 0 | task -1 | - checking sim = 0.005 (12/2463) > 0.100
|
||||
0.01.158.493 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 450962875319
|
||||
0.01.158.531 I slot launch_slot_: id 0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> ?top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
|
||||
0.01.158.555 I slot launch_slot_: id 0 | task -1 | sampler params:
|
||||
repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
|
||||
dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 64
|
||||
top_k = 0, top_p = 0.950, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.000
|
||||
mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000, adaptive_target = -1.000, adaptive_decay = 0.900
|
||||
0.01.158.556 I slot launch_slot_: id 0 | task 5 | processing task, is_child = 0
|
||||
0.01.158.563 I slot operator(): id 0 | task 5 | new prompt, n_ctx_slot = 4096, n_keep = 0, task.n_tokens = 2463
|
||||
0.01.158.564 I slot operator(): id 0 | task 5 | erased invalidated context checkpoint (pos_min = 0, pos_max = 9, n_tokens = 10, n_swa = 512, pos_next = 0, size = 0.411 MiB)
|
||||
0.01.158.565 I slot operator(): id 0 | task 5 | erased invalidated context checkpoint (pos_min = 0, pos_max = 23, n_tokens = 24, n_swa = 512, pos_next = 0, size = 0.985 MiB)
|
||||
0.01.158.566 I slot operator(): id 0 | task 5 | cached n_tokens = 0, memory_seq_rm [0, end)
|
||||
0.01.198.685 I slot operator(): id 0 | task 5 | cached n_tokens = 10, memory_seq_rm [10, end)
|
||||
0.01.198.834 I slot create_check: id 0 | task 5 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 9, n_tokens = 10, size = 0.411 MiB)
|
||||
0.06.951.536 I slot print_timing: id 0 | task 5 | prompt processing, n_tokens = 1947, progress = 0.79, t = 5.79 s / 336.10 tokens per second
|
||||
0.06.951.571 I slot operator(): id 0 | task 5 | cached n_tokens = 1947, memory_seq_rm [1947, end)
|
||||
0.06.962.542 I slot create_check: id 0 | task 5 | created context checkpoint 2 of 32 (pos_min = 923, pos_max = 1946, n_tokens = 1947, size = 21.006 MiB)
|
||||
0.08.575.905 I slot print_timing: id 0 | task 5 | prompt processing, n_tokens = 2459, progress = 1.00, t = 7.42 s / 331.52 tokens per second
|
||||
0.08.575.933 I slot operator(): id 0 | task 5 | cached n_tokens = 2459, memory_seq_rm [2459, end)
|
||||
0.08.576.241 I slot init_sampler: id 0 | task 5 | init sampler, took 0.28 ms, tokens: text = 2463, total = 2463
|
||||
0.08.585.483 I slot create_check: id 0 | task 5 | created context checkpoint 3 of 32 (pos_min = 1435, pos_max = 2458, n_tokens = 2459, size = 21.006 MiB)
|
||||
0.08.717.258 I slot print_timing: id 0 | task 5 | prompt eval time = 7460.58 ms / 2463 tokens ( 3.03 ms per token, 330.14 tokens per second)
|
||||
0.08.717.307 I slot print_timing: id 0 | task 5 | eval time = 98.08 ms / 5 tokens ( 24.52 ms per token, 40.78 tokens per second)
|
||||
0.08.717.308 I slot print_timing: id 0 | task 5 | total time = 7558.67 ms / 2468 tokens
|
||||
0.08.717.309 I slot print_timing: id 0 | task 5 | graphs reused = 4
|
||||
0.08.717.402 I slot release: id 0 | task 5 | stop processing: n_tokens = 2467, truncated = 0
|
||||
0.08.717.439 I srv update_slots: all slots are idle
|
||||
0.08.731.380 I srv server_strea: conv_id= (empty=1)
|
||||
0.08.733.601 I srv operator(): chat format: peg-native
|
||||
0.08.733.747 I slot get_availabl: id 0 | task -1 | - checking sim = 1.000 (2463/2463) > 0.100
|
||||
0.08.733.767 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, f_sim_best = 1.000 (> 0.100 thold), f_keep = 0.998
|
||||
0.08.733.797 I slot launch_slot_: id 0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> ?top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
|
||||
0.08.733.819 I slot launch_slot_: id 0 | task -1 | sampler params:
|
||||
repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
|
||||
dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 64
|
||||
top_k = 0, top_p = 0.950, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.000
|
||||
mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000, adaptive_target = -1.000, adaptive_decay = 0.900
|
||||
0.08.733.820 I slot launch_slot_: id 0 | task 14 | processing task, is_child = 0
|
||||
0.08.733.825 I slot operator(): id 0 | task 14 | new prompt, n_ctx_slot = 4096, n_keep = 0, task.n_tokens = 2463
|
||||
0.08.733.827 I slot operator(): id 0 | task 14 | erased invalidated context checkpoint (pos_min = 0, pos_max = 9, n_tokens = 10, n_swa = 512, pos_next = 0, size = 0.411 MiB)
|
||||
0.08.733.829 I slot operator(): id 0 | task 14 | erased invalidated context checkpoint (pos_min = 923, pos_max = 1946, n_tokens = 1947, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
0.08.735.267 I slot operator(): id 0 | task 14 | erased invalidated context checkpoint (pos_min = 1435, pos_max = 2458, n_tokens = 2459, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
0.08.736.560 I slot operator(): id 0 | task 14 | cached n_tokens = 0, memory_seq_rm [0, end)
|
||||
0.08.777.858 I slot operator(): id 0 | task 14 | cached n_tokens = 10, memory_seq_rm [10, end)
|
||||
0.08.778.045 I slot create_check: id 0 | task 14 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 9, n_tokens = 10, size = 0.411 MiB)
|
||||
0.14.548.438 I slot print_timing: id 0 | task 14 | prompt processing, n_tokens = 1947, progress = 0.79, t = 5.81 s / 334.85 tokens per second
|
||||
0.14.548.466 I slot operator(): id 0 | task 14 | cached n_tokens = 1947, memory_seq_rm [1947, end)
|
||||
0.14.558.923 I slot create_check: id 0 | task 14 | created context checkpoint 2 of 32 (pos_min = 923, pos_max = 1946, n_tokens = 1947, size = 21.006 MiB)
|
||||
0.16.179.151 I slot print_timing: id 0 | task 14 | prompt processing, n_tokens = 2459, progress = 1.00, t = 7.45 s / 330.28 tokens per second
|
||||
0.16.179.183 I slot operator(): id 0 | task 14 | cached n_tokens = 2459, memory_seq_rm [2459, end)
|
||||
0.16.179.528 I slot init_sampler: id 0 | task 14 | init sampler, took 0.32 ms, tokens: text = 2463, total = 2463
|
||||
0.16.190.392 I slot create_check: id 0 | task 14 | created context checkpoint 3 of 32 (pos_min = 1435, pos_max = 2458, n_tokens = 2459, size = 21.006 MiB)
|
||||
0.16.323.144 I slot print_timing: id 0 | task 14 | prompt eval time = 7489.74 ms / 2463 tokens ( 3.04 ms per token, 328.85 tokens per second)
|
||||
0.16.323.177 I slot print_timing: id 0 | task 14 | eval time = 99.55 ms / 5 tokens ( 24.89 ms per token, 40.18 tokens per second)
|
||||
0.16.323.178 I slot print_timing: id 0 | task 14 | total time = 7589.29 ms / 2468 tokens
|
||||
0.16.323.179 I slot print_timing: id 0 | task 14 | graphs reused = 7
|
||||
0.16.323.273 I slot release: id 0 | task 14 | stop processing: n_tokens = 2467, truncated = 0
|
||||
0.16.323.290 I srv update_slots: all slots are idle
|
||||
0.16.351.543 I srv server_strea: conv_id= (empty=1)
|
||||
0.16.351.872 I srv operator(): chat format: peg-native
|
||||
0.16.352.007 I slot get_availabl: id 0 | task -1 | - checking sim = 0.429 (12/28) > 0.100
|
||||
0.16.352.028 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, f_sim_best = 0.429 (> 0.100 thold), f_keep = 0.005
|
||||
0.16.352.059 I slot launch_slot_: id 0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> ?top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
|
||||
0.16.352.083 I slot launch_slot_: id 0 | task -1 | sampler params:
|
||||
repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
|
||||
dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 64
|
||||
top_k = 0, top_p = 0.950, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.000
|
||||
mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000, adaptive_target = -1.000, adaptive_decay = 0.900
|
||||
0.16.352.084 I slot launch_slot_: id 0 | task 23 | processing task, is_child = 0
|
||||
0.16.352.090 I slot operator(): id 0 | task 23 | new prompt, n_ctx_slot = 4096, n_keep = 0, task.n_tokens = 28
|
||||
0.16.352.091 I slot operator(): id 0 | task 23 | erased invalidated context checkpoint (pos_min = 0, pos_max = 9, n_tokens = 10, n_swa = 512, pos_next = 0, size = 0.411 MiB)
|
||||
0.16.352.093 I slot operator(): id 0 | task 23 | erased invalidated context checkpoint (pos_min = 923, pos_max = 1946, n_tokens = 1947, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
0.16.352.094 I slot operator(): id 0 | task 23 | erased invalidated context checkpoint (pos_min = 1435, pos_max = 2458, n_tokens = 2459, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
0.16.354.804 I slot operator(): id 0 | task 23 | cached n_tokens = 0, memory_seq_rm [0, end)
|
||||
0.16.392.962 I slot operator(): id 0 | task 23 | cached n_tokens = 10, memory_seq_rm [10, end)
|
||||
0.16.393.073 I slot create_check: id 0 | task 23 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 9, n_tokens = 10, size = 0.411 MiB)
|
||||
0.16.442.346 I slot operator(): id 0 | task 23 | cached n_tokens = 24, memory_seq_rm [24, end)
|
||||
0.16.442.403 I slot init_sampler: id 0 | task 23 | init sampler, took 0.01 ms, tokens: text = 28, total = 28
|
||||
0.16.442.507 I slot create_check: id 0 | task 23 | created context checkpoint 2 of 32 (pos_min = 0, pos_max = 23, n_tokens = 24, size = 0.985 MiB)
|
||||
0.16.488.638 I slot print_timing: id 0 | task 23 | prompt eval time = 115.47 ms / 28 tokens ( 4.12 ms per token, 242.48 tokens per second)
|
||||
0.16.488.687 I slot print_timing: id 0 | task 23 | eval time = 21.04 ms / 2 tokens ( 21.04 ms per token, 47.52 tokens per second)
|
||||
0.16.488.688 I slot print_timing: id 0 | task 23 | total time = 136.52 ms / 30 tokens
|
||||
0.16.488.690 I slot print_timing: id 0 | task 23 | graphs reused = 7
|
||||
0.16.488.762 I slot release: id 0 | task 23 | stop processing: n_tokens = 29, truncated = 0
|
||||
0.16.488.786 I srv update_slots: all slots are idle
|
||||
0.16.500.589 I srv server_strea: conv_id= (empty=1)
|
||||
0.16.501.049 I srv operator(): chat format: peg-native
|
||||
0.16.501.259 I slot get_availabl: id 0 | task -1 | - checking sim = 0.444 (12/27) > 0.100
|
||||
0.16.501.343 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, f_sim_best = 0.444 (> 0.100 thold), f_keep = 0.414
|
||||
0.16.501.410 I slot launch_slot_: id 0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> ?top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
|
||||
0.16.501.525 I slot launch_slot_: id 0 | task -1 | sampler params:
|
||||
repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
|
||||
dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 64
|
||||
top_k = 0, top_p = 0.950, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.000
|
||||
mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000, adaptive_target = -1.000, adaptive_decay = 0.900
|
||||
0.16.501.583 I slot launch_slot_: id 0 | task 28 | processing task, is_child = 0
|
||||
0.16.501.596 I slot operator(): id 0 | task 28 | new prompt, n_ctx_slot = 4096, n_keep = 0, task.n_tokens = 27
|
||||
0.16.501.601 I slot operator(): id 0 | task 28 | erased invalidated context checkpoint (pos_min = 0, pos_max = 9, n_tokens = 10, n_swa = 512, pos_next = 0, size = 0.411 MiB)
|
||||
0.16.501.603 I slot operator(): id 0 | task 28 | erased invalidated context checkpoint (pos_min = 0, pos_max = 23, n_tokens = 24, n_swa = 512, pos_next = 0, size = 0.985 MiB)
|
||||
0.16.501.607 I slot operator(): id 0 | task 28 | cached n_tokens = 0, memory_seq_rm [0, end)
|
||||
0.16.540.657 I srv operator(): operator(): cleaning up before exit...
|
||||
+127
@@ -0,0 +1,127 @@
|
||||
INFO 09-24 05:03:48 [patch.py:259] NVFP4 W4A4 weight_scale NaN-clamp: installed.
|
||||
INFO 09-24 05:03:48 [patch.py:510] inductor factorable-divisibility patch: installed.
|
||||
INFO 09-24 05:03:51 [stage_init_utils.py:768] [stage_init] Set VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
WARNING 09-24 05:03:51 [stage_init_utils.py:1887] [stage_init] Failed to load transfer config: [Errno 13] Permission denied: 'C:\\Users\\zhout'
|
||||
{
|
||||
"scope": "complete Spark GGUF text requests through Omni StageRuntime/StagePool and external llama.cpp backend",
|
||||
"model_file": "\\\\wsl.localhost\\Ubuntu\\home\\zhout\\project\\edge_infer\\models\\Spark-X2.5-1.7B-GGUF\\Spark-X2.5-1.7B-Q4_K_M.gguf",
|
||||
"device": "cpu",
|
||||
"memory_budget": {
|
||||
"capacities": {
|
||||
"host_ram": 12884901888
|
||||
},
|
||||
"demands": {
|
||||
"host_ram": 4294967296
|
||||
}
|
||||
},
|
||||
"warmup_count": 0,
|
||||
"measured_count": 1,
|
||||
"startup_s": 11.32228349999059,
|
||||
"execution_plan": {
|
||||
"backend": "external.llamacpp.text.v1",
|
||||
"stage_id": 0,
|
||||
"worker_generation": "aa53b6a5524d43a3accb26ed382a2cb7",
|
||||
"model_sha256": "902bde2522394954ac17821b3e5fd0df02defbc6944f122253f2580acf0503f4",
|
||||
"server_sha256": "9ffc5919acb4cb43c7be5f3053b59014cce70a87d3a801fcad49c4f459984f52",
|
||||
"model_alias": "\\\\wsl.localhost\\Ubuntu\\home\\zhout\\project\\edge_infer\\models\\Spark-X2.5-1.7B-GGUF\\Spark-X2.5-1.7B-Q4_K_M.gguf",
|
||||
"requested_device": "cpu",
|
||||
"expected_device_name": null,
|
||||
"layer_assignments": 0,
|
||||
"offloaded_layers": [
|
||||
"0",
|
||||
"29"
|
||||
],
|
||||
"model_buffer_devices": [],
|
||||
"all_model_buffers": [
|
||||
"CPU_Mapped",
|
||||
"CPU_REPACK"
|
||||
],
|
||||
"cpu_only_layout": false,
|
||||
"worker_pid": 65560,
|
||||
"loaded_rss_bytes": 1966919680,
|
||||
"reserved_bytes": {
|
||||
"host_ram": 4294967296
|
||||
},
|
||||
"memory_pool": "host_ram",
|
||||
"memory_overhead_bytes": 2147483648,
|
||||
"context_tokens": 4096,
|
||||
"max_new_tokens": 96,
|
||||
"max_io_bytes": 1048576,
|
||||
"request_capacity": 1,
|
||||
"stateful_session": "llama.cpp-owned; reset for each complete request",
|
||||
"evidence": "B"
|
||||
},
|
||||
"checks": [
|
||||
{
|
||||
"case": "short",
|
||||
"answer": "Paris",
|
||||
"expected": "Paris",
|
||||
"correct": true,
|
||||
"wall_s": 0.17689090000931174,
|
||||
"stage_event": {
|
||||
"request_id": "short-652178859000000",
|
||||
"stage_id": 0,
|
||||
"epoch": 1,
|
||||
"seq": 1,
|
||||
"kind": "text",
|
||||
"worker_generation": "aa53b6a5524d43a3accb26ed382a2cb7",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"llamacpp_wall_s": 0.17144279996864498,
|
||||
"usage": {
|
||||
"completion_tokens": 2,
|
||||
"prompt_tokens": 28,
|
||||
"total_tokens": 30,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"case": "inventory_120",
|
||||
"answer": "1511",
|
||||
"expected": "1511",
|
||||
"correct": true,
|
||||
"wall_s": 6.51016109995544,
|
||||
"stage_event": {
|
||||
"request_id": "inventory_120-652179031000000",
|
||||
"stage_id": 0,
|
||||
"epoch": 2,
|
||||
"seq": 1,
|
||||
"kind": "text",
|
||||
"worker_generation": "aa53b6a5524d43a3accb26ed382a2cb7",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"llamacpp_wall_s": 6.503754999954253,
|
||||
"usage": {
|
||||
"completion_tokens": 5,
|
||||
"prompt_tokens": 2463,
|
||||
"total_tokens": 2468,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"nearest_rank_p50_wall_s": 6.55064739997033,
|
||||
"nearest_rank_p95_wall_s": 6.55064739997033,
|
||||
"status": "passed",
|
||||
"ledger_after_shutdown": {
|
||||
"capacities": {
|
||||
"host_ram": 12884901888
|
||||
},
|
||||
"reserved": {
|
||||
"host_ram": 0
|
||||
},
|
||||
"owners": [],
|
||||
"quarantined": []
|
||||
}
|
||||
}
|
||||
+155
@@ -0,0 +1,155 @@
|
||||
{
|
||||
"scope": "complete Spark GGUF text requests through Omni StageRuntime/StagePool and external llama.cpp backend",
|
||||
"model_file": "\\\\wsl.localhost\\Ubuntu\\home\\zhout\\project\\edge_infer\\models\\Spark-X2.5-1.7B-GGUF\\Spark-X2.5-1.7B-Q4_K_M.gguf",
|
||||
"device": "cpu",
|
||||
"memory_budget": {
|
||||
"capacities": {
|
||||
"host_ram": 12884901888
|
||||
},
|
||||
"demands": {
|
||||
"host_ram": 4294967296
|
||||
}
|
||||
},
|
||||
"warmup_count": 0,
|
||||
"measured_count": 1,
|
||||
"startup_s": 11.32228349999059,
|
||||
"execution_plan": {
|
||||
"backend": "external.llamacpp.text.v1",
|
||||
"stage_id": 0,
|
||||
"worker_generation": "aa53b6a5524d43a3accb26ed382a2cb7",
|
||||
"model_sha256": "902bde2522394954ac17821b3e5fd0df02defbc6944f122253f2580acf0503f4",
|
||||
"server_sha256": "9ffc5919acb4cb43c7be5f3053b59014cce70a87d3a801fcad49c4f459984f52",
|
||||
"model_alias": "\\\\wsl.localhost\\Ubuntu\\home\\zhout\\project\\edge_infer\\models\\Spark-X2.5-1.7B-GGUF\\Spark-X2.5-1.7B-Q4_K_M.gguf",
|
||||
"requested_device": "cpu",
|
||||
"expected_device_name": null,
|
||||
"layer_assignments": 0,
|
||||
"offloaded_layers": [
|
||||
"0",
|
||||
"29"
|
||||
],
|
||||
"model_buffer_devices": [],
|
||||
"all_model_buffers": [
|
||||
"CPU_Mapped",
|
||||
"CPU_REPACK"
|
||||
],
|
||||
"cpu_only_layout": false,
|
||||
"worker_pid": 65560,
|
||||
"loaded_rss_bytes": 1966919680,
|
||||
"reserved_bytes": {
|
||||
"host_ram": 4294967296
|
||||
},
|
||||
"memory_pool": "host_ram",
|
||||
"memory_overhead_bytes": 2147483648,
|
||||
"context_tokens": 4096,
|
||||
"max_new_tokens": 96,
|
||||
"max_io_bytes": 1048576,
|
||||
"request_capacity": 1,
|
||||
"stateful_session": "llama.cpp-owned; reset for each complete request",
|
||||
"evidence": "B"
|
||||
},
|
||||
"checks": [
|
||||
{
|
||||
"case": "short",
|
||||
"answer": "Paris",
|
||||
"expected": "Paris",
|
||||
"correct": true,
|
||||
"wall_s": 0.17689090000931174,
|
||||
"stage_event": {
|
||||
"request_id": "short-652178859000000",
|
||||
"stage_id": 0,
|
||||
"epoch": 1,
|
||||
"seq": 1,
|
||||
"kind": "text",
|
||||
"worker_generation": "aa53b6a5524d43a3accb26ed382a2cb7",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"llamacpp_wall_s": 0.17144279996864498,
|
||||
"usage": {
|
||||
"completion_tokens": 2,
|
||||
"prompt_tokens": 28,
|
||||
"total_tokens": 30,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"case": "inventory_120",
|
||||
"answer": "1511",
|
||||
"expected": "1511",
|
||||
"correct": true,
|
||||
"wall_s": 6.51016109995544,
|
||||
"stage_event": {
|
||||
"request_id": "inventory_120-652179031000000",
|
||||
"stage_id": 0,
|
||||
"epoch": 2,
|
||||
"seq": 1,
|
||||
"kind": "text",
|
||||
"worker_generation": "aa53b6a5524d43a3accb26ed382a2cb7",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"llamacpp_wall_s": 6.503754999954253,
|
||||
"usage": {
|
||||
"completion_tokens": 5,
|
||||
"prompt_tokens": 2463,
|
||||
"total_tokens": 2468,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"warmups": [],
|
||||
"measured": [
|
||||
{
|
||||
"case": "inventory_120",
|
||||
"answer": "1511",
|
||||
"expected": "1511",
|
||||
"correct": true,
|
||||
"wall_s": 6.55064739997033,
|
||||
"stage_event": {
|
||||
"request_id": "inventory_120-652185546000000",
|
||||
"stage_id": 0,
|
||||
"epoch": 3,
|
||||
"seq": 1,
|
||||
"kind": "text",
|
||||
"worker_generation": "aa53b6a5524d43a3accb26ed382a2cb7",
|
||||
"buffers": [],
|
||||
"terminal": true,
|
||||
"error": null
|
||||
},
|
||||
"metrics": {
|
||||
"llamacpp_wall_s": 6.524226800072938,
|
||||
"usage": {
|
||||
"completion_tokens": 5,
|
||||
"prompt_tokens": 2463,
|
||||
"total_tokens": 2468,
|
||||
"prompt_tokens_details": {
|
||||
"cached_tokens": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"nearest_rank_p50_wall_s": 6.55064739997033,
|
||||
"nearest_rank_p95_wall_s": 6.55064739997033,
|
||||
"status": "passed",
|
||||
"ledger_after_shutdown": {
|
||||
"capacities": {
|
||||
"host_ram": 12884901888
|
||||
},
|
||||
"reserved": {
|
||||
"host_ram": 0
|
||||
},
|
||||
"owners": [],
|
||||
"quarantined": []
|
||||
}
|
||||
}
|
||||
+272
@@ -0,0 +1,272 @@
|
||||
0.00.000.769 I srv llama_server: initializing ...
|
||||
0.00.420.651 I cmn common_param: common_params_print_info: build 11124 (1a679828f) with Clang 20.1.8 for Windows x86_64
|
||||
0.00.420.666 I cmn common_param: common_params_print_info: verbosity = 4 (adjust with the `-lv N` CLI arg)
|
||||
0.00.420.667 I cmn common_param: device_info:
|
||||
0.00.424.092 I cmn common_param: - Vulkan0 : NVIDIA GeForce RTX 5090 Laptop GPU (24137 MiB, 23369 MiB free)
|
||||
0.00.427.301 I cmn common_param: - Vulkan1 : AMD Radeon(TM) 890M Graphics (48505 MiB, 46080 MiB free)
|
||||
0.00.427.311 I cmn common_param: - CPU : AMD Ryzen AI 9 HX 370 w/ Radeon 890M (64633 MiB, 26267 MiB free)
|
||||
0.00.427.359 I cmn common_param: system_info: n_threads = 12 (n_threads_batch = 12) / 24 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
|
||||
0.00.427.432 I srv init_listene: running without SSL
|
||||
0.00.427.489 I srv init_listene: The UI is disabled
|
||||
0.00.427.489 I srv init_listene: Use --ui/--no-ui (or deprecated --webui/--no-webui) to enable/disable
|
||||
0.00.427.492 I srv init: using 23 threads for HTTP server
|
||||
0.00.427.552 W srv llama_server: security: no API key is set and CORS allows all origins (see https://github.com/ggml-org/llama.cpp/pull/25655)
|
||||
0.00.433.937 I srv load_model: loading model '\\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\Spark-X2.5-1.7B-GGUF\Spark-X2.5-1.7B-Q4_K_M.gguf'
|
||||
0.00.433.953 I srv load_model: local path '\\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\Spark-X2.5-1.7B-GGUF\Spark-X2.5-1.7B-Q4_K_M.gguf'
|
||||
0.00.931.147 I llama_model_loader: loaded meta data with 37 key-value pairs and 226 tensors from \\wsl.localhost\Ubuntu\home\zhout\project\edge_infer\models\Spark-X2.5-1.7B-GGUF\Spark-X2.5-1.7B-Q4_K_M.gguf (version GGUF V3 (latest))
|
||||
0.00.931.163 I llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
|
||||
0.00.931.171 I llama_model_loader: - kv 0: general.architecture str = spark2_5
|
||||
0.00.931.172 I llama_model_loader: - kv 1: general.type str = model
|
||||
0.00.931.176 I llama_model_loader: - kv 2: general.sampling.top_k i32 = 0
|
||||
0.00.931.184 I llama_model_loader: - kv 3: general.sampling.top_p f32 = 0.950000
|
||||
0.00.931.185 I llama_model_loader: - kv 4: general.sampling.temp f32 = 1.000000
|
||||
0.00.931.185 I llama_model_loader: - kv 5: general.name str = Hf_Format
|
||||
0.00.931.186 I llama_model_loader: - kv 6: general.size_label str = 1.7B
|
||||
0.00.931.189 I llama_model_loader: - kv 7: spark2_5.block_count u32 = 28
|
||||
0.00.931.190 I llama_model_loader: - kv 8: spark2_5.context_length u32 = 1048576
|
||||
0.00.931.190 I llama_model_loader: - kv 9: spark2_5.embedding_length u32 = 2048
|
||||
0.00.931.191 I llama_model_loader: - kv 10: spark2_5.feed_forward_length u32 = 6656
|
||||
0.00.931.191 I llama_model_loader: - kv 11: spark2_5.attention.head_count u32 = 8
|
||||
0.00.931.191 I llama_model_loader: - kv 12: spark2_5.attention.head_count_kv u32 = 2
|
||||
0.00.931.193 I llama_model_loader: - kv 13: spark2_5.rope.freq_base f32 = 5000000.000000
|
||||
0.00.931.194 I llama_model_loader: - kv 14: spark2_5.rope.freq_base_swa f32 = 10000.000000
|
||||
0.00.931.195 I llama_model_loader: - kv 15: spark2_5.attention.layer_norm_rms_epsilon f32 = 0.000001
|
||||
0.00.931.196 I llama_model_loader: - kv 16: spark2_5.attention.key_length u32 = 256
|
||||
0.00.931.196 I llama_model_loader: - kv 17: spark2_5.attention.value_length u32 = 256
|
||||
0.00.931.197 I llama_model_loader: - kv 18: spark2_5.vocab_size u32 = 131072
|
||||
0.00.931.197 I llama_model_loader: - kv 19: spark2_5.attention.sliding_window u32 = 512
|
||||
0.00.931.223 I llama_model_loader: - kv 20: spark2_5.attention.sliding_window_pattern arr[bool,28] = [true, true, true, false, true, true,...
|
||||
0.00.931.223 I llama_model_loader: - kv 21: spark2_5.rope.dimension_count u32 = 64
|
||||
0.00.931.224 I llama_model_loader: - kv 22: spark2_5.rope.dimension_count_swa u32 = 256
|
||||
0.00.931.224 I llama_model_loader: - kv 23: tokenizer.ggml.model str = gpt2
|
||||
0.00.931.225 I llama_model_loader: - kv 24: tokenizer.ggml.pre str = spark2_5
|
||||
0.00.947.061 I llama_model_loader: - kv 25: tokenizer.ggml.tokens arr[str,131072] = ["<｜startâ–�ofâ–�sentence｜>", "<ï...
|
||||
0.00.951.790 I llama_model_loader: - kv 26: tokenizer.ggml.token_type arr[i32,131072] = [3, 3, 3, 4, 4, 3, 3, 3, 4, 4, 4, 3, ...
|
||||
0.00.967.789 I llama_model_loader: - kv 27: tokenizer.ggml.merges arr[str,130716] = ["Ä Ä ", "Ä t", "i n", "Ä a", "Ä Ä...
|
||||
0.00.967.797 I llama_model_loader: - kv 28: tokenizer.ggml.bos_token_id u32 = 0
|
||||
0.00.967.798 I llama_model_loader: - kv 29: tokenizer.ggml.eos_token_id u32 = 1
|
||||
0.00.967.799 I llama_model_loader: - kv 30: tokenizer.ggml.unknown_token_id u32 = 5
|
||||
0.00.967.799 I llama_model_loader: - kv 31: tokenizer.ggml.padding_token_id u32 = 2
|
||||
0.00.967.802 I llama_model_loader: - kv 32: tokenizer.ggml.add_bos_token bool = false
|
||||
0.00.967.803 I llama_model_loader: - kv 33: tokenizer.ggml.add_eos_token bool = false
|
||||
0.00.967.807 I llama_model_loader: - kv 34: tokenizer.chat_template str = {%- if not messages %}\n {{- raise_...
|
||||
0.00.967.807 I llama_model_loader: - kv 35: general.quantization_version u32 = 2
|
||||
0.00.967.808 I llama_model_loader: - kv 36: general.file_type u32 = 15
|
||||
0.00.967.817 I llama_model_loader: - type f32: 57 tensors
|
||||
0.00.967.817 I llama_model_loader: - type q4_K: 140 tensors
|
||||
0.00.967.818 I llama_model_loader: - type q6_K: 29 tensors
|
||||
0.00.967.822 I print_info: file format = GGUF V3 (latest)
|
||||
0.00.967.823 I print_info: file type = Q4_K - Medium
|
||||
0.00.967.828 I print_info: file size = 1.03 GiB (5.16 BPW)
|
||||
0.01.015.171 I load: 86 unused tokens
|
||||
0.01.033.298 I load: printing all EOG tokens:
|
||||
0.01.033.305 I load: - 1 ('<｜end�of�sentence｜>')
|
||||
0.01.033.425 I load: special tokens cache size = 113
|
||||
0.01.048.672 I load: token to piece cache size = 0.7975 MB
|
||||
0.01.048.697 I print_info: arch = spark2_5
|
||||
0.01.048.698 I print_info: vocab_only = 0
|
||||
0.01.048.698 I print_info: no_alloc = 0
|
||||
0.01.048.699 I print_info: n_ctx_train = 1048576
|
||||
0.01.048.699 I print_info: n_embd_inp = 2048
|
||||
0.01.048.700 I print_info: n_embd = 2048
|
||||
0.01.048.700 I print_info: n_embd_out = 2048
|
||||
0.01.048.700 I print_info: n_layer = 28
|
||||
0.01.048.701 I print_info: n_layer_all = 28
|
||||
0.01.048.718 I print_info: n_head = 8
|
||||
0.01.048.720 I print_info: n_head_kv = 2
|
||||
0.01.048.720 I print_info: n_rot = 64
|
||||
0.01.048.721 I print_info: n_swa = 512
|
||||
0.01.048.721 I print_info: is_swa_any = 1
|
||||
0.01.048.721 I print_info: non_causal_type = 0
|
||||
0.01.048.722 I print_info: n_embd_head_k = 256
|
||||
0.01.048.722 I print_info: n_embd_head_v = 256
|
||||
0.01.048.723 I print_info: n_gqa = 4
|
||||
0.01.048.725 I print_info: n_embd_k_gqa = 512
|
||||
0.01.048.726 I print_info: n_embd_v_gqa = 512
|
||||
0.01.048.729 I print_info: f_norm_eps = 0.0e+00
|
||||
0.01.048.730 I print_info: f_norm_rms_eps = 1.0e-06
|
||||
0.01.048.731 I print_info: f_clamp_kqv = 0.0e+00
|
||||
0.01.048.731 I print_info: f_max_alibi_bias = 0.0e+00
|
||||
0.01.048.731 I print_info: f_logit_scale = 0.0e+00
|
||||
0.01.048.732 I print_info: f_attn_scale = 0.0e+00
|
||||
0.01.048.732 I print_info: f_attn_value_scale = 0.0000
|
||||
0.01.048.734 I print_info: n_ff = 6656
|
||||
0.01.048.734 I print_info: n_expert = 0
|
||||
0.01.048.734 I print_info: n_expert_used = 0
|
||||
0.01.048.736 I print_info: n_expert_groups = 0
|
||||
0.01.048.736 I print_info: n_group_used = 0
|
||||
0.01.048.736 I print_info: causal attn = 1
|
||||
0.01.048.737 I print_info: pooling type = -1
|
||||
0.01.048.737 I print_info: rope type = 2
|
||||
0.01.048.737 I print_info: rope scaling = linear
|
||||
0.01.048.738 I print_info: freq_base_train = 5000000.0
|
||||
0.01.048.740 I print_info: freq_scale_train = 1
|
||||
0.01.048.740 I print_info: freq_base_swa = 10000.0
|
||||
0.01.048.741 I print_info: freq_scale_swa = 1
|
||||
0.01.048.741 I print_info: n_embd_head_k_swa = 256
|
||||
0.01.048.742 I print_info: n_embd_head_v_swa = 256
|
||||
0.01.048.742 I print_info: n_rot_swa = 256
|
||||
0.01.048.742 I print_info: n_ctx_orig_yarn = 1048576
|
||||
0.01.048.742 I print_info: rope_yarn_log_mul = 0.0000
|
||||
0.01.048.743 I print_info: rope_finetuned = unknown
|
||||
0.01.048.744 I print_info: model type = 1.7B
|
||||
0.01.048.745 I print_info: model params = 1.71 B
|
||||
0.01.048.745 I print_info: general.name = Hf_Format
|
||||
0.01.048.746 I print_info: vocab type = BPE
|
||||
0.01.048.746 I print_info: n_vocab = 131072
|
||||
0.01.048.746 I print_info: n_merges = 130716
|
||||
0.01.048.747 I print_info: BOS token = 0 '<｜start�of�sentence｜>'
|
||||
0.01.048.747 I print_info: EOS token = 1 '<｜end�of�sentence｜>'
|
||||
0.01.048.748 I print_info: EOT token = 1 '<｜end�of�sentence｜>'
|
||||
0.01.048.748 I print_info: UNK token = 5 '<unk>'
|
||||
0.01.048.749 I print_info: PAD token = 2 '<｜�pad�｜>'
|
||||
0.01.048.749 I print_info: LF token = 198 'ÄŠ'
|
||||
0.01.048.750 I print_info: FIM PRE token = 11 '<｜fim�begin｜>'
|
||||
0.01.048.750 I print_info: FIM SUF token = 12 '<｜fim�hole｜>'
|
||||
0.01.048.750 I print_info: FIM MID token = 13 '<｜fim�end｜>'
|
||||
0.01.048.751 I print_info: EOG token = 1 '<｜end�of�sentence｜>'
|
||||
0.01.048.751 I print_info: max token length = 210
|
||||
0.01.048.753 I load_tensors: loading model tensors, this can take a while... (load_mode = mmap)
|
||||
0.03.584.123 W warning: PrefetchVirtualMemory failed: ÄÚ´æ×ÊÔ´²»×㣬ÎÞ·¨´¦Àí´ËÃüÁî¡£
|
||||
|
||||
|
||||
0.03.584.408 I load_tensors: offloading 0 repeating layers to GPU
|
||||
0.03.584.409 I load_tensors: offloaded 0/29 layers to GPU
|
||||
0.03.584.413 I load_tensors: CPU_Mapped model buffer size = 1043.71 MiB
|
||||
0.03.584.414 I load_tensors: CPU_REPACK model buffer size = 622.37 MiB
|
||||
0.04.260.149 I cmn common_init_: added <｜end�of�sentence｜> logit bias = -inf
|
||||
0.04.261.930 I llama_context: constructing llama_context
|
||||
0.04.261.940 I llama_context: n_seq_max = 1
|
||||
0.04.261.941 I llama_context: n_ctx = 4096
|
||||
0.04.261.941 I llama_context: n_ctx_seq = 4096
|
||||
0.04.261.941 I llama_context: n_batch = 2048
|
||||
0.04.261.941 I llama_context: n_ubatch = 512
|
||||
0.04.261.942 I llama_context: causal_attn = 1
|
||||
0.04.261.943 I llama_context: flash_attn = auto
|
||||
0.04.261.943 I llama_context: kv_unified = false
|
||||
0.04.261.948 I llama_context: freq_base = 5000000.0
|
||||
0.04.261.950 I llama_context: freq_scale = 1
|
||||
0.04.261.950 I llama_context: n_rs_seq = 0
|
||||
0.04.261.951 I llama_context: n_outputs_max = 1
|
||||
0.04.261.951 I llama_context: n_outputs_max_per_seq = 1
|
||||
0.04.261.952 I llama_context: n_ctx_seq (4096) < n_ctx_train (1048576) -- the full capacity of the model will not be utilized
|
||||
0.04.262.075 I llama_context: CPU output buffer size = 0.50 MiB
|
||||
0.04.262.079 I llama_kv_cache_iswa: creating non-SWA KV cache, size = 4096 cells
|
||||
0.04.262.186 I llama_kv_cache: CPU KV buffer size = 56.00 MiB
|
||||
0.04.267.251 I llama_kv_cache: size = 56.00 MiB ( 4096 cells, 7 layers, 1/1 seqs), K (f16): 28.00 MiB, V (f16): 28.00 MiB
|
||||
0.04.267.255 I llama_kv_cache: attn_rot_k = 0, n_embd_head_k_all = 256
|
||||
0.04.267.255 I llama_kv_cache: attn_rot_v = 0, n_embd_head_k_all = 256
|
||||
0.04.267.261 I llama_kv_cache_iswa: creating SWA KV cache, size = 1024 cells
|
||||
0.04.267.330 I llama_kv_cache: CPU KV buffer size = 42.00 MiB
|
||||
0.04.269.981 I llama_kv_cache: size = 42.00 MiB ( 1024 cells, 21 layers, 1/1 seqs), K (f16): 21.00 MiB, V (f16): 21.00 MiB
|
||||
0.04.269.983 I llama_kv_cache: attn_rot_k = 0, n_embd_head_k_all = 256
|
||||
0.04.269.983 I llama_kv_cache: attn_rot_v = 0, n_embd_head_k_all = 256
|
||||
0.04.269.987 I sched_reserve: reserving ...
|
||||
0.04.270.753 I resolve_fused_ops: Flash Attention enabled
|
||||
0.04.270.754 I resolve_fused_ops: resolving fused DeepSeek V4 HC support:
|
||||
0.04.271.019 I resolve_fused_ops: fused DeepSeek V4 HC pre enabled
|
||||
0.04.271.269 I resolve_fused_ops: fused DeepSeek V4 HC comb enabled
|
||||
0.04.271.516 I resolve_fused_ops: fused DeepSeek V4 HC post enabled
|
||||
0.04.272.481 I sched_reserve: CPU compute buffer size = 60.02 MiB
|
||||
0.04.272.483 I sched_reserve: graph: nodes = 986, splits = 1, input objects = 4, input tensors = 9
|
||||
0.04.272.484 I sched_reserve: reserve took 2.49 ms, sched copies = 1
|
||||
0.04.272.582 I cmn init: llama threadpool init, n_threads = 12
|
||||
0.04.272.868 I cmn common_init_: warming up the model with an empty run - please wait ... (--no-warmup to disable)
|
||||
0.04.454.529 I srv load_model: initializing, n_slots = 1, n_ctx_slot = 4096, kv_unified = 'false'
|
||||
0.04.455.118 I spec common_specu: no implementations specified for speculative decoding
|
||||
0.04.455.120 I slot load_model: id 0 | task -1 | new slot, n_ctx = 4096
|
||||
0.04.455.259 I srv load_model: prompt cache is disabled - use `--cache-ram N` to enable it
|
||||
0.04.455.260 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
|
||||
0.04.455.260 I srv load_model: context checkpoints enabled, max = 32, min spacing = 8192
|
||||
0.04.455.600 W srv init: --cache-idle-slots requires --cache-ram, disabling
|
||||
0.04.486.277 I srv init: init: chat template, example_format: '<｜start�of�sentence｜><|System|>
|
||||
you are a helpful assistant.
|
||||
|
||||
You are a helpful assistant<｜end�of�sentence｜><｜start�of�sentence｜><|User|>Hello<｜end�of�sentence｜><｜start�of�sentence｜><|Bot|></think>Hi there<｜end�of�sentence｜><｜start�of�sentence｜><|User|>How are you?<｜end�of�sentence｜><｜start�of�sentence｜><|Bot|></think>'
|
||||
0.04.498.237 I srv init: init: chat template, thinking = 0
|
||||
0.04.498.250 I srv init: preserve_reasoning kwarg: true
|
||||
0.04.498.250 W srv init: chat template supports preserving reasoning, it is enabled by default (may use more tokens, disable via --no-reasoning-preserve)
|
||||
0.04.498.474 I srv llama_server: model loaded
|
||||
0.04.498.477 I srv llama_server: listening on http://127.0.0.1:64110
|
||||
0.04.498.701 I srv update_slots: all slots are idle
|
||||
0.04.667.613 I srv server_strea: conv_id= (empty=1)
|
||||
0.04.668.602 I srv operator(): chat format: peg-native
|
||||
0.04.669.012 I slot get_availabl: id 0 | task -1 | - skipping, slot is empty
|
||||
0.04.669.013 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = -1
|
||||
0.04.669.187 I slot launch_slot_: id 0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> ?top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
|
||||
0.04.669.204 I slot launch_slot_: id 0 | task -1 | sampler params:
|
||||
repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
|
||||
dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 64
|
||||
top_k = 0, top_p = 0.950, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.000
|
||||
mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000, adaptive_target = -1.000, adaptive_decay = 0.900
|
||||
0.04.669.207 I slot launch_slot_: id 0 | task 0 | processing task, is_child = 0
|
||||
0.04.669.383 I slot operator(): id 0 | task 0 | new prompt, n_ctx_slot = 4096, n_keep = 0, task.n_tokens = 28
|
||||
0.04.669.506 I slot operator(): id 0 | task 0 | cached n_tokens = 0, memory_seq_rm [0, end)
|
||||
0.04.704.343 I slot operator(): id 0 | task 0 | cached n_tokens = 10, memory_seq_rm [10, end)
|
||||
0.04.704.517 I slot create_check: id 0 | task 0 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 9, n_tokens = 10, size = 0.411 MiB)
|
||||
0.04.750.993 I slot operator(): id 0 | task 0 | cached n_tokens = 24, memory_seq_rm [24, end)
|
||||
0.04.751.058 I slot init_sampler: id 0 | task 0 | init sampler, took 0.01 ms, tokens: text = 28, total = 28
|
||||
0.04.751.304 I slot create_check: id 0 | task 0 | created context checkpoint 2 of 32 (pos_min = 0, pos_max = 23, n_tokens = 24, size = 0.985 MiB)
|
||||
0.04.796.257 I slot print_timing: id 0 | task 0 | prompt eval time = 106.66 ms / 28 tokens ( 3.81 ms per token, 262.51 tokens per second)
|
||||
0.04.796.268 I slot print_timing: id 0 | task 0 | eval time = 20.18 ms / 2 tokens ( 20.18 ms per token, 49.55 tokens per second)
|
||||
0.04.796.269 I slot print_timing: id 0 | task 0 | total time = 126.84 ms / 30 tokens
|
||||
0.04.796.271 I slot print_timing: id 0 | task 0 | graphs reused = 1
|
||||
0.04.796.308 I slot release: id 0 | task 0 | stop processing: n_tokens = 29, truncated = 0
|
||||
0.04.796.326 I srv update_slots: all slots are idle
|
||||
0.04.838.317 I srv server_strea: conv_id= (empty=1)
|
||||
0.04.840.703 I srv operator(): chat format: peg-native
|
||||
0.04.840.884 I slot get_availabl: id 0 | task -1 | - checking sim = 0.005 (12/2463) > 0.100
|
||||
0.04.840.885 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = 4377727
|
||||
0.04.841.172 I slot launch_slot_: id 0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> ?top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
|
||||
0.04.841.187 I slot launch_slot_: id 0 | task -1 | sampler params:
|
||||
repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
|
||||
dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 64
|
||||
top_k = 0, top_p = 0.950, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.000
|
||||
mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000, adaptive_target = -1.000, adaptive_decay = 0.900
|
||||
0.04.841.190 I slot launch_slot_: id 0 | task 5 | processing task, is_child = 0
|
||||
0.04.841.199 I slot operator(): id 0 | task 5 | new prompt, n_ctx_slot = 4096, n_keep = 0, task.n_tokens = 2463
|
||||
0.04.841.202 I slot operator(): id 0 | task 5 | erased invalidated context checkpoint (pos_min = 0, pos_max = 9, n_tokens = 10, n_swa = 512, pos_next = 0, size = 0.411 MiB)
|
||||
0.04.841.205 I slot operator(): id 0 | task 5 | erased invalidated context checkpoint (pos_min = 0, pos_max = 23, n_tokens = 24, n_swa = 512, pos_next = 0, size = 0.985 MiB)
|
||||
0.04.841.207 I slot operator(): id 0 | task 5 | cached n_tokens = 0, memory_seq_rm [0, end)
|
||||
0.04.877.826 I slot operator(): id 0 | task 5 | cached n_tokens = 10, memory_seq_rm [10, end)
|
||||
0.04.877.968 I slot create_check: id 0 | task 5 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 9, n_tokens = 10, size = 0.411 MiB)
|
||||
0.09.771.220 I slot print_timing: id 0 | task 5 | prompt processing, n_tokens = 1947, progress = 0.79, t = 4.93 s / 394.93 tokens per second
|
||||
0.09.771.232 I slot operator(): id 0 | task 5 | cached n_tokens = 1947, memory_seq_rm [1947, end)
|
||||
0.09.775.757 I slot create_check: id 0 | task 5 | created context checkpoint 2 of 32 (pos_min = 923, pos_max = 1946, n_tokens = 1947, size = 21.006 MiB)
|
||||
0.11.184.886 I slot print_timing: id 0 | task 5 | prompt processing, n_tokens = 2459, progress = 1.00, t = 6.34 s / 387.63 tokens per second
|
||||
0.11.184.894 I slot operator(): id 0 | task 5 | cached n_tokens = 2459, memory_seq_rm [2459, end)
|
||||
0.11.185.255 I slot init_sampler: id 0 | task 5 | init sampler, took 0.34 ms, tokens: text = 2463, total = 2463
|
||||
0.11.189.299 I slot create_check: id 0 | task 5 | created context checkpoint 3 of 32 (pos_min = 1435, pos_max = 2458, n_tokens = 2459, size = 21.006 MiB)
|
||||
0.11.308.112 I slot print_timing: id 0 | task 5 | prompt eval time = 6378.97 ms / 2463 tokens ( 2.59 ms per token, 386.11 tokens per second)
|
||||
0.11.308.122 I slot print_timing: id 0 | task 5 | eval time = 87.93 ms / 5 tokens ( 21.98 ms per token, 45.49 tokens per second)
|
||||
0.11.308.124 I slot print_timing: id 0 | task 5 | total time = 6466.90 ms / 2468 tokens
|
||||
0.11.308.124 I slot print_timing: id 0 | task 5 | graphs reused = 4
|
||||
0.11.308.170 I slot release: id 0 | task 5 | stop processing: n_tokens = 2467, truncated = 0
|
||||
0.11.308.185 I srv update_slots: all slots are idle
|
||||
0.11.349.312 I srv server_strea: conv_id= (empty=1)
|
||||
0.11.352.598 I srv operator(): chat format: peg-native
|
||||
0.11.352.839 I slot get_availabl: id 0 | task -1 | - checking sim = 1.000 (2463/2463) > 0.100
|
||||
0.11.352.843 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, f_sim_best = 1.000 (> 0.100 thold), f_keep = 0.998
|
||||
0.11.353.187 I slot launch_slot_: id 0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> ?top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
|
||||
0.11.353.199 I slot launch_slot_: id 0 | task -1 | sampler params:
|
||||
repeat_last_n = 64, repeat_penalty = 1.000, frequency_penalty = 0.000, presence_penalty = 0.000
|
||||
dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 64
|
||||
top_k = 0, top_p = 0.950, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.000
|
||||
mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000, adaptive_target = -1.000, adaptive_decay = 0.900
|
||||
0.11.353.202 I slot launch_slot_: id 0 | task 14 | processing task, is_child = 0
|
||||
0.11.353.210 I slot operator(): id 0 | task 14 | new prompt, n_ctx_slot = 4096, n_keep = 0, task.n_tokens = 2463
|
||||
0.11.353.213 I slot operator(): id 0 | task 14 | erased invalidated context checkpoint (pos_min = 0, pos_max = 9, n_tokens = 10, n_swa = 512, pos_next = 0, size = 0.411 MiB)
|
||||
0.11.353.216 I slot operator(): id 0 | task 14 | erased invalidated context checkpoint (pos_min = 923, pos_max = 1946, n_tokens = 1947, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
0.11.354.030 I slot operator(): id 0 | task 14 | erased invalidated context checkpoint (pos_min = 1435, pos_max = 2458, n_tokens = 2459, n_swa = 512, pos_next = 0, size = 21.006 MiB)
|
||||
0.11.354.834 I slot operator(): id 0 | task 14 | cached n_tokens = 0, memory_seq_rm [0, end)
|
||||
0.11.389.964 I slot operator(): id 0 | task 14 | cached n_tokens = 10, memory_seq_rm [10, end)
|
||||
0.11.390.094 I slot create_check: id 0 | task 14 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 9, n_tokens = 10, size = 0.411 MiB)
|
||||
0.16.308.808 I slot print_timing: id 0 | task 14 | prompt processing, n_tokens = 1947, progress = 0.79, t = 4.96 s / 392.89 tokens per second
|
||||
0.16.308.824 I slot operator(): id 0 | task 14 | cached n_tokens = 1947, memory_seq_rm [1947, end)
|
||||
0.16.313.126 I slot create_check: id 0 | task 14 | created context checkpoint 2 of 32 (pos_min = 923, pos_max = 1946, n_tokens = 1947, size = 21.006 MiB)
|
||||
0.17.730.697 I slot print_timing: id 0 | task 14 | prompt processing, n_tokens = 2459, progress = 1.00, t = 6.38 s / 385.58 tokens per second
|
||||
0.17.730.705 I slot operator(): id 0 | task 14 | cached n_tokens = 2459, memory_seq_rm [2459, end)
|
||||
0.17.731.063 I slot init_sampler: id 0 | task 14 | init sampler, took 0.34 ms, tokens: text = 2463, total = 2463
|
||||
0.17.735.328 I slot create_check: id 0 | task 14 | created context checkpoint 3 of 32 (pos_min = 1435, pos_max = 2458, n_tokens = 2459, size = 21.006 MiB)
|
||||
0.17.858.770 I slot print_timing: id 0 | task 14 | prompt eval time = 6414.76 ms / 2463 tokens ( 2.60 ms per token, 383.96 tokens per second)
|
||||
0.17.858.780
|
||||
@@ -5,6 +5,8 @@ All 60 pairings have a disposition. The timing tables below remain the 2026-09-2
|
||||
The WSL CPU and CPU+Radeon InternVLA cells also have [bounded Omni complete-policy profiles](../../../e2e_expansion_20260923/evidence/internvla_omni_policy/README.md): one warmup and 20 serial patterned synthetic requests per placement, plus admission refusal, cancellation and a public hybrid `AsyncOmni.generate` request. The [WSL RTX 5090 Laptop CUDA](../../../e2e_expansion_20260923/evidence/internvla_omni_cuda/README.md) and [native Windows RTX CUDA](../../../e2e_expansion_20260923/evidence/internvla_omni_windows_cuda/README.md) cells now have the same scoped Omni whole-policy pattern with 20 serial requests each, explicit host/VRAM admission, refusal, cancellation and public action requests. Separate native Windows [CPU](../../../e2e_expansion_20260923/evidence/internvla_omni_windows_cpu/README.md) and [CPU+Radeon](../../../e2e_expansion_20260923/evidence/internvla_omni_windows_hybrid/README.md) Omni profiles add 20 serial patterned requests per placement, refusal, cancellation and public action requests. These advance only the synthetic action-policy path. Real observations/reference actions, physical action units/order/step time, task quality, loading peak and sustained power/thermal behavior remain unverified. The earlier [Cosmos component and direct hybrid results](../../../e2e_expansion_20260923/evidence/internvla_cosmos_radeon890m/README.md) remain separate experiments. A new [20-request native Radeon MiniCPM-o profile](../../../e2e_expansion_20260923/evidence/minicpmo_omni_radeon_profile20/README.md) passed the scoped combined input/output path but exposed a large memory-sensitive latency regression versus its earlier one-request smoke; its row below supersedes the smoke for repeated-request timing.
|
||||
|
||||
A [follow-up MiniCPM-o storage experiment](../../../e2e_expansion_20260923/evidence/minicpmo_omni_native_weights_io/README.md) copied the same ten GGUF weights to native NTFS. One warmup plus 20 complete requests passed on Windows CPU (p50/p95 23.62/23.88 s) and Radeon+CPU (16.35/17.20 s), with no prefetch warnings. Separate one-request UNC controls were much slower; the original UNC 20-request Radeon profile remains valid for that path. Run order and OS cache were not controlled, so no causal or cross-placement speedup claim follows. A [real InternVLA Cosmos encoder NPU artifact attempt](../../../e2e_expansion_20260923/evidence/internvla_cosmos_amd_npu/README.md) passed fixed-shape CPU export parity but its tested A16W8 candidates failed the CPU numerical gate or VitisAI compilation; the AMD NPU cell remains NOT E2E.
|
||||
|
||||
The [WSL CPU Spark GGUF Omni profile](../../../e2e_expansion_20260923/evidence/spark_omni_wsl_cpu/README.md) adds a separate current-Omni complete-request path to the earlier vLLM INT8 CPU result. It passed two named prompts, 20/20 serial measured inventory requests at p50/p95 7.82/7.97 s, a public text request, context refusal and cancellation cleanup. The CPU-only llama.cpp build required explicit CPU-buffer placement evidence because it omits the Windows offload-count log line. This does not establish token streaming or broad quality.
|
||||
|
||||
## Configuration × model
|
||||
|
||||
@@ -18,7 +20,7 @@ NOT E2E means no complete device/model workload was verified. “Scoped E2E” i
|
||||
|
||||
| Device | Spark-X2.5 | Qwen3-TTS 0.6B CustomVoice | Qwen3.8-27B | MiniCPM-o 4.5 | InternVLA-A1 |
|
||||
|---|---|---|---|---|---|
|
||||
| PC HX370: CPU / WSL | profile completed | scoped E2E: Omni CPU text → complete WAV | NOT E2E | scoped E2E: Omni GGUF combined image+audio → text+speech | synthetic policy pass: Omni CPU |
|
||||
| PC HX370: CPU / WSL | scoped E2E: Omni GGUF Spark text | scoped E2E: Omni CPU text → complete WAV | NOT E2E | scoped E2E: Omni GGUF combined image+audio → text+speech | synthetic policy pass: Omni CPU |
|
||||
| PC HX370: CPU / native Windows | scoped E2E: Omni Spark text | scoped E2E: Omni CPU text → complete WAV | NOT E2E | scoped E2E: Omni GGUF image+audio → text+speech (native-storage 20-run profile) | synthetic policy pass: Omni CPU |
|
||||
| PC HX370 + RTX 5090 Laptop / WSL | profile completed | profile completed | scoped E2E: text + image | scoped E2E: text/image/audio → speech | synthetic policy pass: admitted Omni CUDA |
|
||||
| PC HX370 + RTX 5090 Laptop / Windows | profile completed | profile completed | NOT E2E | scoped E2E: text/image/audio → speech | synthetic policy pass: admitted Omni CUDA |
|
||||
@@ -35,7 +37,7 @@ NOT E2E means no complete device/model workload was verified. “Scoped E2E” i
|
||||
|
||||
| Device | Model | Current profiling disposition | Reason / remaining work | Next step |
|
||||
|---|---|---|---|---|
|
||||
| PC HX370: CPU / WSL | Spark-X2.5 | PROFILE_COMPLETED | not fully qualified: reference-quality and remaining reliability/performance gates must be reviewed | Qualify a matching vLLM 0.29 CPU build and token parity separately. |
|
||||
| PC HX370: CPU / WSL | Spark-X2.5 | SCOPED_E2E_OMNI_COMPLETE_REQUEST | The pinned Spark-X2.5-1.7B Q4_K_M GGUF ran on a CPU-only Linux llama.cpp server through the current Omni 0.29 `StageRuntime`/`StagePool` whole-session backend. Both named prompts passed (`Paris`, `1511`); one warmup plus 20/20 serial inventory requests had nearest-rank complete-request wall p50/p95 7.82/7.97 s, excluding 1.61 s startup. The stage verified model/server hashes, explicit CPU-only tensor buffers (`CPU_Mapped`, `CPU_REPACK`) and no usable GPU, reserved 4 GiB host RAM, refused an over-context request without truncation, and after in-flight cancellation returned no stale output or reservation. A separate public `AsyncOmni.generate` text request passed. The Linux CPU-only build omits the Windows 0/29 offload line; the verifier was extended to require positive CPU buffer evidence rather than inferring CPU from an absent GPU count. This is separate from the historical vLLM INT8 CPU profile and does not prove model-wide quality, incremental token streaming, restart-after-cancel, loading peak or sustained performance. | Broaden text/token quality and context suite, qualify matching vLLM 0.29 CPU execution/token parity, then test streaming, restart, loading peak and sustained load; [WSL GGUF Omni raw evidence](../../../e2e_expansion_20260923/evidence/spark_omni_wsl_cpu/README.md). |
|
||||
| PC HX370: CPU / WSL | Qwen3-TTS 0.6B CustomVoice | SCOPED_E2E_OMNI_COMPLETE_REQUEST | The earlier vLLM 0.28 CPU path passed four short streams but had four simulated playback underruns in one measured request; its wheel remains mismatched with Omni 0.29. Separately, the pinned BF16 0.6B CustomVoice checkpoint ran two named text-to-complete-WAV requests through an Omni 0.29 whole-session stage with an isolated CPU worker on WSL. One warmup plus 20 serial measured requests returned identical 3.44 s audio; nearest-rank complete-request wall p50/p95 was 8.27/8.47 s (RTF 2.40/2.46), excluding 19.82 s startup. Pinned Whisper tiny.en transcribed the first measured WAV exactly (WER 0, one-output intelligibility proxy). Public `AsyncOmni` audio output passed; the stage verified weights, interpreter and CPU placement, reserved 10 GiB host RAM, refused 4 GiB before launch, and drained in-flight cancellation with zero remaining reservation. WSL PCM differs from the same-checkpoint native Windows outputs despite the declared settings; its cause is unqualified. Sampled process-tree RSS reached 3.39 GB, not a loading peak. This worker delivers only a complete WAV, not a playable stream. | Qualify cross-OS numerical/quality difference, playable streaming without underruns, post-cancel restart, loading peak, concurrency and sustained power/thermal behavior; [WSL Omni raw evidence](../../../e2e_expansion_20260923/evidence/qwen_tts_omni_wsl_cpu/README.md), [earlier stream evidence](../../../e2e_recovery_20260923/README.md). |
|
||||
| PC HX370: CPU / WSL | Qwen3.8-27B | NOT_E2E_PROFILEABLE | No compatible CPU artifact/deployment for this model was found in the checked model roots. Available Qwen 27B FP8/NVFP4 CUDA artifacts do not establish a CPU route. | Supply and validate the missing matching artifact/backend, then run the complete specified workload. |
|
||||
| PC HX370: CPU / WSL | MiniCPM-o 4.5 | SCOPED_E2E_OMNI_COMPLETE_REQUEST | The BF16/vLLM 0.28 three-stage path previously passed separate text, synthetic red-square image and synthetic 440 Hz audio inputs to text+speech; its 20-request serial **text-only** profile was p50/p95 21.86/22.97 s with swap and shutdown warnings. A separate pinned Q4_K_M/F16 GGUF worker now runs a **combined spoken question+image → text+speech** request through a bounded Omni StageRuntime/StagePool graph stage on WSL CPU, with installed vLLM 0.29 for the controller and CPU-only C++ model execution. All 20 measured requests after one warmup answered “The square in the image is red.”, emitted complete nonzero speech and a terminal audio event; nearest-rank complete-request wall p50/p95 was 22.67/24.83 s, speech duration 1.60–2.92 s, and maximum sampled worker RSS 13.70 GB. The first measured WAV independently transcribed exactly with pinned Whisper tiny.en (WER 0, one-output intelligibility proxy). Ten weight hashes and CPU placement were verified; 24 GiB shared RAM was reserved, 8 GiB was refused before launch, and in-flight cancellation left no stale output or reservation. Host RAM/swap samples covered only part of the profile. One synthetic fixture and variable speech PCM do not establish general quality, playable streaming, loading peak or sustained performance. | Validate real image/audio suites, speech alignment and quality, video input, concurrent admission, post-cancel restart, loading peak and sustained thermal behavior; [GGUF Omni raw evidence](../../../e2e_expansion_20260923/evidence/minicpmo_omni_wsl_cpu/README.md), [BF16/vLLM CPU evidence](../../../e2e_expansion_20260923/README.md), [earlier standalone GGUF run](../../../e2e_expansion_20260923/evidence/minicpmo_cpp_cpu/README.md). |
|
||||
|
||||
@@ -33,6 +33,7 @@ from vllm_omni.outputs import OmniRequestOutput
|
||||
_LAYER_ASSIGNMENT = re.compile(r"load_tensors: layer\s+\d+ assigned to device (\S+)")
|
||||
_OFFLOADED_LAYERS = re.compile(r"load_tensors: offloaded (\d+)/(\d+) layers to GPU")
|
||||
_MODEL_BUFFER = re.compile(r"load_tensors:\s+(Vulkan\d+) model buffer size")
|
||||
_ANY_MODEL_BUFFER = re.compile(r"load_tensors:\s+(\S+) model buffer size")
|
||||
|
||||
|
||||
def _digest(path: Path) -> str:
|
||||
@@ -160,9 +161,21 @@ class LlamaCppTextStageClient(StageClientBase):
|
||||
assignments = _LAYER_ASSIGNMENT.findall(log_text)
|
||||
offloaded = _OFFLOADED_LAYERS.findall(log_text)
|
||||
device_buffers = _MODEL_BUFFER.findall(log_text)
|
||||
all_model_buffers = _ANY_MODEL_BUFFER.findall(log_text)
|
||||
if self._placement == "cpu":
|
||||
# CPU-only llama.cpp builds can omit the offload-count line.
|
||||
# Require positive CPU tensor-buffer evidence in that layout;
|
||||
# a missing GPU line alone must never imply CPU placement.
|
||||
cpu_only_layout = (
|
||||
not offloaded
|
||||
and "warning: no usable GPU found" in log_text
|
||||
and bool(all_model_buffers)
|
||||
and all(name.startswith("CPU") for name in all_model_buffers)
|
||||
)
|
||||
if (
|
||||
not offloaded or offloaded[-1][0] != "0" or device_buffers
|
||||
(not cpu_only_layout and (not offloaded or offloaded[-1][0] != "0"))
|
||||
or device_buffers
|
||||
or (all_model_buffers and any(not name.startswith("CPU") for name in all_model_buffers))
|
||||
or (assignments and any(name != "CPU" for name in assignments))
|
||||
):
|
||||
raise RuntimeError("CPU stage assigned model layers to an accelerator")
|
||||
@@ -188,8 +201,10 @@ class LlamaCppTextStageClient(StageClientBase):
|
||||
"requested_device": self._placement,
|
||||
"expected_device_name": expected_device_name,
|
||||
"layer_assignments": len(assignments),
|
||||
"offloaded_layers": offloaded[-1],
|
||||
"offloaded_layers": offloaded[-1] if offloaded else None,
|
||||
"model_buffer_devices": device_buffers,
|
||||
"all_model_buffers": all_model_buffers,
|
||||
"cpu_only_layout": cpu_only_layout if self._placement == "cpu" else False,
|
||||
"worker_pid": self._proc.pid,
|
||||
"loaded_rss_bytes": self._loaded_rss_bytes,
|
||||
"reserved_bytes": dict(reservation.demands),
|
||||
|
||||
Reference in new issue
Block a user