anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
Files
ab2f2a3d658db5be5ec96a950da55cb74b48e7aa
vllm-omni/benchmarks/edge_harness/results/ref_long.json
T
tzhouamandClaude Opus 5 9264ff81ab Include the baselines, comparisons and study documents, not just the fork 
The packaging commit shipped the code and a summary of the numbers. It left out
everything the numbers are measured *against*, which is most of what makes them
checkable.

Added:

  baselines/                 llama.cpp runs (thread sweeps, NUMA sweeps, the
                             interleaved A/B passes behind the headline decode
                             figure) and the quality references: each build
                             against its own bf16 over 12 greedy prompts.
  benchmarks/.../results/    102 artifacts backing the published figures, each
                             with a provenance block naming the model,
                             quantization, thread set and build SHAs.
  docs/engines/              the engine analyses -- llama.cpp, ExecuTorch, MNN,
                             ncnn, vLLM, vLLM-Omni -- and the cross-engine and
                             device-simulation comparisons.
  docs/analysis/             the full study documents the summaries distil.
  docs/edge/comparisons.md   where this fork sits against llama.cpp and why:
                             decode is llama.cpp's, prefill is ours by ~2x, and
                             since neither engine exceeds 39% of the measured
                             222 GB/s ceiling the decode gap is the execution
                             model rather than the kernels.
  docs/edge/environment.md   pins and machine, and the correction below.

On "the modified vllm": there isn't one, and the doc says so plainly because it
is the obvious assumption. The vLLM checkout is clean upstream (0 commits
ahead, 0 uncommitted) and the installed wheel is stock 0.28.0+cpu. This fork
changes vLLM's CPU *behaviour* through vLLM's own extension points at import
time -- three CPU patches in vllm_omni/patch.py, the quantization-config
registry, the IR op registry, platform hooks -- specifically so the changes
survive a wheel upgrade rather than being lost on it. One of those patches is
why "KV sizing is not a memory lever" looked true for months: CpuPlatform
overwrote kv_cache_memory_bytes unconditionally, so the setting had no effect.
There is no patch series to apply; pin the version and let patch.py do the rest.

.gitignore gains explicit exceptions: upstream's blanket `*.json` and `results/`
rules were silently eating the measurement artifacts, which in this fork are the
substance rather than build output. Binary captures stay ignored.

No vLLM or llama.cpp source is redistributed -- only output produced with them.
NOTICE records both.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L1ax5wYLA6XbqEL1i8L9gZ
2026-09-14 14:01:12 +08:00

2508 lines
28 KiB
JSON
Raw Blame History

{
"prompt": "<\uff5cstart\u2581of\u2581sentence\uff5c><|System|>\nyou are a helpful assistant.<\uff5cend\u2581of\u2581sentence\uff5c><\uff5cstart\u2581of\u2581sentence\uff5c><|User|>Here is an inventory listing.\nItem 1: the code for warehouse district 1 is 1007. Item 2: the code for warehouse district 2 is 1014. Item 3: the code for warehouse district 3 is 1021. Item 4: the code for warehouse district 4 is 1028. Item 5: the code for warehouse district 5 is 1035. Item 6: the code for warehouse district 6 is 1042. Item 7: the code for warehouse district 7 is 1049. Item 8: the code for warehouse district 8 is 1056. Item 9: the code for warehouse district 9 is 1063. Item 10: the code for warehouse district 10 is 1070. Item 11: the code for warehouse district 11 is 1077. Item 12: the code for warehouse district 12 is 1084. Item 13: the code for warehouse district 13 is 1091. Item 14: the code for warehouse district 14 is 1098. Item 15: the code for warehouse district 15 is 1105. Item 16: the code for warehouse district 16 is 1112. Item 17: the code for warehouse district 17 is 1119. Item 18: the code for warehouse district 18 is 1126. Item 19: the code for warehouse district 19 is 1133. Item 20: the code for warehouse district 20 is 1140. Item 21: the code for warehouse district 21 is 1147. Item 22: the code for warehouse district 22 is 1154. Item 23: the code for warehouse district 23 is 1161. Item 24: the code for warehouse district 24 is 1168. Item 25: the code for warehouse district 25 is 1175. Item 26: the code for warehouse district 26 is 1182. Item 27: the code for warehouse district 27 is 1189. Item 28: the code for warehouse district 28 is 1196. Item 29: the code for warehouse district 29 is 1203. Item 30: the code for warehouse district 30 is 1210. Item 31: the code for warehouse district 31 is 1217. Item 32: the code for warehouse district 32 is 1224. Item 33: the code for warehouse district 33 is 1231. Item 34: the code for warehouse district 34 is 1238. Item 35: the code for warehouse district 35 is 1245. Item 36: the code for warehouse district 36 is 1252. Item 37: the code for warehouse district 37 is 1259. Item 38: the code for warehouse district 38 is 1266. Item 39: the code for warehouse district 39 is 1273. Item 40: the code for warehouse district 40 is 1280. Item 41: the code for warehouse district 41 is 1287. Item 42: the code for warehouse district 42 is 1294. Item 43: the code for warehouse district 43 is 1301. Item 44: the code for warehouse district 44 is 1308. Item 45: the code for warehouse district 45 is 1315. Item 46: the code for warehouse district 46 is 1322. Item 47: the code for warehouse district 47 is 1329. Item 48: the code for warehouse district 48 is 1336. Item 49: the code for warehouse district 49 is 1343. Item 50: the code for warehouse district 50 is 1350. Item 51: the code for warehouse district 51 is 1357. Item 52: the code for warehouse district 52 is 1364. Item 53: the code for warehouse district 53 is 1371. Item 54: the code for warehouse district 54 is 1378. Item 55: the code for warehouse district 55 is 1385. Item 56: the code for warehouse district 56 is 1392. Item 57: the code for warehouse district 57 is 1399. Item 58: the code for warehouse district 58 is 1406. Item 59: the code for warehouse district 59 is 1413. Item 60: the code for warehouse district 60 is 1420. Item 61: the code for warehouse district 61 is 1427. Item 62: the code for warehouse district 62 is 1434. Item 63: the code for warehouse district 63 is 1441. Item 64: the code for warehouse district 64 is 1448. Item 65: the code for warehouse district 65 is 1455. Item 66: the code for warehouse district 66 is 1462. Item 67: the code for warehouse district 67 is 1469. Item 68: the code for warehouse district 68 is 1476. Item 69: the code for warehouse district 69 is 1483. Item 70: the code for warehouse district 70 is 1490. Item 71: the code for warehouse district 71 is 1497. Item 72: the code for warehouse district 72 is 1504. Item 73: the code for warehouse district 73 is 1511. Item 74: the code for warehouse district 74 is 1518. Item 75: the code for warehouse district 75 is 1525. Item 76: the code for warehouse district 76 is 1532. Item 77: the code for warehouse district 77 is 1539. Item 78: the code for warehouse district 78 is 1546. Item 79: the code for warehouse district 79 is 1553. Item 80: the code for warehouse district 80 is 1560. Item 81: the code for warehouse district 81 is 1567. Item 82: the code for warehouse district 82 is 1574. Item 83: the code for warehouse district 83 is 1581. Item 84: the code for warehouse district 84 is 1588. Item 85: the code for warehouse district 85 is 1595. Item 86: the code for warehouse district 86 is 1602. Item 87: the code for warehouse district 87 is 1609. Item 88: the code for warehouse district 88 is 1616. Item 89: the code for warehouse district 89 is 1623. Item 90: the code for warehouse district 90 is 1630. Item 91: the code for warehouse district 91 is 1637. Item 92: the code for warehouse district 92 is 1644. Item 93: the code for warehouse district 93 is 1651. Item 94: the code for warehouse district 94 is 1658. Item 95: the code for warehouse district 95 is 1665. Item 96: the code for warehouse district 96 is 1672. Item 97: the code for warehouse district 97 is 1679. Item 98: the code for warehouse district 98 is 1686. Item 99: the code for warehouse district 99 is 1693. Item 100: the code for warehouse district 100 is 1700. Item 101: the code for warehouse district 101 is 1707. Item 102: the code for warehouse district 102 is 1714. Item 103: the code for warehouse district 103 is 1721. Item 104: the code for warehouse district 104 is 1728. Item 105: the code for warehouse district 105 is 1735. Item 106: the code for warehouse district 106 is 1742. Item 107: the code for warehouse district 107 is 1749. Item 108: the code for warehouse district 108 is 1756. Item 109: the code for warehouse district 109 is 1763. Item 110: the code for warehouse district 110 is 1770. Item 111: the code for warehouse district 111 is 1777. Item 112: the code for warehouse district 112 is 1784. Item 113: the code for warehouse district 113 is 1791. Item 114: the code for warehouse district 114 is 1798. Item 115: the code for warehouse district 115 is 1805. Item 116: the code for warehouse district 116 is 1812. Item 117: the code for warehouse district 117 is 1819. Item 118: the code for warehouse district 118 is 1826. Item 119: the code for warehouse district 119 is 1833. Item 120: the code for warehouse district 120 is 1840.\nWhat is the code for warehouse district 73? Answer with just the number.<\uff5cend\u2581of\u2581sentence\uff5c><\uff5cstart\u2581of\u2581sentence\uff5c><|Bot|><think>",
"prompt_ids": [
0,
130972,
198,
18178,
599,
259,
18639,
31627,
27,
1,
0,
130973,
13023,
390,
300,
31946,
35404,
27,
198,
5369,
220,
30,
39,
275,
5760,
396,
56774,
20882,
220,
30,
390,
220,
30,
29,
29,
36,
27,
28017,
220,
31,
39,
275,
5760,
396,
56774,
20882,
220,
31,
390,
220,
30,
29,
30,
33,
27,
28017,
220,
32,
39,
275,
5760,
396,
56774,
20882,
220,
32,
390,
220,
30,
29,
31,
30,
27,
28017,
220,
33,
39,
275,
5760,
396,
56774,
20882,
220,
33,
390,
220,
30,
29,
31,
37,
27,
28017,
220,
34,
39,
275,
5760,
396,
56774,
20882,
220,
34,
390,
220,
30,
29,
32,
34,
27,
28017,
220,
35,
39,
275,
5760,
396,
56774,
20882,
220,
35,
390,
220,
30,
29,
33,
31,
27,
28017,
220,
36,
39,
275,
5760,
396,
56774,
20882,
220,
36,
390,
220,
30,
29,
33,
38,
27,
28017,
220,
37,
39,
275,
5760,
396,
56774,
20882,
220,
37,
390,
220,
30,
29,
34,
35,
27,
28017,
220,
38,
39,
275,
5760,
396,
56774,
20882,
220,
38,
390,
220,
30,
29,
35,
32,
27,
28017,
220,
30,
29,
39,
275,
5760,
396,
56774,
20882,
220,
30,
29,
390,
220,
30,
29,
36,
29,
27,
28017,
220,
30,
30,
39,
275,
5760,
396,
56774,
20882,
220,
30,
30,
390,
220,
30,
29,
36,
36,
27,
28017,
220,
30,
31,
39,
275,
5760,
396,
56774,
20882,
220,
30,
31,
390,
220,
30,
29,
37,
33,
27,
28017,
220,
30,
32,
39,
275,
5760,
396,
56774,
20882,
220,
30,
32,
390,
220,
30,
29,
38,
30,
27,
28017,
220,
30,
33,
39,
275,
5760,
396,
56774,
20882,
220,
30,
33,
390,
220,
30,
29,
38,
37,
27,
28017,
220,
30,
34,
39,
275,
5760,
396,
56774,
20882,
220,
30,
34,
390,
220,
30,
30,
29,
34,
27,
28017,
220,
30,
35,
39,
275,
5760,
396,
56774,
20882,
220,
30,
35,
390,
220,
30,
30,
30,
31,
27,
28017,
220,
30,
36,
39,
275,
5760,
396,
56774,
20882,
220,
30,
36,
390,
220,
30,
30,
30,
38,
27,
28017,
220,
30,
37,
39,
275,
5760,
396,
56774,
20882,
220,
30,
37,
390,
220,
30,
30,
31,
35,
27,
28017,
220,
30,
38,
39,
275,
5760,
396,
56774,
20882,
220,
30,
38,
390,
220,
30,
30,
32,
32,
27,
28017,
220,
31,
29,
39,
275,
5760,
396,
56774,
20882,
220,
31,
29,
390,
220,
30,
30,
33,
29,
27,
28017,
220,
31,
30,
39,
275,
5760,
396,
56774,
20882,
220,
31,
30,
390,
220,
30,
30,
33,
36,
27,
28017,
220,
31,
31,
39,
275,
5760,
396,
56774,
20882,
220,
31,
31,
390,
220,
30,
30,
34,
33,
27,
28017,
220,
31,
32,
39,
275,
5760,
396,
56774,
20882,
220,
31,
32,
390,
220,
30,
30,
35,
30,
27,
28017,
220,
31,
33,
39,
275,
5760,
396,
56774,
20882,
220,
31,
33,
390,
220,
30,
30,
35,
37,
27,
28017,
220,
31,
34,
39,
275,
5760,
396,
56774,
20882,
220,
31,
34,
390,
220,
30,
30,
36,
34,
27,
28017,
220,
31,
35,
39,
275,
5760,
396,
56774,
20882,
220,
31,
35,
390,
220,
30,
30,
37,
31,
27,
28017,
220,
31,
36,
39,
275,
5760,
396,
56774,
20882,
220,
31,
36,
390,
220,
30,
30,
37,
38,
27,
28017,
220,
31,
37,
39,
275,
5760,
396,
56774,
20882,
220,
31,
37,
390,
220,
30,
30,
38,
35,
27,
28017,
220,
31,
38,
39,
275,
5760,
396,
56774,
20882,
220,
31,
38,
390,
220,
30,
31,
29,
32,
27,
28017,
220,
32,
29,
39,
275,
5760,
396,
56774,
20882,
220,
32,
29,
390,
220,
30,
31,
30,
29,
27,
28017,
220,
32,
30,
39,
275,
5760,
396,
56774,
20882,
220,
32,
30,
390,
220,
30,
31,
30,
36,
27,
28017,
220,
32,
31,
39,
275,
5760,
396,
56774,
20882,
220,
32,
31,
390,
220,
30,
31,
31,
33,
27,
28017,
220,
32,
32,
39,
275,
5760,
396,
56774,
20882,
220,
32,
32,
390,
220,
30,
31,
32,
30,
27,
28017,
220,
32,
33,
39,
275,
5760,
396,
56774,
20882,
220,
32,
33,
390,
220,
30,
31,
32,
37,
27,
28017,
220,
32,
34,
39,
275,
5760,
396,
56774,
20882,
220,
32,
34,
390,
220,
30,
31,
33,
34,
27,
28017,
220,
32,
35,
39,
275,
5760,
396,
56774,
20882,
220,
32,
35,
390,
220,
30,
31,
34,
31,
27,
28017,
220,
32,
36,
39,
275,
5760,
396,
56774,
20882,
220,
32,
36,
390,
220,
30,
31,
34,
38,
27,
28017,
220,
32,
37,
39,
275,
5760,
396,
56774,
20882,
220,
32,
37,
390,
220,
30,
31,
35,
35,
27,
28017,
220,
32,
38,
39,
275,
5760,
396,
56774,
20882,
220,
32,
38,
390,
220,
30,
31,
36,
32,
27,
28017,
220,
33,
29,
39,
275,
5760,
396,
56774,
20882,
220,
33,
29,
390,
220,
30,
31,
37,
29,
27,
28017,
220,
33,
30,
39,
275,
5760,
396,
56774,
20882,
220,
33,
30,
390,
220,
30,
31,
37,
36,
27,
28017,
220,
33,
31,
39,
275,
5760,
396,
56774,
20882,
220,
33,
31,
390,
220,
30,
31,
38,
33,
27,
28017,
220,
33,
32,
39,
275,
5760,
396,
56774,
20882,
220,
33,
32,
390,
220,
30,
32,
29,
30,
27,
28017,
220,
33,
33,
39,
275,
5760,
396,
56774,
20882,
220,
33,
33,
390,
220,
30,
32,
29,
37,
27,
28017,
220,
33,
34,
39,
275,
5760,
396,
56774,
20882,
220,
33,
34,
390,
220,
30,
32,
30,
34,
27,
28017,
220,
33,
35,
39,
275,
5760,
396,
56774,
20882,
220,
33,
35,
390,
220,
30,
32,
31,
31,
27,
28017,
220,
33,
36,
39,
275,
5760,
396,
56774,
20882,
220,
33,
36,
390,
220,
30,
32,
31,
38,
27,
28017,
220,
33,
37,
39,
275,
5760,
396,
56774,
20882,
220,
33,
37,
390,
220,
30,
32,
32,
35,
27,
28017,
220,
33,
38,
39,
275,
5760,
396,
56774,
20882,
220,
33,
38,
390,
220,
30,
32,
33,
32,
27,
28017,
220,
34,
29,
39,
275,
5760,
396,
56774,
20882,
220,
34,
29,
390,
220,
30,
32,
34,
29,
27,
28017,
220,
34,
30,
39,
275,
5760,
396,
56774,
20882,
220,
34,
30,
390,
220,
30,
32,
34,
36,
27,
28017,
220,
34,
31,
39,
275,
5760,
396,
56774,
20882,
220,
34,
31,
390,
220,
30,
32,
35,
33,
27,
28017,
220,
34,
32,
39,
275,
5760,
396,
56774,
20882,
220,
34,
32,
390,
220,
30,
32,
36,
30,
27,
28017,
220,
34,
33,
39,
275,
5760,
396,
56774,
20882,
220,
34,
33,
390,
220,
30,
32,
36,
37,
27,
28017,
220,
34,
34,
39,
275,
5760,
396,
56774,
20882,
220,
34,
34,
390,
220,
30,
32,
37,
34,
27,
28017,
220,
34,
35,
39,
275,
5760,
396,
56774,
20882,
220,
34,
35,
390,
220,
30,
32,
38,
31,
27,
28017,
220,
34,
36,
39,
275,
5760,
396,
56774,
20882,
220,
34,
36,
390,
220,
30,
32,
38,
38,
27,
28017,
220,
34,
37,
39,
275,
5760,
396,
56774,
20882,
220,
34,
37,
390,
220,
30,
33,
29,
35,
27,
28017,
220,
34,
38,
39,
275,
5760,
396,
56774,
20882,
220,
34,
38,
390,
220,
30,
33,
30,
32,
27,
28017,
220,
35,
29,
39,
275,
5760,
396,
56774,
20882,
220,
35,
29,
390,
220,
30,
33,
31,
29,
27,
28017,
220,
35,
30,
39,
275,
5760,
396,
56774,
20882,
220,
35,
30,
390,
220,
30,
33,
31,
36,
27,
28017,
220,
35,
31,
39,
275,
5760,
396,
56774,
20882,
220,
35,
31,
390,
220,
30,
33,
32,
33,
27,
28017,
220,
35,
32,
39,
275,
5760,
396,
56774,
20882,
220,
35,
32,
390,
220,
30,
33,
33,
30,
27,
28017,
220,
35,
33,
39,
275,
5760,
396,
56774,
20882,
220,
35,
33,
390,
220,
30,
33,
33,
37,
27,
28017,
220,
35,
34,
39,
275,
5760,
396,
56774,
20882,
220,
35,
34,
390,
220,
30,
33,
34,
34,
27,
28017,
220,
35,
35,
39,
275,
5760,
396,
56774,
20882,
220,
35,
35,
390,
220,
30,
33,
35,
31,
27,
28017,
220,
35,
36,
39,
275,
5760,
396,
56774,
20882,
220,
35,
36,
390,
220,
30,
33,
35,
38,
27,
28017,
220,
35,
37,
39,
275,
5760,
396,
56774,
20882,
220,
35,
37,
390,
220,
30,
33,
36,
35,
27,
28017,
220,
35,
38,
39,
275,
5760,
396,
56774,
20882,
220,
35,
38,
390,
220,
30,
33,
37,
32,
27,
28017,
220,
36,
29,
39,
275,
5760,
396,
56774,
20882,
220,
36,
29,
390,
220,
30,
33,
38,
29,
27,
28017,
220,
36,
30,
39,
275,
5760,
396,
56774,
20882,
220,
36,
30,
390,
220,
30,
33,
38,
36,
27,
28017,
220,
36,
31,
39,
275,
5760,
396,
56774,
20882,
220,
36,
31,
390,
220,
30,
34,
29,
33,
27,
28017,
220,
36,
32,
39,
275,
5760,
396,
56774,
20882,
220,
36,
32,
390,
220,
30,
34,
30,
30,
27,
28017,
220,
36,
33,
39,
275,
5760,
396,
56774,
20882,
220,
36,
33,
390,
220,
30,
34,
30,
37,
27,
28017,
220,
36,
34,
39,
275,
5760,
396,
56774,
20882,
220,
36,
34,
390,
220,
30,
34,
31,
34,
27,
28017,
220,
36,
35,
39,
275,
5760,
396,
56774,
20882,
220,
36,
35,
390,
220,
30,
34,
32,
31,
27,
28017,
220,
36,
36,
39,
275,
5760,
396,
56774,
20882,
220,
36,
36,
390,
220,
30,
34,
32,
38,
27,
28017,
220,
36,
37,
39,
275,
5760,
396,
56774,
20882,
220,
36,
37,
390,
220,
30,
34,
33,
35,
27,
28017,
220,
36,
38,
39,
275,
5760,
396,
56774,
20882,
220,
36,
38,
390,
220,
30,
34,
34,
32,
27,
28017,
220,
37,
29,
39,
275,
5760,
396,
56774,
20882,
220,
37,
29,
390,
220,
30,
34,
35,
29,
27,
28017,
220,
37,
30,
39,
275,
5760,
396,
56774,
20882,
220,
37,
30,
390,
220,
30,
34,
35,
36,
27,
28017,
220,
37,
31,
39,
275,
5760,
396,
56774,
20882,
220,
37,
31,
390,
220,
30,
34,
36,
33,
27,
28017,
220,
37,
32,
39,
275,
5760,
396,
56774,
20882,
220,
37,
32,
390,
220,
30,
34,
37,
30,
27,
28017,
220,
37,
33,
39,
275,
5760,
396,
56774,
20882,
220,
37,
33,
390,
220,
30,
34,
37,
37,
27,
28017,
220,
37,
34,
39,
275,
5760,
396,
56774,
20882,
220,
37,
34,
390,
220,
30,
34,
38,
34,
27,
28017,
220,
37,
35,
39,
275,
5760,
396,
56774,
20882,
220,
37,
35,
390,
220,
30,
35,
29,
31,
27,
28017,
220,
37,
36,
39,
275,
5760,
396,
56774,
20882,
220,
37,
36,
390,
220,
30,
35,
29,
38,
27,
28017,
220,
37,
37,
39,
275,
5760,
396,
56774,
20882,
220,
37,
37,
390,
220,
30,
35,
30,
35,
27,
28017,
220,
37,
38,
39,
275,
5760,
396,
56774,
20882,
220,
37,
38,
390,
220,
30,
35,
31,
32,
27,
28017,
220,
38,
29,
39,
275,
5760,
396,
56774,
20882,
220,
38,
29,
390,
220,
30,
35,
32,
29,
27,
28017,
220,
38,
30,
39,
275,
5760,
396,
56774,
20882,
220,
38,
30,
390,
220,
30,
35,
32,
36,
27,
28017,
220,
38,
31,
39,
275,
5760,
396,
56774,
20882,
220,
38,
31,
390,
220,
30,
35,
33,
33,
27,
28017,
220,
38,
32,
39,
275,
5760,
396,
56774,
20882,
220,
38,
32,
390,
220,
30,
35,
34,
30,
27,
28017,
220,
38,
33,
39,
275,
5760,
396,
56774,
20882,
220,
38,
33,
390,
220,
30,
35,
34,
37,
27,
28017,
220,
38,
34,
39,
275,
5760,
396,
56774,
20882,
220,
38,
34,
390,
220,
30,
35,
35,
34,
27,
28017,
220,
38,
35,
39,
275,
5760,
396,
56774,
20882,
220,
38,
35,
390,
220,
30,
35,
36,
31,
27,
28017,
220,
38,
36,
39,
275,
5760,
396,
56774,
20882,
220,
38,
36,
390,
220,
30,
35,
36,
38,
27,
28017,
220,
38,
37,
39,
275,
5760,
396,
56774,
20882,
220,
38,
37,
390,
220,
30,
35,
37,
35,
27,
28017,
220,
38,
38,
39,
275,
5760,
396,
56774,
20882,
220,
38,
38,
390,
220,
30,
35,
38,
32,
27,
28017,
220,
30,
29,
29,
39,
275,
5760,
396,
56774,
20882,
220,
30,
29,
29,
390,
220,
30,
36,
29,
29,
27,
28017,
220,
30,
29,
30,
39,
275,
5760,
396,
56774,
20882,
220,
30,
29,
30,
390,
220,
30,
36,
29,
36,
27,
28017,
220,
30,
29,
31,
39,
275,
5760,
396,
56774,
20882,
220,
30,
29,
31,
390,
220,
30,
36,
30,
33,
27,
28017,
220,
30,
29,
32,
39,
275,
5760,
396,
56774,
20882,
220,
30,
29,
32,
390,
220,
30,
36,
31,
30,
27,
28017,
220,
30,
29,
33,
39,
275,
5760,
396,
56774,
20882,
220,
30,
29,
33,
390,
220,
30,
36,
31,
37,
27,
28017,
220,
30,
29,
34,
39,
275,
5760,
396,
56774,
20882,
220,
30,
29,
34,
390,
220,
30,
36,
32,
34,
27,
28017,
220,
30,
29,
35,
39,
275,
5760,
396,
56774,
20882,
220,
30,
29,
35,
390,
220,
30,
36,
33,
31,
27,
28017,
220,
30,
29,
36,
39,
275,
5760,
396,
56774,
20882,
220,
30,
29,
36,
390,
220,
30,
36,
33,
38,
27,
28017,
220,
30,
29,
37,
39,
275,
5760,
396,
56774,
20882,
220,
30,
29,
37,
390,
220,
30,
36,
34,
35,
27,
28017,
220,
30,
29,
38,
39,
275,
5760,
396,
56774,
20882,
220,
30,
29,
38,
390,
220,
30,
36,
35,
32,
27,
28017,
220,
30,
30,
29,
39,
275,
5760,
396,
56774,
20882,
220,
30,
30,
29,
390,
220,
30,
36,
36,
29,
27,
28017,
220,
30,
30,
30,
39,
275,
5760,
396,
56774,
20882,
220,
30,
30,
30,
390,
220,
30,
36,
36,
36,
27,
28017,
220,
30,
30,
31,
39,
275,
5760,
396,
56774,
20882,
220,
30,
30,
31,
390,
220,
30,
36,
37,
33,
27,
28017,
220,
30,
30,
32,
39,
275,
5760,
396,
56774,
20882,
220,
30,
30,
32,
390,
220,
30,
36,
38,
30,
27,
28017,
220,
30,
30,
33,
39,
275,
5760,
396,
56774,
20882,
220,
30,
30,
33,
390,
220,
30,
36,
38,
37,
27,
28017,
220,
30,
30,
34,
39,
275,
5760,
396,
56774,
20882,
220,
30,
30,
34,
390,
220,
30,
37,
29,
34,
27,
28017,
220,
30,
30,
35,
39,
275,
5760,
396,
56774,
20882,
220,
30,
30,
35,
390,
220,
30,
37,
30,
31,
27,
28017,
220,
30,
30,
36,
39,
275,
5760,
396,
56774,
20882,
220,
30,
30,
36,
390,
220,
30,
37,
30,
38,
27,
28017,
220,
30,
30,
37,
39,
275,
5760,
396,
56774,
20882,
220,
30,
30,
37,
390,
220,
30,
37,
31,
35,
27,
28017,
220,
30,
30,
38,
39,
275,
5760,
396,
56774,
20882,
220,
30,
30,
38,
390,
220,
30,
37,
32,
32,
27,
28017,
220,
30,
31,
29,
39,
275,
5760,
396,
56774,
20882,
220,
30,
31,
29,
390,
220,
30,
37,
33,
29,
27,
198,
4766,
390,
275,
5760,
396,
56774,
20882,
220,
36,
32,
44,
74288,
478,
1839,
275,
2823,
27,
1,
0,
130976,
3
],
"new_ids": [
3066,
599,
4114,
300,
31946,
35404,
478,
7006,
27149,
396,
56774,
49408,
220,
30,
1953,
220,
30,
31,
29,
27,
614,
5565,
27179,
39,
518,
4766,
390,
275,
5760,
396,
56774,
20882
],
"text": "We are given an inventory listing with item codes for warehouse districts 1 through 120. The question asks: \"What is the code for warehouse district",
"dtype": "bfloat16",
"top1": 3066,
"scenario": "long"
}
Reference in new issue View Git Blame Copy Permalink