[Edge] Agentic evidence for the 4-bit choice: W4A16, not W4A8
Driving a phone through the Android agent harness turns section 2c's fidelity
numbers into a task outcome. Same simulated device, one run per cell:
build decode "Turn on Wi-Fi." "Set an alarm"
bf16 41.0 tok/s done in 4 steps wrong app, stuck
W4A16 g128 80.0 tok/s done in 4 steps right app, no completion
W4A8 g128 86.6 tok/s turns it on, then off (x2) reaches Alarm, loops
Every build navigates home -> Settings -> Network -> Wi-Fi correctly. Only
some notice the switch now reads checked=true and call task_complete. W4A8 --
the fastest build, and the one the speed-first path recommends -- ends with
the task undone, on both runs.
So "W4A16 reproduces 2/12 greedy prompts and W4A8 1/12" was not a small
difference on a proxy metric: it is finishing versus reversing. The selector
now says so where someone choosing a build will read it, and points at
priority="fidelity" for agentic work.
Two tasks, one run per cell: enough to show the failure reproduces, not to
rank success rates.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
c4e61133fb
commit
bc7f8237e3
1 file changed
+12
@@ -42,6 +42,18 @@ Fidelity is the reason both 4-bit rows stay: against each build's own bf16
|
||||
output over 12 greedy prompts, llama.cpp's Q4_K_M reproduced 4/12 exactly,
|
||||
w4a16 2/12 and w4a8 1/12. W4A8 pins the zero point at 8 (so the grid must be
|
||||
symmetric) *and* quantizes activations to int8; W4A16 does neither.
|
||||
|
||||
That difference has a task-level consequence, which is why ``priority`` is not
|
||||
a matter of taste. Driving a phone through the Android agent harness
|
||||
(``analysis/experiments/spark_edge/android_agent``), every build navigates a
|
||||
multi-screen task correctly -- and only some of them notice they have arrived.
|
||||
W4A8 turned Wi-Fi on and then tapped the switch off again, on both runs;
|
||||
W4A16 called ``task_complete`` citing the switch reading ``checked=true``, and
|
||||
on a second task it picked the right app where bf16 picked the wrong one.
|
||||
**Navigation survives quantization; recognising completion does not.** For
|
||||
agentic work choose ``priority="fidelity"``; the 8% of decode it costs buys
|
||||
the difference between finishing a task and undoing it. (Two tasks, one run
|
||||
per cell -- enough to show the failure reproduces, not to rank success rates.)
|
||||
"""
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
Reference in new issue
Block a user