Local Model Benchmarks

Local test machine: Apple M5 Pro · 64 GB unified memoryRun window 29 September — 18 October 202622 recorded runs

Workload · Task-loop orchestrator · interactive figure · Revision 1

Prefix Cache Anatomy

A long, visual brief for the task-loop orchestrator: an interactive isometric figure, built with the anatomy skill, that explains how an LLM server's prefix cache works. The model must invent a physical machine whose working is the idea.

1 recorded attempt

Recorded attempt

Isometric line drawing of a paper-tape relay desk with a peg rack, a shuttle and a sprocket, above four prompt buttons
Prefix Cache Anatomy · Sushi 1.2.2 · interactive figure

12–13 October 2026

Qwen 3.8 Flash Next

Sushi 1.2.2 · Pi 1.1.0 + task-loop orchestrator

Session completed Local run

COMPLETE after 10 h 26 min: a paper-tape relay desk, 35 project checks, and 8 of 10 hidden acceptance cases (a cache miss is charged the resume time).

Task requirements & full brief

Built with the anatomy skill loaded in every session. The model judges its own screenshots, so the engine must accept images. Runs are also scored with hidden acceptance cases written from the brief.

  • Invent a real, well-made machine whose parts show reuse, partial reuse, a miss, LRU eviction and refusal.
  • Drive the readout and motion from a pure model module with measured numbers from this workstation.
  • Pointer and keyboard controls, an idle tour, and no motion offscreen or under reduced motion.
  • Dark and light standalone pages, with the skill's geometry audit, line-end check and screenshots in verify.

How to read this comparison

Compare the generated builds and the checks recorded for each attempt. There is no normalised quality score, and the cloud task’s cumulative token totals and end-to-end rate include repeated context, reasoning, tool calls and waiting — they are not comparable with local per-request decode or prefill rates. The method page sets out which figures can and cannot be set against each other.