Local test machine: Apple M5 Pro · 64 GB unified memoryRun window 29 September — 18 October 202622 recorded runs
Workload · Agentic coding · isometric 3D arcade · Revision 1
A one-shot coding challenge to build a playable Digger-inspired game with deterministic grid movement and excavation, collectibles, enemies, and multi-level progression in an isometric Three.js scene.
2 recorded attempts
02 October 2026
Sushi 1.1.0 · Pi coding agent
A deterministic grid simulation with an isometric presentation. The TypeScript check and production build passed, along with 28 of 28 self-tests and validation across all five levels.
03 October 2026
OpenAI Codex · xhigh reasoning · one-shot
Session notes report 28/28 simulation self-tests passing and a small interactive smoke check; manual acceptance remained partial.
From the run records
Each attempt used its own engine, harness version and settings, and some sessions include waiting or outages. Hover a value for its exact definition, and read the run sheets before drawing conclusions. Cloud wall time includes provider queueing; local decode speed is per request on this machine.
Each attempt starts from a project scaffold and must deliver a complete, playable game loop rather than a partial scene. The checks recorded for each attempt are listed on its run sheet.
How to read this comparison
Compare the generated builds and the checks recorded for each attempt. There is no normalised quality score, and the cloud task’s cumulative token totals and end-to-end rate include repeated context, reasoning, tool calls and waiting — they are not comparable with local per-request decode or prefill rates. The method page sets out which figures can and cannot be set against each other.