Local Model Benchmarks

Local test machine: Apple M5 Pro · 64 GB unified memoryRun window 29 September — 18 October 202622 recorded runs

4 October 2026 · Agentic game · 3D baseball

Hardball 3D · TensorFold 0.6.4

Session completed Local run

Produced a complete playable 3D arcade baseball game. Pi’s final session summary reports gameplay and restart checks; those checks were not independently rerun while archiving.

Hardball 3D TensorFold gameplay from behind the batter, with the inning scoreboard and field visible
Hardball 3D · Qwen 3.8 27B TensorFold build · gameplay capture

What was checked

  • The archived production dist contains a relative-path HTML entry and one JavaScript bundle (538,782 bytes).
  • Pi’s final session summary reports all eight game states exercised, batting contact from both sides, scoring, inning progression with score carry, game-over and restart.
  • Pi’s final session summary reports zero console or page errors during multi-minute autoplay and a pixel census confirming rendered scene content.
  • The source project, Pi transcript and server logs are excluded; the shared task prompt is linked from the Hardball 3D challenge page.

Limits of this evidence

The full 4 October Pi session has 136 usage-bearing assistant responses. Only 134 request summaries are unambiguously attributable to this session: its first two counter triples are identical to those in a separate 3 October Pi session, and TensorFold’s text log has no timestamps or correlatable request IDs. Those two responses are excluded from decode, prefill, TTFT and MTP calculations; session-wide Pi counters still cover all 136 responses. Four runtime completions remain unused after matching the 134-response subset. One matched request ended with finish=length at 16,384 output tokens, but the configured output limit was not retained. No peak process or system memory measurement was recorded. The 3 October session is not merged into this run.

Configuration

The model, serving engine and agent used for this execution.

Recorded configuration and setup details
FieldValueDetail
Model · verifierQwen 3.8 27B · verifier · reasoning mediumMLX affine 4-bit, group size 64 · 4 bpw
Checkpoint · verifierQwen3.8-27B-MLX-Serve-4bitverifier model artifact
Model · drafterQwen 3.8 27B DFlash2 · drafter4-bit; draft block size 8 · 4 bpw
RuntimeTensorFold 0.6.4 · MLX 0.32.2inference engine and exact build
HarnessPi coding agentthe agent or client that drove the run
HardwareApple MacBook Pro · 64 GB unified memoryhardware recorded for this execution
WorkloadHardball 3D · revision 1One-shot agentic 3D baseball coding challenge; one session, not a repeated fixed-task suite.
Context window131,072 tokensmaximum context configured for the serving runtime
Repetitions1executions recorded for this workload; one means an indicative result, not an estimate

Inference measurements

Engine-side rates for this run. Each figure states its own statistic; figures from different runs are not interchangeable.

Inference measurements: recorded value and measurement definition
MetricValueDefinition
Decode · median27.4 tok/sMedian per-request TensorFold rate; excludes the first output token. · n=134
Decode · 10th percentile22.1 tok/sP10; linear interpolation. · n=134
Decode · 90th percentile42.37 tok/sP90; linear interpolation. · n=134
Decode · output-weighted29.0015 tok/sTotal matched output tokens / sum(output tokens / per-request logged rate); rates rounded to 0.1 tok/s. · n=134
Prefill · median153.56 tok/sMedian fresh input tokens / recorded prefill seconds; cached tokens excluded from numerator. · n=134 · 133 of 134 requests reused a nonzero prompt prefix.
Prefill · mean160.55 tok/sMean fresh input tokens / recorded prefill seconds; cached tokens excluded from numerator. · n=134 · 133 of 134 requests reused a nonzero prompt prefix.
TTFT · median8.13 sMedian engine request-start to first output token. · n=134 · 133 of 134 requests reused a nonzero prompt prefix.
TTFT · 90th percentile21.45 sP90 engine request-start to first output token; linear interpolation. · n=134
MTP rounds40,785 roundsSum of speculative decode rounds. · n=134
MTP acceptance23.3941 %Total accepted draft tokens / total drafted tokens. · n=655057 · 153,245 accepted / 655,057 drafted tokens across 134 unambiguous requests.

Agent session counters

Agent-wide usage totals. These include reasoning, tool calls and waiting, and are not model decode speed.

Agent session counters: recorded value and measurement definition
MetricValueDefinition
Session elapsed194.5 minPi session start to final assistant response, rounded to the nearest second; includes coding, tools and checks. · n=1
Model responses136 responsesAssistant responses with usage metadata in the full 4 October Pi session. · n=1
Matched model requests134 requestsUnambiguous one-to-one Pi/TensorFold matches; two overlapping responses excluded from inference metrics. · n=1
Tool calls142 callsTotal Pi tool calls in the full session. · n=1
Tool errors12 errorsTool results marked as errors in the full Pi session. · n=1
Context compactions2 compactionsTotal Pi session compactions. · n=1

Context and token counters

Token totals as reported by the harness or engine. Cached and cumulative figures are not additive with fresh input.

Context and token counters: recorded value and measurement definition
MetricValueDefinition
Maximum prompt context114,614 tokensLargest total prompt, fresh plus cached, in the full Pi session. · n=136
Fresh input330,711 tokensSum of Pi uncached input usage counters across all 136 assistant responses. · n=1
Cached input9,813,340 tokensSum of Pi cache-read counters across all 136 assistant responses. · n=1
Cumulative input10,144,051 tokensFresh input plus cache-read tokens from Pi usage counters. · n=1
Output tokens193,993 tokensPi output usage total across all 136 responses; includes reasoning tokens. · n=1
of which reasoning output104,279 tokensPi-reported reasoning tokens, included in output tokens. · n=1
Request records & provenance

Contains 134 numeric per-request rows for Pi assistant responses 3–136. Each is matched in sequence by total prompt, cached input and output token counts. The first two responses are ambiguous because their counter triples also occur in the separate 3 October session. Four remaining TensorFold completions were not assigned to this unique subset. Request IDs, timestamps, output text, prompt content and hashes are excluded.

Run ID: 2026-10-04-qwen38-27b-tensorfold-pi-hardball-3d

One execution, with checks transcribed from the retained record. A completed session is not a quality score. Measurement method · All runs