12 October 2026 · Agentic task loop · command-line app
Pocket Ledger
The orchestrator reached COMPLETE without intervention in 5 h 9 min. All 16 model-planned tasks are done, and the project's verify passes 50 checks. The final reviewer accepted coverage of all 20 requirements, and an independent end-to-end check of the brief's acceptance scenario passed apart from one wording issue.
What was checked
- Bootstrap was accepted on its first session. The orchestrator validated 20 requirements, 16 task cards, an acyclic dependency graph, full requirement coverage and a deterministic verify (two identical --all results).
- Each task was accepted only after the orchestrator itself ran verify --all and verify --task <ID> (at least one passing task check) and a fresh reviewer session returned ACCEPT. T002 and T004 were sent back once by their reviewers. T015 was retried because its first attempt added no task check.
- The T004 attempt-2 reviewer fell into a repetition loop in its reasoning (about 14k output tokens) and wrote no verdict. That task was accepted on verify alone.
- The T013 builder was aborted at 60 turns. It had finished the code by about turn 25, then alternated rewriting its handover note and counting its lines. Its work passed verify and review.
- Independent check after the run, outside the orchestrator: import of a two-month CSV with quoted commas, doubled quotes, DD/MM/YYYY dates and "-1,234.56" amounts; dry-run; duplicate skip on re-import; rules and recategorize; budget warning; report with over-budget mark and month-over-month change; undo; export and re-import with no loss; exit code 2 for bad dates, amounts, IDs and options; a corrupt data file reported with exit 1 and left unchanged. All behaved as specified, except that import --dry-run prints "added N" rather than a would-add wording.
Limits of this evidence
The engine and orchestrator settings were tuned only once before this run (enable --thinking, then a 2 GiB prompt cache with --prompt-cache-over-cap). After the run the orchestrator was changed: a windowed repeat-call watchdog, a reviewer retry when no verdict is written, contextBudget 0.65, and softer note-length rules. Later runs use a different configuration and are not repetitions of this one. Memory was not sampled during the run.
Configuration
The model, serving engine and agent used for this execution.
| Field | Value | Detail |
|---|---|---|
| Model · primary | Qwen 3.8 27B · primary · reasoning medium | MLX affine 4-bit, group size 64 · 4 bpw |
| Checkpoint · primary | TensorFold/Qwen3.8-27B-MLX-4bit | primary model artifact · revision 22d8d538154e0e5b6f8dcb1fcc74a60a0798104c |
| Model · drafter | Qwen 3.8 27B DFlash2 · drafter | Loaded with --drafter-bits 4 |
| Checkpoint · drafter | z-lab/Qwen3.8-27B-DFlash2 | drafter model artifact · revision 50307d4c4cde6860d4eee73e2547cd786fe8e8a4 |
| Runtime | TensorFold native (Zig) 1.0.7 · OpenAI-compatible server, one stream, greedy sampling, DFlash2 drafts, 2 GiB prompt cache above the 70% memory cap mode | inference engine and exact build |
| Harness | Pi coding agent, driven by the task-loop orchestrator 1.1.0 | the agent or client that drove the run |
| Hardware | Apple MacBook Pro · 64 GB unified memory | hardware recorded for this execution |
| Workload | Pocket Ledger · revision 1 | First real run of the generic task-loop orchestrator (scripts/task-loop.mjs). The orchestrator ran 42 fresh Pi sessions in sequence: bootstrap, builders, fresh-context reviewers, replans and a final reviewer. No human input after launch. |
| Context window | 131,072 tokens | maximum context configured for the serving runtime |
| Repetitions | 1 | executions recorded for this workload; one means an indicative result, not an estimate |
Inference measurements
Engine-side rates for this run. Each figure states its own statistic; figures from different runs are not interchangeable.
| Metric | Value | Definition |
|---|---|---|
| Draft acceptance | 33.97 % | Total accepted DFlash2 draft tokens / total drafted tokens over matched requests. · n=480 · 250545 accepted of 737625 drafted. |
Agent session counters
Agent-wide usage totals. These include reasoning, tool calls and waiting, and are not model decode speed.
| Metric | Value | Definition |
|---|---|---|
| Run elapsed | 309.2 min | Orchestrator start (after preflight) to the run's COMPLETE record; includes all sessions, verify runs, git bookkeeping and review. · n=1 |
| Agent sessions | 42 sessions | Fresh Pi sessions started by the orchestrator: 1 bootstrap, 19 builder, 18 reviewer, 3 replan, 1 final reviewer. · n=1 |
| Sessions stopped by a watchdog | 1 sessions | Sessions aborted by an orchestrator watchdog. · n=1 · T013 builder, turn budget (60 of 40). |
| Model responses | 481 responses | Assistant responses with usage records across all 42 sessions. · n=1 · Includes one empty aborted response. |
| Matched model requests | 480 requests | Successful Pi responses matched monotonically and uniquely to TensorFold done summaries by prompt, cached and output token counts. · n=1 · Four preceding engine summaries belong to preflight probes and are excluded. |
| Tool calls | 650 calls | Pi tool calls across all sessions. · n=1 |
| Context compactions | 0 compactions | Pi context compactions across all sessions. · n=1 |
Context and token counters
Token totals as reported by the harness or engine. Cached and cumulative figures are not additive with fresh input.
| Metric | Value | Definition |
|---|---|---|
| Maximum prompt context | 42,457 tokens | Largest engine-reported prompt among matched requests. · n=480 |
| Fresh input | 2,409,139 tokens | Sum of Pi uncached input counters across all sessions. · n=1 |
| Cached input | 7,018,644 tokens | Sum of Pi cache-read counters across all sessions; equals the engine's cached counts for matched requests. · n=1 |
| Cumulative input | 9,427,783 tokens | Fresh plus cached input tokens across all sessions. · n=1 |
| Prompt-cache token share | 74.45 % | Cached input tokens / cumulative input tokens over matched requests. · n=480 · The 2 GiB cache logged 1153 evictions and 326 refused states (states above about 25k tokens did not fit). |
| Output tokens | 299,741 tokens | Pi output usage across all sessions; includes reasoning tokens. · n=1 |
| of which reasoning output | 140,799 tokens | Pi-reported reasoning tokens included in output tokens. · n=1 |
Recorded check counters
Counts from the checks recorded with this run.
| Metric | Value | Definition |
|---|---|---|
| Tasks completed | 16 tasks | Task cards with status done at the end; none split, blocked or dropped. · n=1 |
| Build attempts | 19 attempts | Builder sessions across all tasks; 16 tasks needed 19 attempts. · n=1 |
| Reviews returning changes | 2 reviews | Reviewer sessions that returned CHANGES. · n=1 |
| Project checks passing | 50 checks | Checks passed by the project's own ./verify --all at the end. · n=1 |
| Hidden acceptance cases passed | 19 checks | Hidden acceptance cases passed, of 19: written from the brief, never shown to the model, and scored the same way for every run. Added after the run. · n=1 · All passed. The cases were checked by planting four faults in a copy of this build; each was caught. |
Request records & provenance
Per-request numbers are not published. All 480 successful Pi responses were matched in order to TensorFold summaries (prompt, cached and output tokens), with no ambiguous match. TensorFold 1.0.7 request summaries record no TTFT or decode time, and its cumulative /metrics histograms were not sampled per request during this run, so no latency or throughput figure is given.
Run ID: 2026-10-12-qwen38-27b-tensorfold-native-107-pi-task-loop-pocket-ledger
One execution, with checks transcribed from the retained record. A completed session is not a quality score. Measurement method · All runs