2 October 2026 · Inference benchmark · context rungs
LLMProbe performance suite · Sushi 1.1.0
Session completed Local run
Completed the benchmark-only run across all three context rungs.
What was checked
- The 4K, 16K, and 64K rungs completed.
- The recorded report identifies a single sample per scenario.
Limits of this evidence
No raw LLMProbe JSON, coverage entry labels, endpoint, prompt text, or server logs are included.
Configuration
The model, serving engine and agent used for this execution.
| Field | Value | Detail |
|---|---|---|
| Model · primary | Qwen 3.8 Flash Next · primary | EXL3 K=2.625 · 2.625 bpw |
| Runtime | Sushi 1.1.0 · standalone mode | inference engine and exact build |
| Harness | LLMProbe 0.6.12 | the agent or client that drove the run |
| Hardware | Apple M5 Pro · 64 GB unified memory | hardware recorded for this execution |
| Workload | LLMProbe performance suite · revision 1 | Benchmark-only protocol with 4K, 16K, and 64K context rungs. One sample per scenario; decode samples were fixed at 192 tokens. |
| Context window | 131,072 tokens | maximum context configured for the serving runtime |
| Repetitions | 1 | executions recorded for this workload; one means an indicative result, not an estimate |
Benchmark suite measurements
LLMProbe scenarios. Every scenario was measured once, so these are indicative rather than stable estimates.
| Metric | Value | Definition |
|---|---|---|
| General decode | 47.4 tok/s | single 192-token decode sample · n=1 |
| Reference TTFT | 461 ms | single sample, 2040 input tokens · n=1 |
| Reference prefill | 853.2 tok/s | single sample, 2040 input tokens · n=1 |
| MTP · predictable text | 73.3 tok/s | single sample · n=1 |
| MTP · novel text | 37.4 tok/s | single sample · n=1 |
| Predictable-to-novel MTP ratio | 1.96 ratio | predictable decode / novel decode · n=1 |
| Prefix cache · cold TTFT | 1,902 ms | single sample · n=1 |
| Prefix cache · warm TTFT | 359 ms | single sample · n=1 |
| Four-stream aggregate throughput | 54.1 tok/s | four-stream aggregate · n=1 |
| Four-stream batching efficiency | 0.37 ratio | four-stream run · n=1 |
| Decode drift under measured load | -1.5 % | change from first to last decode measurement · n=1 |
| Decode · 4K rung | 38.8 tok/s | single 4248-token rung · n=1 |
| TTFT · 4K rung | 5,889 ms | single 4248-token rung · n=1 |
| Prefill · 4K rung | 721 tok/s | single 4248-token rung · n=1 |
| Decode · 16K rung | 42.1 tok/s | single 16452-token rung · n=1 |
| TTFT · 16K rung | 23,002 ms | single 16452-token rung · n=1 |
| Prefill · 16K rung | 715 tok/s | single 16452-token rung · n=1 |
| Decode · 64K rung | 41.3 tok/s | single 65580-token rung · n=1 |
| TTFT · 64K rung | 94,883 ms | single 65580-token rung · n=1 |
| Prefill · 64K rung | 691 tok/s | single 65580-token rung · n=1 |
Agent session counters
Agent-wide usage totals. These include reasoning, tool calls and waiting, and are not model decode speed.
| Metric | Value | Definition |
|---|---|---|
| Suite wall time | 3.6 min | wall time · n=1 |
Run ID: 2026-10-02-qwen38-flash-next-sushi-standalone-llmprobe
One execution, with checks transcribed from the retained record. A completed session is not a quality score. Measurement method · All runs