26 October 2026 · task-set v0.21.3
Claude Haiku 5.5 · Medium 90
Two fresh runs cover exactly the canonical 90 Medium tasks. Each task has one retained scored attempt, including failures. Infrastructure recovery attempts are excluded from public scores and counters; verifier failures were not retried to select better answers. Both final artifacts report valid measurements, zero unresolved infrastructure failures and 90/90 task coverage. The completion audit passed. These are separate measured configurations; they are not combined with earlier Haiku results.
| Harness | Model | Passed | Score | Steps | Tokens |
|---|---|---|---|---|---|
| mini-SWE-agent 2.4.6 | Claude Haiku 5.5 (default) | 63/90 | 70.0% | 2,777 · partial 88/90 | 493,730,606 · partial 88/90 |
| DeepAgents 0.6.12 | Claude Haiku 5.5 (default) | 60/90 | 66.7% | 4,078 · partial 88/90 | 191,772,875 · partial 89/90 |
Protocol
Requested API model: anthropic/claude-haiku-5.5. Reasoning and sampling use provider defaults without overrides; the output limit is 16,384 tokens per response. Four concurrent tasks per harness; Linux/bubblewrap isolation and a host-owned trusted service. DeepAgents forwards reasoning history and enables its native subagents; mini-SWE-agent uses its native DefaultAgent. The base budget is 160 graph supersteps for DeepAgents or 160 model calls for mini-SWE-agent, raised by numeric task-specific floors. These units are different. Ordinary execution has a 3,600-second guard, long-context tasks retain their 7,200-second floor, and selected coupled workflows retain their 900-second limit and 320 service requests. Setup and verification are outside execution limits.
Accounting and limits
Published per-task outcomes and counters come from final retained attempts only. Superseded attempts and separate smoke checks are excluded. Counters are native recorded subtotals, not complete API spend: mini-SWE-agent has steps and tokens for 88/90 tasks; DeepAgents has steps for 88/90 and tokens for 89/90. Missing counters are null in the downloadable data. Request-level completeness has not been independently audited; the leaderboard marks these totals partial. Source fingerprints identify the private final artifacts and protocol without publishing raw prompts, responses, credentials or internal endpoints.
One run per harness; no independent repeats or repeat-variability interval. The observed difference is three tasks, not evidence of causal harness superiority. Historical measurements keep their original scopes and profiles.
Public outcomes, native counters and source fingerprints · Medium membership and comparison policy