20 November 2026 · task-set v0.21.3
Claude Code + Haiku 5.5 (OpenRouter)
59/84 passed (70.2%) among tasks with valid verdicts. Coverage is 84/90. 6 tasks remain unmeasured after repeated HTTP 504 errors (393, 395, 399, 405, 406, 408). This partial result has no leaderboard rank.
Claude Code 2.1.278, model anthropic/claude-haiku-5.5 with OpenRouter as the display label, default reasoning and temperature, up to 16 concurrent tasks, Linux bubblewrap. The first clean attempt on each task determines its verdict. Checkpoint revision was unavailable.
Recorded retained usage was 4,601 tool calls and 111,698,173 tokens. Token counters cover 83 of the 84 tasks with valid verdicts. Unresolved and superseded attempts are excluded from usage. Task 400 keeps its first clean passing attempt. An extra retry queued by an audit error was stopped and excluded.
Recorded usage across the first attempts of all 90 tasks was 104,345,716 tokens, including attempts without an accepted verdict. Task retries are excluded. This total includes 133,203 tokens recovered from the interrupted transcript for task 527 and covers the responses with recorded usage.