Wave 19 · 19 October 2026
New complete v0.19.0 measurements, split into Light and Medium. Leaderboard →
| Model | Light | Medium |
|---|
| DeepSeek V4.1 Flash (OpenRouter) (high) | 385/391 (98.5%) | 59/80 (73.8%) |
| Claude Haiku 5.5 (OpenRouter) (default) | 377/391 (96.4%) | 50/80 (62.5%) |
| GLM-5.3 (OpenRouter) (high) | 378/391 (96.7%) | 36/80 (45.0%) |
| GigaChat 3.5 (PROM), build 32.9.16.9 (default) | 292/391 (74.7%) | 0/80 (0.0%) |
Protocol
All four configurations: stock deepagents 0.6.12 (no GigaChat profile), one run, 471 unique tasks, 4 concurrent tasks, max_tokens=16384, bwrap isolation, reasoning history forwarded, native subagents available. Default timeout 900 seconds; long-context floor 7200 seconds, reflection floor 1800 seconds; wave 19 hard limit 120 seconds. Recursion limit 80 with task-specific floors. Source commit 3796b9084093e5f9383077d7bde8f1ce1f0d57b5. All runs completed with measurement_valid=true and zero infrastructure failures.
GLM and DeepSeek: high reasoning, thinking enabled through the API adapter. Haiku and GigaChat: default reasoning, no explicit thinking setting. Scores compare these full configurations; reasoning modes differ. GigaChat uses the production API (PROM), build 32.9.16.9.
Light: tasks 1–391. Medium: tasks 392–471, including long context (20), tool reflection (20), wave 19 feasibility (40). Historical 40-task Medium scores remain separate. Compared with Haiku, DeepSeek: Light +2.0 percentage points, Medium +11.3 points; GLM: Light +0.3 points, Medium −17.5 points. These are single runs, not a repeat-based significance claim.
95% Wilson intervals are available in aggregate data and score tooltips. They describe task outcomes, not repeat variability. Full traces remain private; raw source SHA256 fingerprints accompany each aggregate. Detailed failure-by-failure audit is pending. Measurement aggregates →