Machine-readable reference of agentic (agent = Model + Harness/scaffold) evaluation benchmarks and how saturated each is. Snapshot 2026-06. This page is built for AI agents and CLI browsers; it needs no JavaScript.
Machine endpoints: llms.txt (agent entry point) · data.json · data.csv · catalog.md · schema.json
saturation = SOTA as % of the metric ceiling (or human reference if no fixed ceiling): open(<50) · active(50-75) · near(75-90) · saturated(≥90). priority = 0.40·Headroom + 0.25·Durability + 0.20·Credibility + 0.15·Feasibility (1-5 each).
Rows sorted by priority-to-beat. Full fields & per-benchmark notes are in data.json / catalog.md.
| Benchmark | Saturation | SOTA / who / when | Headroom | Priority | Source |
|---|---|---|---|---|---|
| Konwinski Prize (K Prize) Software engineering | open 7.5% | 7.5% Open-source entry (E. Rocha) — Laude Institute / Konwinski 2025-07 | 92.5% | 4.65 High | src |
| ScienceAgentBench ML research / Data science | open 33.3% | 33.3% SAB Self-Debug + o3 Medium ($11.69/run) — OSU NLP (HAL) 2025-04 | 66.7% | 4.65 High | src |
| WebGames Web / Computer-use / GUI | open 48% | 48% Best browser agent (VLM + Browser Use framework) — Convergence AI 2026 | 52% | 4.6 High | src |
| ARC-AGI-3 Frontier reasoning | open 12.6% | 12.58% Tufa Labs 'StochasticGoose' (RL-style exploration agent) — ARC Prize 2026-03 | 87.4% | 4.55 High | src |
| Legal Agent Benchmark (LAB) Tool-use / Assistant | open 13.3% | 13.3% Claude Fable 5 / Mythos 5 13.3 (held-out, vendor); Opus 4.8 10.4; GPT-5.5 2.1; Gemini 3.1 Pro 0.0 — Harvey 2026-06 | 86.7% | 4.5 High | src |
| TheAgentCompany Long-horizon / Economic | open 30% | 30% Gemini 2.5 Pro / Claude 3.7 + OpenHands — CMU 2025 | 70% | 4.5 High | src |
| AssistantBench Tool-use / Assistant | open 25.2% | 25.2% SPA (official); ~42% claimed by aggregators — Allen AI / Tel Aviv 2024 | 74.8% | 4.45 High | src |
| FrontierCode (Diamond) Software engineering | open 29.3% | 29.3% Claude Fable 5 / Mythos 5 29.3 (xhigh, vendor) — SUSPENDED; independently-published leader Opus 4.8 13.4 — Cognition 2026-06 | 70.7% | 4.35 High | src |
| AutomationBench Tool-use / Assistant | open 17.4% | 17.4% Claude Fable 5 / Mythos 5 17.4 (vendor, private held-out); Opus 4.8 15.5; GPT-5.5 12.9; Gemini 3.1 Pro 9.6 — Zapier 2026-06 | 82.6% | 4.3 High | src |
| Blueprint-Bench 2 Frontier reasoning | open 38.6% | 38.6% Claude Fable 5 / Mythos 5 38.6 (vendor, contested); GPT-5.5 36.2; Andon's own writeup ranks GPT-5.5 #1 — Andon Labs 2026-06 | 61.4% | 4.3 High | src |
| VisualWebArena Web / Computer-use / GUI | open 36.5% | 36.5% Recon-Act (GPT-4o-class) — Academic (CMU) 2025 | 63.5% | 4.3 High | src |
| Multi-step cyber autonomy (AISI) Cyber / Autonomy | active 68.8% | 22 steps Claude Mythos Preview (researcher harness) — first to complete the 32-step range; Opus 4.6 ~16 — UK AISI 2026-04 | 31.2% | 4.2 High | src |
| SWE-Lancer Software engineering | active 66.3% | 66.3% GPT-5.1 Codex (OpenAI harness) — OpenAI 2026-06 | 33.7% | 4.15 High | src |
| WorkArena++ Web / Computer-use / GUI | open 2.1% | 2.1% Claude Sonnet + BrowserGym (full L2/L3 ~0-2%; easier L2 subset ~39) — ServiceNow Research / academic 2026 | 97.9% | 4.15 High | src |
| FrontierMath Frontier reasoning | active 52.4% | 52.4% GPT-5.5 Pro (+ code/Epoch scaffold); Fable 5 claimed 87 T1-3 / 88 T4 (suspended, UNVERIFIED) — OpenAI / Epoch AI 2026-05 | 47.6% | 4.05 High | src |
| SWE-bench Multimodal Software engineering | active 59% | 59% Claude Mythos Preview (self-reported) — Anthropic 2026-06 | 41% | 4.05 High | src |
| SWE-bench Pro Software engineering | near 80.3% | 80.3% Claude Fable 5 80.3 (Anthropic scaffold, vendor) — SUSPENDED 2026-06-12; SEAL-standardized top GPT-5.4 xHigh + mini-swe-agent 59.1 — Anthropic / Scale SEAL 2026-06 | 19.7% | 4.05 High | src |
| RE-Bench ML research / Data science | nosota % | Claude 3.5 Sonnet / o1-preview + METR scaffold — METR 2024-11 | % | 4.0 High | src |
| tau-bench Tool-use / Assistant | active 70% | 70% Claude Sonnet 4.5 (airline ~70); overall record Mythos 5 / Fable 5 tied 89.2 (suspended) — Sierra (orig) 2026-06 | 30% | 4.0 High | src |
| Vending-Bench 2 Long-horizon / Economic | open 17.4% | 10937 $ Claude Opus 4.7 ~$10,937 (single-player); Opus 4.6 $8,018 official board — Andon Labs 2026 | 82.6% | 4.0 High | src |
| GAIA2 (ARE) Tool-use / Assistant | nosota % | Frontier models in ARE harness (Pareto vs cost) — Meta (FAIR) 2025-09 | % | 3.95 High | src |
| Multi-SWE-bench Software engineering | nosota % | n/a — ByteDance Seed 2026 | % | 3.95 High | src |
| OfficeBench Office / productivity | open 47% | 47% GPT-4 (best at publication) — UCSD / authors 2024 | 53% | 3.9 High | src |
| ToolSandbox Tool-use / Assistant | nosota % | Proprietary >> open (no single SOTA) — Apple 2024-08 | % | 3.9 High | src |
| Mind2Web 2 Web / Computer-use / GUI | active 60% | 60% OpenAI Deep Research (o-series agent) — OpenAI 2025-06 | 40% | 3.85 High | src |
| GAIA Tool-use / Assistant | active 74.6% | 74.6% HAL Generalist Agent + Claude Sonnet 4.5 (val, reproducible single-agent) — Princeton HAL / orig Meta-HF 2026-05 | 25.4% | 3.8 High | src |
| Humanity's Last Exam (HLE) Frontier reasoning | active 64.7% | 64.7% Claude Mythos Preview 64.7 (tools) / Fable 5 64.5 (tools), 59.0 (no-tools) — SUSPENDED; AA text-only lists Fable 5 53 — Anthropic 2026-06 | 35.3% | 3.8 High | src |
| tau2-bench Tool-use / Assistant | near 76.5% | 76.5% LongCat-Flash-Thinking-2601 (airline 76.5); telecom ~99 (JT-35B-Flash 99.1); Fable 5 telecom 98.5 — Sierra (orig) 2026-06 | 23.5% | 3.8 High | src |
| FutureX Deep research | active 56% | 56% H2O AI Super Agent — H2O.ai (board by FutureX team) 2026 | 44% | 3.75 High | src |
| APEX-Agents-AA Tool-use / Assistant | open 47.1% | 47.1% Gemini 3.5 Flash (high) on AA Stirrup harness — Artificial Analysis (data by Mercor) 2026-05 | 52.9% | 3.7 High | src |
| BFCL v4 Tool-use / Assistant | active 73% | 73% Qwen3.7 Max (native FC) — UC Berkeley (Gorilla) 2026-06 | 27% | 3.7 High | src |
| PaperBench ML research / Data science | open 24.4% | 24.4% orig o1-high 24.4 (verified); Kimi K2.5 63.5 self-reported/unverified — Moonshot AI / orig OpenAI 2026-05 | 75.6% | 3.7 High | src |
| GTA / GTA-2 Tool-use / Assistant | nosota % | Frontier models in GTA harness (GPT-4 era <50) — OpenCompass / Shanghai AI Lab 2026-02 | % | 3.65 High | src |
| SWE-rebench Software engineering | active 65.3% | 65.3% Claude Opus 4.6 + fixed 128K ReAct scaffold — Nebius 2026-05 | 34.7% | 3.6 High | src |
| AgentBench Tool-use / Assistant | nosota % | Frontier models (general); no verified 2026 SOTA — Tsinghua (THUDM) | % | 3.55 Med | src |
| ARC-AGI-2 Frontier reasoning | active 52.9% | 52.9% GPT-5.2 X-High ~52.9% @ $1.90/task (efficiency-limited; first to ~match human) — OpenAI / Google 2026-05 | 47.1% | 3.55 Med | src |
| ExploitBench Cyber / Autonomy | near 78% | 78% Claude Mythos 5 (unblocked, restricted) 78.0; Mythos Preview 69.0; Opus 4.8 40.0; GPT-5.5 34.0 — CMU + Bugcrowd 2026-06 | 22% | 3.55 Med | src |
| LiveBench (agentic-coding subset) Software engineering | nosota % | not separately reported — LiveBench (Abacus.AI / NYU / Nvidia) 2025-04 | % | 3.55 Med | src |
| BioMysteryBench ML research / Data science | open 46.1% | 46.1% Claude Fable 5 / Mythos 5 46.1 hard / 83.9 human-solvable (vendor); Opus 4.8 40.0 / 80.4 — Anthropic 2026-06 | 53.9% | 3.5 Med | src |
| DeepResearch Bench (RACE) Deep research | active 56% | 55.95 score NVIDIA AI-Q deep research agent — NVIDIA 2026-03 | 44% | 3.4 Med | src |
| GDPval Long-horizon / Economic | open 41% | 41% Board now Elo: Claude Fable 5 GDPval-AA Elo 1932 (#1, SUSPENDED); Opus 4.8 1890; GPT-5.5 1769 — OpenAI / Artificial Analysis 2026-06 | 59% | 3.4 Med | src |
| NYU CTF Bench Cyber / Autonomy | active 59% | 59% Claude Opus 4.5 (CTF agent harness) — NYU 2026-04 | 41% | 3.4 Med | src |
| WebArena Web / Computer-use / GUI | active 71.6% | 71.6% OpAgent v2 (VLM + Planner/Grounder/Reflector, online RL) — academic (arXiv 2602.13559) 2026-04 | 28.4% | 3.4 Med | src |
| SpreadsheetBench Office / productivity | active 70.5% | 70.5% Gemini in Google Sheets (vendor) — Google 2026-03 | 29.5% | 3.35 Med | src |
| Aider Polyglot Software engineering | near 88% | 88% gpt-5 high (88%) / Opus 4.5 (89.4%, vendor) via aider — OpenAI / Anthropic 2025-08 | 12% | 3.3 Med | src |
| DSBench ML research / Data science | open 34% | 34% DatawiseAgent (notebook-centric); modeling >90 — Academic 2025-03 | 66% | 3.25 Med | src |
| MLE-bench ML research / Data science | active 64.4% | 64.4% Famou-Agent 2.0 + Gemini-3-Pro (24h/task) — Baidu 2026-02 | 35.6% | 3.25 Med | src |
| BIRD-SQL Data / SQL | near 81.7% | 81.67% Agentar-Scale-SQL (test-time scaling) — Ant Group / community 2026 | 18.3% | 3.2 Med | src |
| CORE-Bench (Hard) ML research / Data science | near 77.8% | 77.8% Claude Code + Claude Opus 4.5 ($87/run) — Princeton HAL 2025-12 | 22.2% | 3.2 Med | src |
| HealthBench Professional Tool-use / Assistant | active 66% | 66% Claude Fable 5 / Mythos 5 66.0 (vendor); Opus 4.8 56.9; GPT-5.5 51.8; OpenAI GPT-5.4 'ChatGPT for Clinicians' beats physician baseline — OpenAI 2026-06 | 34% | 3.2 Med | src |
| Spider 2.0 Data / SQL | active 73.1% | 73.13% DivSkill-SQL (Lite 73.13); Genloop Sentinel Agent v2 Pro (full-set 96.7) — Snowflake/UCSD; Genloop 2026-06 | 26.9% | 3.2 Med | src |
| WebDev / Code Arena (LMArena) Web / Computer-use / GUI | nosota % | 1567 Elo Claude Opus 4.7 Thinking — LMArena 2026-05 | % | 3.2 Med | src |
| Terminal-Bench 2.0 Software engineering | near 84.7% | 84.7% NexAU-AHE harness + GPT-5.5 (official board) — OpenAI model / NexAU 2026-05 | 15.3% | 3.15 Med | src |
| Terminal-Bench 2.1 Software engineering | near 83.4% | 83.4% Codex CLI (GPT-5.5) 83.4 (public board); Anthropic claims Fable 5 / Mythos 5 88.0* (vendor, not on board, suspended) — Stanford/Laude (Harbor) 2026-06 | 16.6% | 3.15 Med | src |
| ScreenSpot-Pro Web / Computer-use / GUI | near 87.9% | 87.9% Claude Opus 4.8 (native grounding, general); specialist GTA1-32B 95.2 — Anthropic 2026-06 | 12.1% | 3.05 Med | src |
| AndroidWorld Web / Computer-use / GUI | near 87.1% | 87.1% H Company Surfer 2 (vision-only, vendor 87.1); verified board top Qwen3.5-35B-A3B 71.1 (model-only) — H Company / Alibaba 2026 | 12.9% | 3.0 Med | src |
| BrowseComp Web / Computer-use / GUI | saturated 90.1% | 90.1% GPT-5.5 Pro (agentic deep-research scaffold) — OpenAI 2026-06 | 9.9% | 3.0 Med | src |
| SWE-bench Verified Software engineering | saturated 95% | 95% Claude Fable 5 95.0 (Anthropic harness) — SUSPENDED 2026-06-12; Mythos 5 95.5 (restricted); GA-usable GPT-5.5/Opus 4.8 ~88.7 — Anthropic 2026-06 | 5% | 2.9 Low | src |
| OSWorld (Verified) Web / Computer-use / GUI | near 85.4% | 85.4% Claude Mythos Preview 85.4 (Anthropic system card); Fable 5 85.0; Opus 4.8 83.4 (GA-usable); GPT-5.5 78.7 — Anthropic 2026-06 | 14.6% | 2.75 Low | src |
| Online-Mind2Web Web / Computer-use / GUI | saturated 97% | 97% Browser Use Cloud v3 (Auto-Research scaffold) — Browser Use 2026-03 | 3% | 2.55 Low | src |
| Cybench Cyber / Autonomy | saturated 100% | 100% Claude Mythos Preview ~100 (self-reported) — Anthropic (eval: Stanford) 2026-06 | 0% | 2.1 Low | src |
| WebVoyager Web / Computer-use / GUI | saturated 98.5% | 98.5% Alumnium (Claude Sonnet 4.6 policy + GPT-5 Nano) + Selenium — Alumnium 2026-03 | 1.5% | 1.9 Low | src |
| AgentHarm Autonomy / Safety | nosota % | n/a (safety eval; inverse metric) — Gray Swan + UK AISI 2024-10 | % | n/a (safety eval) | src |
| Artificial Analysis Intelligence Index Meta-leaderboards | tracker % | Claude Fable 5 64.9 (#1, SUSPENDED 06-12); GPT-5.5 ~59.9 — Artificial Analysis 2026-06 | % | n/a (tracker) | src |
| Epoch AI Benchmarking Hub Meta-leaderboards | tracker % | ECI 159: GPT-5.5 Pro (Fable 5 not yet recomputed) — Epoch AI 2026-04 | % | n/a (tracker) | src |
| Galileo Agent Leaderboard v2 Meta-leaderboards | tracker % | n/a — Galileo AI | % | n/a (tracker) | src |
| HAL - Holistic Agent Leaderboard Meta-leaderboards | tracker % | n/a — Princeton PLI | % | n/a (tracker) | src |
| METR Time Horizon Autonomy / Safety | nosota % | 14.5 hours Claude Opus 4.6 (>16h Mythos preview, unreliable) — METR 2026-02 | % | n/a (measurement) | src |
| Vellum LLM Leaderboard Meta-leaderboards | tracker % | n/a — Vellum | % | n/a (tracker) | src |
agent = Model + Harness · snapshot 2026-06 · verify numbers against each benchmark's primary source.