Benchmark Registry

SWE-bench

SWE-bench results reported for 98 models across 2 versions. Updated 2026-10-08.

Report an issue

Versions

2 versions, plus 14 variants and configurations · Metric: Resolved Tasks

VersionReleasedModelsResults
SWE-bench Verified
9 variants of Verified
VariantConfigurationReleasedModelsResults
SWE-bench Verified — bash-only harnessbash-only harnessJuly 15, 202622
SWE-bench Verified — GitHub Copilot production harnessGitHub Copilot production harnessJune 2, 202611
SWE-bench Verified — Muse Spark corrected testsMuse Spark corrected testsApril 8, 202611
SWE-bench Verified — Droid harnessDroid harnessFebruary 12, 202611
SWE-bench Verified — OpenCode harnessOpenCode harnessFebruary 12, 202611
SWE-bench Verified — mini-swe-agentmini-swe-agent harnessJuly 12, 20253036
SWE-bench Verified — Epoch AI harnessEpoch AI harnessAugust 13, 20242628
SWE-bench Verified — Mercor mini-swe-agent, 1000 stepsMercor mini-swe-agent, 1000 steps, 3 hoursAugust 13, 20243435
SWE-bench Verified — Vals AI mini-swe-agentmini-swe-agent harnessAugust 13, 20245959
August 13, 20244551
SWE-bench MultilingualLatest
5 variants of Multilingual
VariantConfigurationReleasedModelsResults
SWE-bench Multilingual — Cursor standard harnessCursor standard harnessJune 25, 202611
SWE-bench Multilingual — Cursor strict harnessCursor strict harnessJune 25, 202611
SWE-bench Multilingual — GitHub Copilot production harnessGitHub Copilot production harnessJune 2, 202611
SWE-bench Multilingual — mini-swe-agentmini-swe-agent harnessFebruary 24, 20261212
SWE-bench Multilingual — Mercor mini-swe-agent, 1000 stepsMercor mini-swe-agent, 1000 steps, 3 hoursMay 6, 20253434
May 6, 20253135

Recently reported results: SWE-bench Multilingual

Latest version: SWE-bench Multilingual

ModelScoreSource
Claude Haiku 5.5 (max)83.7%anthropic.comArchived copyResolved TasksReported System cardAnthropicSelf-reportedRecord
Claude Sonnet 5.5 (max)90.3%anthropic.comResolved TasksReported System cardAnthropicSelf-reportedRecord
Claude Opus 5.5 (max)93.9%www-cdn.anthropic.comResolved TasksReported System cardAnthropicSelf-reportedRecord
Claude Fable 5.1 (max)89.1%www-cdn.anthropic.comResolved TasksReported System cardAnthropicSelf-reportedRecord
Claude Opus 5 (max)89.5%www-cdn.anthropic.comResolved TasksReported System cardAnthropicSelf-reportedRecord