Benchmark Registry

SWE-bench Pro

SWE-bench Pro results reported for 43 models across 3 versions. Updated 2026-10-08.

Report an issue

Versions

3 versions, plus 5 variants and configurations · Metric: Public tasks resolved

VersionReleasedModelsResults
SWE-bench Pro PublicLatest
5 variants of Public
VariantConfigurationReleasedModelsResults
SWE-bench Pro Public — GitHub Copilot production harnessGitHub Copilot production harnessJune 2, 202611
SWE-bench Pro Public — Qwen refined tasksQwen refined tasksMay 20, 202622
SWE-bench Pro Public — Qwen3.6 refined tasksQwen3.6 refined tasksApril 2, 202611
SWE-bench Pro Public — Scale AI SWE-AgentSWE-Agent harness, 250 turns, uncapped costSeptember 19, 20251111
SWE-bench Pro Public — Scale AI mini-swe-agentmini-swe-agent harnessSeptember 19, 202555
September 19, 20252727
SWE-bench Pro Private — Scale AI SWE-AgentConfiguration: SWE-Agent harness, 250 turns, uncapped costSeptember 19, 202566
SWE-bench Pro Private — Scale AI mini-swe-agentConfiguration: mini-swe-agent harnessSeptember 19, 202555

Recently reported results: SWE-bench Pro Public

Latest version: SWE-bench Pro Public

ModelScoreSource
Claude Haiku 5.5 (max)64.8%anthropic.comArchived copyPublic tasks resolvedReported System cardAnthropicSelf-reportedRecord
Claude Sonnet 5.5 (max)81.3%anthropic.comPublic tasks resolvedReported System cardAnthropicSelf-reportedRecord
Claude Opus 5.5 (max)89.9%www-cdn.anthropic.comPublic tasks resolvedReported System cardAnthropicSelf-reportedRecord
MAI-Thinking-152.8%microsoft.aiPublic tasks resolvedReported Technical reportMicrosoft AISelf-reportedMicrosoftRecord
Muse Glimmer 30B (high)51.2%dev.meta.aiPublic tasks resolvedReported Model pageMetaSelf-reportedRecord