Benchmark Registry

SWE Atlas

SWE Atlas results reported for 42 models across 8 versions. Updated 2026-10-07.

Report an issue

Versions

8 versions · Metric: Mean pass@1

VersionReleasedModelsResults
SWE Atlas Refactoring — Claude CodeConfiguration: Claude Code harnessMay 7, 202666
SWE Atlas Refactoring — CodexConfiguration: Codex harnessMay 7, 202644
SWE Atlas Refactoring — Gemini CLIConfiguration: Gemini CLI harnessMay 7, 202611
SWE Atlas Refactoring — mini-swe-agentConfiguration: mini-swe-agent harnessMay 7, 202666
SWE Atlas Codebase QnA — Mercor mini-swe-agent, 250 stepsConfiguration: Mercor mini-swe-agent, 250 steps, 1 hourMarch 4, 20263234
SWE Atlas Codebase QnA — Public; Claude CodeConfiguration: Claude Code harnessMarch 4, 202677
SWE Atlas Codebase QnA — Public; CodexConfiguration: Codex harnessMarch 4, 202655
SWE Atlas Codebase QnA — Public; mini-swe-agentConfiguration: mini-swe-agent harnessMarch 4, 20261313

Recently reported results: SWE Atlas Refactoring — Claude Code, Claude Code harness

Most recent configuration: SWE Atlas Refactoring — Claude Code

ModelScoreSource
Claude Opus 4.635.6%labs.scale.comArchived copyMean pass@1Reported Benchmark leaderboardScale AIIndependentRecord
Claude Opus 4.748.6%labs.scale.comArchived copyMean pass@1Reported Benchmark leaderboardScale AIIndependentRecord
Claude Fable 5 (xHigh)54.8%labs.scale.comArchived copyMean pass@1Reported Benchmark leaderboardScale AIIndependentRecord
Claude Sonnet 4.632.2%labs.scale.comArchived copyMean pass@1Reported Benchmark leaderboardScale AIIndependentRecord
Claude Fable 5.1 (xHigh)56.7%labs.scale.comArchived copyMean pass@1Reported Benchmark leaderboardScale AIIndependentRecord