Results

5 results

SWE-bench Pro Public — Scale AI mini-swe-agent results
ProviderModelScoreSourceRegistry No.
OpenAIGPT-5.4 (xHigh)59.1%labs.scale.com for GPT-5.4 (xHigh) on SWE-bench Pro Public — Scale AI mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Public tasks resolvedReported Benchmark leaderboardScale AIIndependentRecord for GPT-5.4 on SWE-bench Pro Public — Scale AI mini-swe-agent10007
AnthropicClaude Opus 4.6 (thinking)51.9%labs.scale.com for Claude Opus 4.6 (thinking) on SWE-bench Pro Public — Scale AI mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Public tasks resolvedReported Benchmark leaderboardScale AIIndependentRecord for Claude Opus 4.6 on SWE-bench Pro Public — Scale AI mini-swe-agent20008
Google (DeepMind)Gemini 3.1 Pro (thinking)46.1%labs.scale.com for Gemini 3.1 Pro (thinking) on SWE-bench Pro Public — Scale AI mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Public tasks resolvedReported Benchmark leaderboardScale AIIndependentRecord for Gemini 3.1 Pro on SWE-bench Pro Public — Scale AI mini-swe-agent30005
Meta AI (originally Facebook AI Research)Muse Spark55.0%labs.scale.com for Muse Spark on SWE-bench Pro Public — Scale AI mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Public tasks resolvedReported Benchmark leaderboardScale AIIndependentRecord for Muse Spark on SWE-bench Pro Public — Scale AI mini-swe-agent80001
Meta AI (originally Facebook AI Research)Muse Spark 1.161.5%labs.scale.com for Muse Spark 1.1 on SWE-bench Pro Public — Scale AI mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Public tasks resolvedReported Benchmark leaderboardScale AIIndependentRecord for Muse Spark 1.1 on SWE-bench Pro Public — Scale AI mini-swe-agent80002