Results

11 results

SWE-bench Pro Public — Scale AI SWE-Agent results
ProviderModelScoreSourceRegistry No.
OpenAIGPT-5.229.9%labs.scale.com for GPT-5.2 on SWE-bench Pro Public — Scale AI SWE-Agent (opens in a new tab)Archived copy (opens in a new tab)Public tasks resolvedReported Benchmark leaderboardScale AIIndependentRecord for GPT-5.2 on SWE-bench Pro Public — Scale AI SWE-Agent10012
OpenAIGPT-5 (High)41.8%labs.scale.com for GPT-5 (High) on SWE-bench Pro Public — Scale AI SWE-Agent (opens in a new tab)Archived copy (opens in a new tab)Public tasks resolvedReported Benchmark leaderboardScale AIIndependentRecord for GPT-5 on SWE-bench Pro Public — Scale AI SWE-Agent10018
DeepSeek AIDeepSeek-V3.215.6%labs.scale.com for DeepSeek-V3.2 on SWE-bench Pro Public — Scale AI SWE-Agent (opens in a new tab)Archived copy (opens in a new tab)Public tasks resolvedReported Benchmark leaderboardScale AIIndependentRecord for DeepSeek-V3.2 on SWE-bench Pro Public — Scale AI SWE-Agent110003
Z.aiGLM-4.69.7%labs.scale.com for GLM-4.6 on SWE-bench Pro Public — Scale AI SWE-Agent (opens in a new tab)Archived copy (opens in a new tab)Public tasks resolvedReported Benchmark leaderboardScale AIIndependentRecord for GLM-4.6 on SWE-bench Pro Public — Scale AI SWE-Agent150009
OpenAIgpt-oss-120b16.2%labs.scale.com for gpt-oss-120b on SWE-bench Pro Public — Scale AI SWE-Agent (opens in a new tab)Archived copy (opens in a new tab)Public tasks resolvedReported Benchmark leaderboardScale AIIndependentRecord for gpt-oss-120b on SWE-bench Pro Public — Scale AI SWE-Agent15001
AnthropicClaude Sonnet 442.7%labs.scale.com for Claude Sonnet 4 on SWE-bench Pro Public — Scale AI SWE-Agent (opens in a new tab)Archived copy (opens in a new tab)Public tasks resolvedReported Benchmark leaderboardScale AIIndependentRecord for Claude Sonnet 4 on SWE-bench Pro Public — Scale AI SWE-Agent20003
AnthropicClaude Sonnet 4.543.6%labs.scale.com for Claude Sonnet 4.5 on SWE-bench Pro Public — Scale AI SWE-Agent (opens in a new tab)Archived copy (opens in a new tab)Public tasks resolvedReported Benchmark leaderboardScale AIIndependentRecord for Claude Sonnet 4.5 on SWE-bench Pro Public — Scale AI SWE-Agent20005
AnthropicClaude Haiku 4.539.5%labs.scale.com for Claude Haiku 4.5 on SWE-bench Pro Public — Scale AI SWE-Agent (opens in a new tab)Archived copy (opens in a new tab)Public tasks resolvedReported Benchmark leaderboardScale AIIndependentRecord for Claude Haiku 4.5 on SWE-bench Pro Public — Scale AI SWE-Agent20006
AnthropicClaude Opus 4.545.9%labs.scale.com for Claude Opus 4.5 on SWE-bench Pro Public — Scale AI SWE-Agent (opens in a new tab)Archived copy (opens in a new tab)Public tasks resolvedReported Benchmark leaderboardScale AIIndependentRecord for Claude Opus 4.5 on SWE-bench Pro Public — Scale AI SWE-Agent20007
Google (DeepMind)Gemini 3 Pro43.3%labs.scale.com for Gemini 3 Pro on SWE-bench Pro Public — Scale AI SWE-Agent (opens in a new tab)Archived copy (opens in a new tab)Public tasks resolvedReported Benchmark leaderboardScale AIIndependentRecord for Gemini 3 Pro on SWE-bench Pro Public — Scale AI SWE-Agent30008
Google (DeepMind)Gemini 3 Flash34.6%labs.scale.com for Gemini 3 Flash on SWE-bench Pro Public — Scale AI SWE-Agent (opens in a new tab)Archived copy (opens in a new tab)Public tasks resolvedReported Benchmark leaderboardScale AIIndependentRecord for Gemini 3 Flash on SWE-bench Pro Public — Scale AI SWE-Agent30009