Results

13 results

SWE Atlas Codebase QnA — Public; mini-swe-agent results
ProviderModelScoreSourceRegistry No.
OpenAIGPT-5.4 (xHigh)36.3%labs.scale.com for GPT-5.4 (xHigh) on SWE Atlas Codebase QnA — Public; mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Mean pass@1Reported Benchmark leaderboardScale AIIndependentRecord for GPT-5.4 on SWE Atlas Codebase QnA — Public; mini-swe-agent10007
DeepSeek AIDeepSeek-V4-Pro27.2%labs.scale.com for DeepSeek-V4-Pro on SWE Atlas Codebase QnA — Public; mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Mean pass@1Reported Benchmark leaderboardScale AIIndependentRecord for DeepSeek-V4-Pro on SWE Atlas Codebase QnA — Public; mini-swe-agent110002
Moonshot AIKimi K2.513.1%labs.scale.com for Kimi K2.5 on SWE Atlas Codebase QnA — Public; mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Mean pass@1Reported Benchmark leaderboardScale AIIndependentRecord for Kimi K2.5 on SWE Atlas Codebase QnA — Public; mini-swe-agent120001
MiniMaxMiniMax M2.510.3%labs.scale.com for MiniMax M2.5 on SWE Atlas Codebase QnA — Public; mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Mean pass@1Reported Benchmark leaderboardScale AIIndependentRecord for MiniMax M2.5 on SWE Atlas Codebase QnA — Public; mini-swe-agent140001
Z.aiGLM-520.5%labs.scale.com for GLM-5 on SWE Atlas Codebase QnA — Public; mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Mean pass@1Reported Benchmark leaderboardScale AIIndependentRecord for GLM-5 on SWE Atlas Codebase QnA — Public; mini-swe-agent150002
Z.aiGLM-5.248.1%labs.scale.com for GLM-5.2 on SWE Atlas Codebase QnA — Public; mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Mean pass@1Reported Benchmark leaderboardScale AIIndependentRecord for GLM-5.2 on SWE Atlas Codebase QnA — Public; mini-swe-agent150004
AnthropicClaude Opus 4.630.0%labs.scale.com for Claude Opus 4.6 on SWE Atlas Codebase QnA — Public; mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Mean pass@1Reported Benchmark leaderboardScale AIIndependentRecord for Claude Opus 4.6 on SWE Atlas Codebase QnA — Public; mini-swe-agent20008
Google (DeepMind)Gemini 3.8 Flash47.0%labs.scale.com for Gemini 3.8 Flash on SWE Atlas Codebase QnA — Public; mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Mean pass@1Reported Benchmark leaderboardScale AIIndependentRecord for Gemini 3.8 Flash on SWE Atlas Codebase QnA — Public; mini-swe-agent30004
Google (DeepMind)Gemini 3.1 Pro13.5%labs.scale.com for Gemini 3.1 Pro on SWE Atlas Codebase QnA — Public; mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Mean pass@1Reported Benchmark leaderboardScale AIIndependentRecord for Gemini 3.1 Pro on SWE Atlas Codebase QnA — Public; mini-swe-agent30005
Google (DeepMind)Gemini 3 Flash8.2%labs.scale.com for Gemini 3 Flash on SWE Atlas Codebase QnA — Public; mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Mean pass@1Reported Benchmark leaderboardScale AIIndependentRecord for Gemini 3 Flash on SWE Atlas Codebase QnA — Public; mini-swe-agent30009
Meta AI (originally Facebook AI Research)Muse Spark 1.1 (xHigh)42.2%labs.scale.com for Muse Spark 1.1 (xHigh) on SWE Atlas Codebase QnA — Public; mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Mean pass@1Reported Benchmark leaderboardScale AIIndependentRecord for Muse Spark 1.1 on SWE Atlas Codebase QnA — Public; mini-swe-agent80002
Meta AI (originally Facebook AI Research)Muse Spark 1.2 (xhigh)46.2%dev.meta.ai for Muse Spark 1.2 (xhigh) on SWE Atlas Codebase QnA — Public; mini-swe-agent (opens in a new tab)Mean pass@1Reported Model pageMetaSelf-reportedRecord for Muse Spark 1.2 on SWE Atlas Codebase QnA — Public; mini-swe-agent80003
Meta AI (originally Facebook AI Research)Muse Spark 1.3 (max)59.4%dev.meta.ai for Muse Spark 1.3 (max) on SWE Atlas Codebase QnA — Public; mini-swe-agent (opens in a new tab)Mean pass@1Reported Model pageMetaSelf-reportedRecord for Muse Spark 1.3 on SWE Atlas Codebase QnA — Public; mini-swe-agent80005