OpenAI GPT-5.4 (xHigh)36.3% labs.scale.com for GPT-5.4 (xHigh) on SWE Atlas Codebase QnA — Public; mini-swe-agent (opens in a new tab) Archived copy (opens in a new tab) Mean pass@1 Reported October 7, 2026 Benchmark leaderboard Scale AI Independent Record for GPT-5.4 on SWE Atlas Codebase QnA — Public; mini-swe-agent 10007 DeepSeek AI DeepSeek-V4-Pro 27.2% labs.scale.com for DeepSeek-V4-Pro on SWE Atlas Codebase QnA — Public; mini-swe-agent (opens in a new tab) Archived copy (opens in a new tab) Mean pass@1 Reported October 7, 2026 Benchmark leaderboard Scale AI Independent Record for DeepSeek-V4-Pro on SWE Atlas Codebase QnA — Public; mini-swe-agent 110002 Moonshot AI Kimi K2.5 13.1% labs.scale.com for Kimi K2.5 on SWE Atlas Codebase QnA — Public; mini-swe-agent (opens in a new tab) Archived copy (opens in a new tab) Mean pass@1 Reported October 7, 2026 Benchmark leaderboard Scale AI Independent Record for Kimi K2.5 on SWE Atlas Codebase QnA — Public; mini-swe-agent 120001 MiniMax MiniMax M2.5 10.3% labs.scale.com for MiniMax M2.5 on SWE Atlas Codebase QnA — Public; mini-swe-agent (opens in a new tab) Archived copy (opens in a new tab) Mean pass@1 Reported October 7, 2026 Benchmark leaderboard Scale AI Independent Record for MiniMax M2.5 on SWE Atlas Codebase QnA — Public; mini-swe-agent 140001 Z.ai GLM-5 20.5% labs.scale.com for GLM-5 on SWE Atlas Codebase QnA — Public; mini-swe-agent (opens in a new tab) Archived copy (opens in a new tab) Mean pass@1 Reported October 7, 2026 Benchmark leaderboard Scale AI Independent Record for GLM-5 on SWE Atlas Codebase QnA — Public; mini-swe-agent 150002 Z.ai GLM-5.2 48.1% labs.scale.com for GLM-5.2 on SWE Atlas Codebase QnA — Public; mini-swe-agent (opens in a new tab) Archived copy (opens in a new tab) Mean pass@1 Reported October 7, 2026 Benchmark leaderboard Scale AI Independent Record for GLM-5.2 on SWE Atlas Codebase QnA — Public; mini-swe-agent 150004 Anthropic Claude Opus 4.6 30.0% labs.scale.com for Claude Opus 4.6 on SWE Atlas Codebase QnA — Public; mini-swe-agent (opens in a new tab) Archived copy (opens in a new tab) Mean pass@1 Reported October 7, 2026 Benchmark leaderboard Scale AI Independent Record for Claude Opus 4.6 on SWE Atlas Codebase QnA — Public; mini-swe-agent 20008 Google (DeepMind) Gemini 3.8 Flash 47.0% labs.scale.com for Gemini 3.8 Flash on SWE Atlas Codebase QnA — Public; mini-swe-agent (opens in a new tab) Archived copy (opens in a new tab) Mean pass@1 Reported October 7, 2026 Benchmark leaderboard Scale AI Independent Record for Gemini 3.8 Flash on SWE Atlas Codebase QnA — Public; mini-swe-agent 30004 Google (DeepMind) Gemini 3.1 Pro 13.5% labs.scale.com for Gemini 3.1 Pro on SWE Atlas Codebase QnA — Public; mini-swe-agent (opens in a new tab) Archived copy (opens in a new tab) Mean pass@1 Reported October 7, 2026 Benchmark leaderboard Scale AI Independent Record for Gemini 3.1 Pro on SWE Atlas Codebase QnA — Public; mini-swe-agent 30005 Google (DeepMind) Gemini 3 Flash 8.2% labs.scale.com for Gemini 3 Flash on SWE Atlas Codebase QnA — Public; mini-swe-agent (opens in a new tab) Archived copy (opens in a new tab) Mean pass@1 Reported October 7, 2026 Benchmark leaderboard Scale AI Independent Record for Gemini 3 Flash on SWE Atlas Codebase QnA — Public; mini-swe-agent 30009 Meta AI (originally Facebook AI Research) Muse Spark 1.1 (xHigh)42.2% labs.scale.com for Muse Spark 1.1 (xHigh) on SWE Atlas Codebase QnA — Public; mini-swe-agent (opens in a new tab) Archived copy (opens in a new tab) Mean pass@1 Reported October 7, 2026 Benchmark leaderboard Scale AI Independent Record for Muse Spark 1.1 on SWE Atlas Codebase QnA — Public; mini-swe-agent 80002 Meta AI (originally Facebook AI Research) Muse Spark 1.2 (xhigh)46.2% dev.meta.ai for Muse Spark 1.2 (xhigh) on SWE Atlas Codebase QnA — Public; mini-swe-agent (opens in a new tab) Mean pass@1 Reported September 2, 2026 Model page Meta Self-reported Record for Muse Spark 1.2 on SWE Atlas Codebase QnA — Public; mini-swe-agent 80003 Meta AI (originally Facebook AI Research) Muse Spark 1.3 (max)59.4% dev.meta.ai for Muse Spark 1.3 (max) on SWE Atlas Codebase QnA — Public; mini-swe-agent (opens in a new tab) Mean pass@1 Reported September 2, 2026 Model page Meta Self-reported Record for Muse Spark 1.3 on SWE Atlas Codebase QnA — Public; mini-swe-agent 80005