Results

27 results

SWE-bench Pro Public results
ProviderModelScoreSourceRegistry No.
AnthropicClaude Haiku 5.5 (max)64.8%anthropic.com for Claude Haiku 5.5 (max) on SWE-bench Pro Public (opens in a new tab)Archived copy (opens in a new tab)Public tasks resolvedReported System cardAnthropicSelf-reportedRecord for Claude Haiku 5.5 on SWE-bench Pro Public20018
AnthropicClaude Sonnet 5.5 (max)81.3%anthropic.com for Claude Sonnet 5.5 (max) on SWE-bench Pro Public (opens in a new tab)Public tasks resolvedReported System cardAnthropicSelf-reportedRecord for Claude Sonnet 5.5 on SWE-bench Pro Public20016
AnthropicClaude Opus 5.5 (max)89.9%www-cdn.anthropic.com for Claude Opus 5.5 (max) on SWE-bench Pro Public (opens in a new tab)Public tasks resolvedReported System cardAnthropicSelf-reportedRecord for Claude Opus 5.5 on SWE-bench Pro Public20015
Microsoft AIMAI-Thinking-152.8%microsoft.ai for MAI-Thinking-1 on SWE-bench Pro Public (opens in a new tab)Public tasks resolvedReported Technical reportMicrosoft AISelf-reportedMicrosoftRecord for MAI-Thinking-1 on SWE-bench Pro Public70003
Meta AI (originally Facebook AI Research)Muse Glimmer 30B (high)51.2%dev.meta.ai for Muse Glimmer 30B (high) on SWE-bench Pro Public (opens in a new tab)Public tasks resolvedReported Model pageMetaSelf-reportedRecord for Muse Glimmer 30B on SWE-bench Pro Public80004
Thinking Machines LabInkling-Small (effort=0.99)55.9%thinkingmachines.ai for Inkling-Small (effort=0.99) on SWE-bench Pro Public (opens in a new tab)Public tasks resolvedReported Launch postThinking Machines LabSelf-reportedRecord for Inkling-Small on SWE-bench Pro Public160002
Google (DeepMind)Gemini 3.6 Flash58.7%deepmind.google for Gemini 3.6 Flash on SWE-bench Pro Public (opens in a new tab)Public tasks resolvedReported Model cardGoogle DeepMindSelf-reportedRecord for Gemini 3.6 Flash on SWE-bench Pro Public30007
SpaceXAIGrok 4.564.7%x.ai for Grok 4.5 on SWE-bench Pro Public (opens in a new tab)Public tasks resolvedReported Launch postSpaceXAISelf-reportedRecord for Grok 4.5 on SWE-bench Pro Public40001
Thinking Machines LabInkling (effort=0.99)54.3%thinkingmachines.ai for Inkling (effort=0.99) on SWE-bench Pro Public (opens in a new tab)Public tasks resolvedReported Launch postThinking Machines LabSelf-reportedRecord for Inkling on SWE-bench Pro Public160001
Meta AI (originally Facebook AI Research)Muse Spark 1.1 (xhigh)61.5%dev.meta.ai for Muse Spark 1.1 (xhigh) on SWE-bench Pro Public (opens in a new tab)Public tasks resolvedReported Model pageMetaSelf-reportedRecord for Muse Spark 1.1 on SWE-bench Pro Public80002
AnthropicClaude Sonnet 5 (max)63.2%www-cdn.anthropic.com for Claude Sonnet 5 (max) on SWE-bench Pro Public (opens in a new tab)Public tasks resolvedReported System cardAnthropicSelf-reportedRecord for Claude Sonnet 5 on SWE-bench Pro Public20013
Z.aiGLM-5.262.1%z.ai for GLM-5.2 on SWE-bench Pro Public (opens in a new tab)Public tasks resolvedReported Launch postZ.aiSelf-reportedRecord for GLM-5.2 on SWE-bench Pro Public150004
MiniMaxMiniMax M359.0%minimax.io for MiniMax M3 on SWE-bench Pro Public (opens in a new tab)Public tasks resolvedReported Launch postMiniMaxSelf-reportedRecord for MiniMax M3 on SWE-bench Pro Public140003
AnthropicClaude Opus 4.8 (adaptive thinking, max)69.2%www-cdn.anthropic.com for Claude Opus 4.8 (adaptive thinking, max) on SWE-bench Pro Public (opens in a new tab)Public tasks resolvedReported System cardAnthropicSelf-reportedRecord for Claude Opus 4.8 on SWE-bench Pro Public20011
Google (DeepMind)Gemini 3.5 Flash55.1%deepmind.google for Gemini 3.5 Flash on SWE-bench Pro Public (opens in a new tab)Public tasks resolvedReported Model cardGoogle DeepMindSelf-reportedRecord for Gemini 3.5 Flash on SWE-bench Pro Public30006
DeepSeek AIDeepSeek-V4-Flash (max)52.6%huggingface.co for DeepSeek-V4-Flash (max) on SWE-bench Pro Public (opens in a new tab)Public tasks resolvedReported Model cardDeepSeekSelf-reportedDeepSeek AIRecord for DeepSeek-V4-Flash on SWE-bench Pro Public110001
DeepSeek AIDeepSeek-V4-Pro (max)55.4%huggingface.co for DeepSeek-V4-Pro (max) on SWE-bench Pro Public (opens in a new tab)Public tasks resolvedReported Model cardDeepSeekSelf-reportedDeepSeek AIRecord for DeepSeek-V4-Pro on SWE-bench Pro Public110002
Moonshot AIKimi K2.6 (thinking)58.6%huggingface.co for Kimi K2.6 (thinking) on SWE-bench Pro Public (opens in a new tab)Public tasks resolvedReported Model cardMoonshot AISelf-reportedRecord for Kimi K2.6 on SWE-bench Pro Public120002
AnthropicClaude Opus 4.7 (adaptive thinking, max)64.3%www-cdn.anthropic.com for Claude Opus 4.7 (adaptive thinking, max) on SWE-bench Pro Public (opens in a new tab)Public tasks resolvedReported System cardAnthropicSelf-reportedRecord for Claude Opus 4.7 on SWE-bench Pro Public20010
Meta AI (originally Facebook AI Research)Muse Spark (thinking)52.4%ai.meta.com for Muse Spark (thinking) on SWE-bench Pro Public (opens in a new tab)Public tasks resolvedReported Evaluation methodologyMetaSelf-reportedRecord for Muse Spark on SWE-bench Pro Public80001
Z.aiGLM-5.158.4%z.ai for GLM-5.1 on SWE-bench Pro Public (opens in a new tab)Public tasks resolvedReported Launch postZ.aiSelf-reportedRecord for GLM-5.1 on SWE-bench Pro Public150003
MiniMaxMiniMax M2.756.2%minimax.io for MiniMax M2.7 on SWE-bench Pro Public (opens in a new tab)Public tasks resolvedReported Launch postMiniMaxSelf-reportedRecord for MiniMax M2.7 on SWE-bench Pro Public140002
OpenAIGPT-5.4 (xhigh)57.7%openai.com for GPT-5.4 (xhigh) on SWE-bench Pro Public (opens in a new tab)Public tasks resolvedReported Launch postOpenAISelf-reportedRecord for GPT-5.4 on SWE-bench Pro Public10007
Google (DeepMind)Gemini 3.1 Pro (High)54.2%deepmind.google for Gemini 3.1 Pro (High) on SWE-bench Pro Public (opens in a new tab)Public tasks resolvedReported Model cardGoogle DeepMindSelf-reportedRecord for Gemini 3.1 Pro on SWE-bench Pro Public30005
OpenAIGPT-5.3-Codex (xhigh)56.8%openai.com for GPT-5.3-Codex (xhigh) on SWE-bench Pro Public (opens in a new tab)Public tasks resolvedReported Launch postOpenAISelf-reportedRecord for GPT-5.3-Codex on SWE-bench Pro Public10006
Moonshot AIKimi K2.5 (non-thinking)50.7%huggingface.co for Kimi K2.5 (non-thinking) on SWE-bench Pro Public (opens in a new tab)Public tasks resolvedReported Model cardMoonshot AISelf-reportedRecord for Kimi K2.5 on SWE-bench Pro Public120001
OpenAIGPT-5.2 (xhigh)55.6%openai.com for GPT-5.2 (xhigh) on SWE-bench Pro Public (opens in a new tab)Public tasks resolvedReported Launch postOpenAISelf-reportedRecord for GPT-5.2 on SWE-bench Pro Public10012