Versions

1 version, plus 3 variants and configurations · Metric: DeepSWE Pass@1

Versions of DeepSWE
VersionReleasedModelsResults
DeepSWE 1.1Latest
3 variants of 1.1
Variants of DeepSWE 1.1
VariantConfigurationReleasedModelsResults
DeepSWE 1.1 — Mercor mini-swe-agent, 500 stepsMercor mini-swe-agent, 500 steps, 2 hoursJune 15, 20264447
DeepSWE 1.1 — Muse CodeMuse Code harnessJune 15, 202611
DeepSWE 1.1 — mini-swe-agentmini-swe-agent harnessJune 15, 202644
June 15, 20262864

Recently reported results: DeepSWE 1.1

Latest version: DeepSWE 1.1

Recently reported results for DeepSWE 1.1
ModelScoreSource
GPT-6.1 Sol (high)75.2%openai.com for GPT-6.1 Sol (high) on DeepSWE 1.1 (opens in a new tab)DeepSWE Pass@1Reported Launch postOpenAISelf-reportedRecord for GPT-6.1 Sol on DeepSWE 1.1
GPT-6.1 Sol (low)64.4%openai.com for GPT-6.1 Sol (low) on DeepSWE 1.1 (opens in a new tab)DeepSWE Pass@1Reported Launch postOpenAISelf-reportedRecord for GPT-6.1 Sol on DeepSWE 1.1
GPT-6.1 Sol (medium)73.0%openai.com for GPT-6.1 Sol (medium) on DeepSWE 1.1 (opens in a new tab)DeepSWE Pass@1Reported Launch postOpenAISelf-reportedRecord for GPT-6.1 Sol on DeepSWE 1.1
GPT-6.1 Sol (xhigh)71.9%openai.com for GPT-6.1 Sol (xhigh) on DeepSWE 1.1 (opens in a new tab)DeepSWE Pass@1Reported Launch postOpenAISelf-reportedRecord for GPT-6.1 Sol on DeepSWE 1.1
GPT-6.1 Sol (max)71.9%openai.com for GPT-6.1 Sol (max) on DeepSWE 1.1 (opens in a new tab)DeepSWE Pass@1Reported Launch postOpenAISelf-reportedRecord for GPT-6.1 Sol on DeepSWE 1.1