DeepSWE
Updated
Cite
Fill in the access date before citing.
Plain text
Benchmark Registry. DeepSWE 1.1 benchmark results. Benchmark identifier: deep-swe/1.1. https://benchmarkregistry.org/benchmarks/deep-swe/1.1. Accessed [YYYY-MM-DD]. Data license: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Attribution: Benchmark Registry, a project of Densa Labs — https://benchmarkregistry.org/ (CC BY 4.0). Include the cited record URL and identify any changes.
BibTeX
@misc{br_deep_swe_1_1,
title = {DeepSWE 1.1 benchmark results},
publisher = {Benchmark Registry},
url = {https://benchmarkregistry.org/benchmarks/deep-swe/1.1},
urldate = {YYYY-MM-DD},
note = {Benchmark identifier: deep-swe/1.1; data license: CC BY 4.0; access date to be filled by the reader}
}Showing the selected evaluation. Show all results
Results
1 result
| Provider | Model | Score | Source | Registry No. |
|---|---|---|---|---|
| OpenAI | GPT-6.1 Sol (high) | 75.2% | openai.com for GPT-6.1 Sol (high) on DeepSWE 1.1 (opens in a new tab)DeepSWE Pass@1Reported OpenAIReport an issueCiteFill in the access date before citing. Plain text Benchmark Registry. GPT-6.1 Sol: DeepSWE 1.1, 75.2% (high). Registry No. 10015; Record No. 0bff3294cc9bd43f90a648f91749beae5421bae99832831ecaa40e7038e0361a. https://benchmarkregistry.org/benchmarks/deep-swe/1.1?view=history&result=0bff3294cc9bd43f90a648f91749beae5421bae99832831ecaa40e7038e0361a#BR-0bff3294cc9bd43f90a648f91749beae5421bae99832831ecaa40e7038e0361a. Accessed [YYYY-MM-DD]. Data license: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Attribution: Benchmark Registry, a project of Densa Labs — https://benchmarkregistry.org/ (CC BY 4.0). Include the cited record URL and identify any changes. BibTeX @misc{br_0bff3294cc9bd43f90a648f91749beae5421bae99832831ecaa40e7038e0361a,
title = {GPT-6.1 Sol: DeepSWE 1.1, 75.2\% (high)},
publisher = {Benchmark Registry},
url = {https://benchmarkregistry.org/benchmarks/deep-swe/1.1?view=history\&result=0bff3294cc9bd43f90a648f91749beae5421bae99832831ecaa40e7038e0361a\#BR-0bff3294cc9bd43f90a648f91749beae5421bae99832831ecaa40e7038e0361a},
urldate = {YYYY-MM-DD},
note = {Registry No. 10015; Record No. 0bff3294cc9bd43f90a648f91749beae5421bae99832831ecaa40e7038e0361a; data license: CC BY 4.0; access date to be filled by the reader}
}Badge for this model's latest reported score in the benchmark family; it may differ from this record. Markdown  HTML <img src="https://benchmarkregistry.org/badge/10015/deep-swe.svg" alt="GPT-6.1 Sol: DeepSWE" height="24"> | 10015 |