SWE-bench
Updated
Cite
Fill in the access date before citing.
Plain text
Benchmark Registry. SWE-bench Verified — bash-only harness benchmark results. Benchmark identifier: swe-bench/verified-bash-only. https://benchmarkregistry.org/benchmarks/swe-bench/verified-bash-only. Accessed [YYYY-MM-DD]. Data license: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Attribution: Benchmark Registry, a project of Densa Labs — https://benchmarkregistry.org/ (CC BY 4.0). Include the cited record URL and identify any changes.
BibTeX
@misc{br_swe_bench_verified_bash_only,
title = {SWE-bench Verified — bash-only harness benchmark results},
publisher = {Benchmark Registry},
url = {https://benchmarkregistry.org/benchmarks/swe-bench/verified-bash-only},
urldate = {YYYY-MM-DD},
note = {Benchmark identifier: swe-bench/verified-bash-only; data license: CC BY 4.0; access date to be filled by the reader}
}Results
2 results
| Provider | Model | Score | Source | Registry No. |
|---|---|---|---|---|
| Thinking Machines Lab | Inkling-Small (effort=0.99) | 80.2% | thinkingmachines.ai for Inkling-Small (effort=0.99) on SWE-bench Verified — bash-only harness (opens in a new tab)Resolved TasksReported Launch postThinking Machines LabSelf-reportedRecord for Inkling-Small on SWE-bench Verified — bash-only harness | 160002 |
| Thinking Machines Lab | Inkling (effort=0.99) | 77.6% | thinkingmachines.ai for Inkling (effort=0.99) on SWE-bench Verified — bash-only harness (opens in a new tab)Resolved TasksReported Launch postThinking Machines LabSelf-reportedRecord for Inkling on SWE-bench Verified — bash-only harness | 160001 |