Benchmarks

49 results across 23 benchmarks

Benchmark results for GPT-6.1 Sol
BenchmarkScoreSource
APEX-Agents OriginalEffort: Max60.0%mercor.com for GPT-6.1 Sol (max) on APEX-Agents Original (opens in a new tab)Archived copy (opens in a new tab)Pass rateReported Benchmark leaderboardMercorIndependentRecord for GPT-6.1 Sol on APEX-Agents Original
AutomationBench 1.0.6Effort: Low24.7%openai.com for GPT-6.1 Sol (low) on AutomationBench 1.0.6 (opens in a new tab)Task pass rateReported Launch postOpenAISelf-reportedRecord for GPT-6.1 Sol on AutomationBench 1.0.6
AutomationBench 1.0.6Effort: Medium31.7%openai.com for GPT-6.1 Sol (medium) on AutomationBench 1.0.6 (opens in a new tab)Task pass rateReported Launch postOpenAISelf-reportedRecord for GPT-6.1 Sol on AutomationBench 1.0.6
AutomationBench 1.0.6Effort: High33.2%openai.com for GPT-6.1 Sol (high) on AutomationBench 1.0.6 (opens in a new tab)Task pass rateReported Launch postOpenAISelf-reportedRecord for GPT-6.1 Sol on AutomationBench 1.0.6
AutomationBench 1.0.6Effort: xHigh35.5%openai.com for GPT-6.1 Sol (xhigh) on AutomationBench 1.0.6 (opens in a new tab)Task pass rateReported Launch postOpenAISelf-reportedRecord for GPT-6.1 Sol on AutomationBench 1.0.6
AutomationBench 1.0.6Effort: Max36.1%openai.com for GPT-6.1 Sol (max) on AutomationBench 1.0.6 (opens in a new tab)Task pass rateReported Launch postOpenAISelf-reportedRecord for GPT-6.1 Sol on AutomationBench 1.0.6
BrowseComp 130-question subset — Mercor web research agentEffort: MaxConfiguration: Mercor web research agent, 100 steps, 3 hours95.4%mercor.com for GPT-6.1 Sol (max) on BrowseComp 130-question subset — Mercor web research agent (opens in a new tab)Archived copy (opens in a new tab)BrowseComp AccuracyReported Benchmark leaderboardMercorIndependentRecord for GPT-6.1 Sol on BrowseComp 130-question subset — Mercor web research agent
CharXiv Descriptive and reasoning — Mercor single-shotEffort: MaxConfiguration: Mercor single-shot agent96.0%mercor.com for GPT-6.1 Sol (max) on CharXiv Descriptive and reasoning — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)CharXiv AccuracyReported Benchmark leaderboardMercorIndependentRecord for GPT-6.1 Sol on CharXiv Descriptive and reasoning — Mercor single-shot
Coronavirus–ACE2 Cell-Entry Screen Original0.423deploymentsafety.openai.com for GPT-6.1 Sol on Coronavirus–ACE2 Cell-Entry Screen Original (opens in a new tab)Composite scoreReported System card addendumOpenAISelf-reportedRecord for GPT-6.1 Sol on Coronavirus–ACE2 Cell-Entry Screen Original
DeepSWE 1.1Effort: Low64.4%openai.com for GPT-6.1 Sol (low) on DeepSWE 1.1 (opens in a new tab)DeepSWE Pass@1Reported Launch postOpenAISelf-reportedRecord for GPT-6.1 Sol on DeepSWE 1.1
DeepSWE 1.1Effort: Medium73.0%openai.com for GPT-6.1 Sol (medium) on DeepSWE 1.1 (opens in a new tab)DeepSWE Pass@1Reported Launch postOpenAISelf-reportedRecord for GPT-6.1 Sol on DeepSWE 1.1
DeepSWE 1.1Effort: High75.2%openai.com for GPT-6.1 Sol (high) on DeepSWE 1.1 (opens in a new tab)DeepSWE Pass@1Reported Launch postOpenAISelf-reportedRecord for GPT-6.1 Sol on DeepSWE 1.1
DeepSWE 1.1Effort: xHigh71.9%openai.com for GPT-6.1 Sol (xhigh) on DeepSWE 1.1 (opens in a new tab)DeepSWE Pass@1Reported Launch postOpenAISelf-reportedRecord for GPT-6.1 Sol on DeepSWE 1.1
DeepSWE 1.1Effort: Max71.9%openai.com for GPT-6.1 Sol (max) on DeepSWE 1.1 (opens in a new tab)DeepSWE Pass@1Reported Launch postOpenAISelf-reportedRecord for GPT-6.1 Sol on DeepSWE 1.1
DeepSWE 1.1 — Mercor mini-swe-agent, 500 stepsEffort: MaxConfiguration: Mercor mini-swe-agent, 500 steps, 2 hours72.3%mercor.com for GPT-6.1 Sol (max) on DeepSWE 1.1 — Mercor mini-swe-agent, 500 steps (opens in a new tab)Archived copy (opens in a new tab)DeepSWE Pass@1Reported Benchmark leaderboardMercorIndependentRecord for GPT-6.1 Sol on DeepSWE 1.1 — Mercor mini-swe-agent, 500 steps
Finance Agent Benchmark 2 — October 2026 evaluationEffort: Max52.0%vals.ai for GPT-6.1 Sol (max) on Finance Agent Benchmark 2 — October 2026 evaluation (opens in a new tab)Archived copy (opens in a new tab)Finance Agent AccuracyReported Benchmark leaderboardVals AIIndependentRecord for GPT-6.1 Sol on Finance Agent Benchmark 2 — October 2026 evaluation
GDP.pdfEffort: Low27.0%openai.com for GPT-6.1 Sol (low) on GDP.pdf GDP.pdf (opens in a new tab)GDP.pdf All Pass RateReported Launch postOpenAISelf-reportedRecord for GPT-6.1 Sol on GDP.pdf GDP.pdf
GDP.pdfEffort: Medium30.0%openai.com for GPT-6.1 Sol (medium) on GDP.pdf GDP.pdf (opens in a new tab)GDP.pdf All Pass RateReported Launch postOpenAISelf-reportedRecord for GPT-6.1 Sol on GDP.pdf GDP.pdf
GDP.pdfEffort: High32.0%openai.com for GPT-6.1 Sol (high) on GDP.pdf GDP.pdf (opens in a new tab)GDP.pdf All Pass RateReported Launch postOpenAISelf-reportedRecord for GPT-6.1 Sol on GDP.pdf GDP.pdf
GDP.pdfEffort: xHigh31.8%openai.com for GPT-6.1 Sol (xhigh) on GDP.pdf GDP.pdf (opens in a new tab)GDP.pdf All Pass RateReported Launch postOpenAISelf-reportedRecord for GPT-6.1 Sol on GDP.pdf GDP.pdf
GDP.pdfEffort: Max31.0%openai.com for GPT-6.1 Sol (max) on GDP.pdf GDP.pdf (opens in a new tab)GDP.pdf All Pass RateReported Launch postOpenAISelf-reportedRecord for GPT-6.1 Sol on GDP.pdf GDP.pdf
GPQA Diamond — Epoch AI harnessEffort: MaxConfiguration: Epoch AI harness95.4%epoch.ai for GPT-6.1 Sol (max) on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness (opens in a new tab)Archived copy (opens in a new tab)GPQA Diamond AccuracyReported Evaluated Benchmark leaderboardEpoch AIIndependentRecord for GPT-6.1 Sol on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness
GPQA Diamond — Mercor single-shotEffort: MaxConfiguration: Mercor single-shot agent95.1%mercor.com for GPT-6.1 Sol (max) on Graduate-Level Google-Proof Q&A Diamond — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)GPQA Diamond AccuracyReported Benchmark leaderboardMercorIndependentRecord for GPT-6.1 Sol on Graduate-Level Google-Proof Q&A Diamond — Mercor single-shot
HLAB October 2026 evaluationEffort: Max5.4%vals.ai for GPT-6.1 Sol (max) on Harvey's Legal Agent Benchmark October 2026 evaluation (opens in a new tab)Archived copy (opens in a new tab)HLAB Task Pass RateReported Benchmark leaderboardVals AIIndependentRecord for GPT-6.1 Sol on Harvey's Legal Agent Benchmark October 2026 evaluation
MedXpertQA MM — Mercor single-shotEffort: MaxConfiguration: Mercor single-shot agent85.7%mercor.com for GPT-6.1 Sol (max) on MedXpertQA MM — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)AccuracyReported Benchmark leaderboardMercorIndependentRecord for GPT-6.1 Sol on MedXpertQA MM — Mercor single-shot
MMLU-Pro Original — Mercor single-shotEffort: MaxConfiguration: Mercor single-shot agent91.0%mercor.com for GPT-6.1 Sol (max) on MMLU-Pro Original — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)AccuracyReported Benchmark leaderboardMercorIndependentRecord for GPT-6.1 Sol on MMLU-Pro Original — Mercor single-shot
MMMU-Pro 1,730 questions — Mercor single-shotEffort: MaxConfiguration: Mercor single-shot agent85.8%mercor.com for GPT-6.1 Sol (max) on MMMU-Pro 1,730 questions — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Not statedReported Benchmark leaderboardMercorIndependentRecord for GPT-6.1 Sol on MMMU-Pro 1,730 questions — Mercor single-shot
OSWorld 2.0 v2026.08.08 Offline — PartialEffort: Low59.0%openai.com for GPT-6.1 Sol (low) on OSWorld 2.0 v2026.08.08 Offline — Partial (opens in a new tab)OSWorld Partial ScoreReported Launch postOpenAISelf-reportedRecord for GPT-6.1 Sol on OSWorld 2.0 v2026.08.08 Offline — Partial
OSWorld 2.0 v2026.08.08 Offline — PartialEffort: Medium66.8%openai.com for GPT-6.1 Sol (medium) on OSWorld 2.0 v2026.08.08 Offline — Partial (opens in a new tab)OSWorld Partial ScoreReported Launch postOpenAISelf-reportedRecord for GPT-6.1 Sol on OSWorld 2.0 v2026.08.08 Offline — Partial
OSWorld 2.0 v2026.08.08 Offline — PartialEffort: High69.6%openai.com for GPT-6.1 Sol (high) on OSWorld 2.0 v2026.08.08 Offline — Partial (opens in a new tab)OSWorld Partial ScoreReported Launch postOpenAISelf-reportedRecord for GPT-6.1 Sol on OSWorld 2.0 v2026.08.08 Offline — Partial
OSWorld 2.0 v2026.08.08 Offline — PartialEffort: xHigh69.4%openai.com for GPT-6.1 Sol (xhigh) on OSWorld 2.0 v2026.08.08 Offline — Partial (opens in a new tab)OSWorld Partial ScoreReported Launch postOpenAISelf-reportedRecord for GPT-6.1 Sol on OSWorld 2.0 v2026.08.08 Offline — Partial
OSWorld 2.0 v2026.08.08 Offline — PartialEffort: Max71.4%openai.com for GPT-6.1 Sol (max) on OSWorld 2.0 v2026.08.08 Offline — Partial (opens in a new tab)OSWorld Partial ScoreReported Launch postOpenAISelf-reportedRecord for GPT-6.1 Sol on OSWorld 2.0 v2026.08.08 Offline — Partial
Phage–plasmid Co-evolution Original12.946deploymentsafety.openai.com for GPT-6.1 Sol on Phage–plasmid Co-evolution Original (opens in a new tab)Negative log-likelihoodReported System card addendumOpenAISelf-reportedRecord for GPT-6.1 Sol on Phage–plasmid Co-evolution Original
PostTrainBench 1.2 — Codex CLIEffort: MaxConfiguration: Codex harness38.8%posttrainbench.com for GPT-6.1 Sol (Max) on PostTrainBench 1.2 — Codex CLI (opens in a new tab)Archived copy (opens in a new tab)PostTrainBench Weighted AggregateReported Benchmark leaderboardPostTrainBenchIndependentPostTrainBench TeamRecord for GPT-6.1 Sol on PostTrainBench 1.2 — Codex CLI
SHP2 Protein Function Prediction Original0.332deploymentsafety.openai.com for GPT-6.1 Sol on SHP2 Protein Function Prediction Original (opens in a new tab)Mean R² across three assay datasetsReported System card addendumOpenAISelf-reportedRecord for GPT-6.1 Sol on SHP2 Protein Function Prediction Original
SimpleQA Verified Original — Epoch AI harnessEffort: MaxConfiguration: Epoch AI harness73.9%epoch.ai for GPT-6.1 Sol (max) on SimpleQA Verified Original — Epoch AI harness (opens in a new tab)Archived copy (opens in a new tab)AccuracyReported Evaluated Benchmark leaderboardEpoch AIIndependentRecord for GPT-6.1 Sol on SimpleQA Verified Original — Epoch AI harness
SWE Atlas Codebase QnA — Mercor mini-swe-agent, 250 stepsEffort: MaxConfiguration: Mercor mini-swe-agent, 250 steps, 1 hour48.4%mercor.com for GPT-6.1 Sol (max) on SWE Atlas Codebase QnA — Mercor mini-swe-agent, 250 steps (opens in a new tab)Archived copy (opens in a new tab)Mean pass@1Reported Benchmark leaderboardMercorIndependentRecord for GPT-6.1 Sol on SWE Atlas Codebase QnA — Mercor mini-swe-agent, 250 steps
SWE-bench Multilingual — Mercor mini-swe-agent, 1000 stepsEffort: MaxConfiguration: Mercor mini-swe-agent, 1000 steps, 3 hours84.9%mercor.com for GPT-6.1 Sol (max) on SWE-bench Multilingual — Mercor mini-swe-agent, 1000 steps (opens in a new tab)Archived copy (opens in a new tab)Resolved TasksReported Benchmark leaderboardMercorIndependentRecord for GPT-6.1 Sol on SWE-bench Multilingual — Mercor mini-swe-agent, 1000 steps
SWE-bench Verified — Mercor mini-swe-agent, 1000 stepsEffort: MaxConfiguration: Mercor mini-swe-agent, 1000 steps, 3 hours87.3%mercor.com for GPT-6.1 Sol (max) on SWE-bench Verified — Mercor mini-swe-agent, 1000 steps (opens in a new tab)Archived copy (opens in a new tab)Resolved TasksReported Benchmark leaderboardMercorIndependentRecord for GPT-6.1 Sol on SWE-bench Verified — Mercor mini-swe-agent, 1000 steps
Terminal-Bench 4.0 — CodexEffort: MaxConfiguration: Codex harness58.2%tbench.ai for GPT-6.1 Sol (max) on Terminal-Bench 4.0 — Codex (opens in a new tab)Archived copy (opens in a new tab)Terminal-Bench AccuracyReported Benchmark leaderboardTerminal-BenchIndependentTerminal-Bench TeamRecord for GPT-6.1 Sol on Terminal-Bench 4.0 — Codex
Terminal-Bench 4.0 — Mercor mini-swe-agentEffort: MaxConfiguration: Mercor mini-swe-agent, 1000 steps, 8 hours59.6%mercor.com for GPT-6.1 Sol (max) on Terminal-Bench 4.0 — Mercor mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Terminal-Bench AccuracyReported Benchmark leaderboardMercorIndependentRecord for GPT-6.1 Sol on Terminal-Bench 4.0 — Mercor mini-swe-agent
Terminal-Bench 4.0 — Vals AI mini-swe-agentEffort: MaxConfiguration: mini-swe-agent harness55.1%vals.ai for GPT-6.1 Sol (max) on Terminal-Bench 4.0 — Vals AI mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Terminal-Bench AccuracyReported Benchmark leaderboardVals AIIndependentRecord for GPT-6.1 Sol on Terminal-Bench 4.0 — Vals AI mini-swe-agent
Terminal-Bench Science 0.1Effort: Low43.7%openai.com for GPT-6.1 Sol (low) on Terminal-Bench Science 0.1 (opens in a new tab)Terminal-Bench AccuracyReported Launch postOpenAISelf-reportedRecord for GPT-6.1 Sol on Terminal-Bench Science 0.1
Terminal-Bench Science 0.1Effort: Medium47.6%openai.com for GPT-6.1 Sol (medium) on Terminal-Bench Science 0.1 (opens in a new tab)Terminal-Bench AccuracyReported Launch postOpenAISelf-reportedRecord for GPT-6.1 Sol on Terminal-Bench Science 0.1
Terminal-Bench Science 0.1Effort: High51.1%openai.com for GPT-6.1 Sol (high) on Terminal-Bench Science 0.1 (opens in a new tab)Terminal-Bench AccuracyReported Launch postOpenAISelf-reportedRecord for GPT-6.1 Sol on Terminal-Bench Science 0.1
Terminal-Bench Science 0.1Effort: xHigh53.7%openai.com for GPT-6.1 Sol (xhigh) on Terminal-Bench Science 0.1 (opens in a new tab)Terminal-Bench AccuracyReported Launch postOpenAISelf-reportedRecord for GPT-6.1 Sol on Terminal-Bench Science 0.1
Terminal-Bench Science 0.1Effort: Max57.0%openai.com for GPT-6.1 Sol (max) on Terminal-Bench Science 0.1 (opens in a new tab)Terminal-Bench AccuracyReported Launch postOpenAISelf-reportedRecord for GPT-6.1 Sol on Terminal-Bench Science 0.1
Vals Index 2.1Effort: Max61.2%vals.ai for GPT-6.1 Sol (max) on Vals Index 2.1 (opens in a new tab)Archived copy (opens in a new tab)Vals Index GDP-weighted AccuracyReported Benchmark leaderboardVals AIIndependentRecord for GPT-6.1 Sol on Vals Index 2.1
Vibe Code Bench 1.1 — OpenHands, October 2026 evaluationEffort: MaxConfiguration: OpenHands harness88.9%vals.ai for GPT-6.1 Sol (max) on Vibe Code Bench 1.1 — OpenHands, October 2026 evaluation (opens in a new tab)Archived copy (opens in a new tab)Vibe Code Bench ScoreReported Benchmark leaderboardVals AIIndependentRecord for GPT-6.1 Sol on Vibe Code Bench 1.1 — OpenHands, October 2026 evaluation