Benchmarks

20 results across 9 benchmarks

Benchmark results for OpenAI o3
BenchmarkScoreSource
GPQA Diamond — Epoch AI harnessEffort: LowConfiguration: Epoch AI harness79.8%epoch.ai for OpenAI o3 (low) on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness (opens in a new tab)Archived copy (opens in a new tab)GPQA Diamond AccuracyReported Evaluated Benchmark leaderboardEpoch AIIndependentRecord for OpenAI o3 on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness
GPQA Diamond — Epoch AI harnessEffort: MediumConfiguration: Epoch AI harness80.8%epoch.ai for OpenAI o3 (medium) on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness (opens in a new tab)Archived copy (opens in a new tab)GPQA Diamond AccuracyReported Evaluated Benchmark leaderboardEpoch AIIndependentRecord for OpenAI o3 on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness
GPQA Diamond — Epoch AI harnessEffort: HighConfiguration: Epoch AI harness81.8%epoch.ai for OpenAI o3 (high) on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness (opens in a new tab)Archived copy (opens in a new tab)GPQA Diamond AccuracyReported Evaluated Benchmark leaderboardEpoch AIIndependentRecord for OpenAI o3 on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness
GPQA Diamond — Vals AI harnessEffort: HighConfiguration: Vals AI harness: mean of zero-shot and few-shot CoT84.1%vals.ai for OpenAI o3 (high) on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)GPQA Diamond AccuracyReported Benchmark leaderboardVals AIIndependentRecord for OpenAI o3 on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness
HealthBenchEffort: High59.8%deploymentsafety.openai.com for OpenAI o3 (high) on HealthBench HealthBench (opens in a new tab)HealthBench ScoreReported Model cardOpenAISelf-reportedRecord for OpenAI o3 on HealthBench HealthBench
HealthBench HealthBench ConsensusEffort: High92.8%deploymentsafety.openai.com for OpenAI o3 (high) on HealthBench HealthBench Consensus (opens in a new tab)HealthBench ScoreReported Model cardOpenAISelf-reportedRecord for OpenAI o3 on HealthBench HealthBench Consensus
HealthBench HealthBench HardEffort: High31.6%deploymentsafety.openai.com for OpenAI o3 (high) on HealthBench HealthBench Hard (opens in a new tab)HealthBench ScoreReported Model cardOpenAISelf-reportedRecord for OpenAI o3 on HealthBench HealthBench Hard
Humanity's Last Exam Full set — Scale AI evaluationEffort: Medium19.2%labs.scale.com for OpenAI o3 (medium) on Humanity's Last Exam Full set — Scale AI evaluation (opens in a new tab)Archived copy (opens in a new tab)Humanity's Last Exam AccuracyReported Benchmark leaderboardScale AIIndependentRecord for OpenAI o3 on Humanity's Last Exam Full set — Scale AI evaluation
Humanity's Last Exam Full set — Scale AI evaluationEffort: High20.3%labs.scale.com for OpenAI o3 (high) on Humanity's Last Exam Full set — Scale AI evaluation (opens in a new tab)Archived copy (opens in a new tab)Humanity's Last Exam AccuracyReported Benchmark leaderboardScale AIIndependentRecord for OpenAI o3 on Humanity's Last Exam Full set — Scale AI evaluation
Humanity's Last Exam Text only — Scale AI evaluationEffort: Medium19.8%labs.scale.com for OpenAI o3 (medium) on Humanity's Last Exam Text only — Scale AI evaluation (opens in a new tab)Archived copy (opens in a new tab)Humanity's Last Exam AccuracyReported Benchmark leaderboardScale AIIndependentRecord for OpenAI o3 on Humanity's Last Exam Text only — Scale AI evaluation
Humanity's Last Exam Text only — Scale AI evaluationEffort: High20.6%labs.scale.com for OpenAI o3 (high) on Humanity's Last Exam Text only — Scale AI evaluation (opens in a new tab)Archived copy (opens in a new tab)Humanity's Last Exam AccuracyReported Benchmark leaderboardScale AIIndependentRecord for OpenAI o3 on Humanity's Last Exam Text only — Scale AI evaluation
MMLU-Pro Original — Vals AI harnessEffort: HighConfiguration: Vals AI harness85.6%vals.ai for OpenAI o3 (high) on MMLU-Pro Original — Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)AccuracyReported Benchmark leaderboardVals AIIndependentRecord for OpenAI o3 on MMLU-Pro Original — Vals AI harness
MMMU-Pro Vals AI harnessEffort: HighConfiguration: Vals AI harness80.4%vals.ai for OpenAI o3 (high) on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for OpenAI o3 on MMMU-Pro Vals AI harness
OSWorld Verified — 100 stepsConfiguration: 100-step limit23.0%os-world.github.io for OpenAI o3 on OSWorld Verified — 100 steps (opens in a new tab)Archived copy (opens in a new tab)OSWorld-Verified success rateReported Benchmark leaderboardOSWorldIndependentXLang LabRecord for OpenAI o3 on OSWorld Verified — 100 steps
OSWorld Verified — 15 stepsConfiguration: 15-step limit9.1%os-world.github.io for OpenAI o3 on OSWorld Verified — 15 steps (opens in a new tab)Archived copy (opens in a new tab)OSWorld-Verified success rateReported Benchmark leaderboardOSWorldIndependentXLang LabRecord for OpenAI o3 on OSWorld Verified — 15 steps
OSWorld Verified — 50 stepsConfiguration: 50-step limit17.2%os-world.github.io for OpenAI o3 on OSWorld Verified — 50 steps (opens in a new tab)Archived copy (opens in a new tab)OSWorld-Verified success rateReported Benchmark leaderboardOSWorldIndependentXLang LabRecord for OpenAI o3 on OSWorld Verified — 50 steps
SimpleQA Verified Original — Epoch AI harnessEffort: HighConfiguration: Epoch AI harness49.4%epoch.ai for OpenAI o3 (high) on SimpleQA Verified Original — Epoch AI harness (opens in a new tab)Archived copy (opens in a new tab)AccuracyReported Evaluated Benchmark leaderboardEpoch AIIndependentRecord for OpenAI o3 on SimpleQA Verified Original — Epoch AI harness
SWE-bench Verified — Epoch AI harnessEffort: MediumConfiguration: Epoch AI harness62.3%epoch.ai for OpenAI o3 (medium) on SWE-bench Verified — Epoch AI harness (opens in a new tab)Archived copy (opens in a new tab)Resolved TasksReported Evaluated Benchmark leaderboardEpoch AIIndependentRecord for OpenAI o3 on SWE-bench Verified — Epoch AI harness
SWE-bench Verified — mini-swe-agentConfiguration: mini-swe-agent harness58.4%github.com for OpenAI o3 on SWE-bench Verified — mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Resolved TasksReported Benchmark repositorySWE-benchIndependentSWE-bench TeamRecord for OpenAI o3 on SWE-bench Verified — mini-swe-agent
Toolathlon Original17.0%toolathlon.xyz for OpenAI o3 on Toolathlon Original (opens in a new tab)Archived copy (opens in a new tab)Tasks completedReported Benchmark leaderboardToolathlonIndependentToolathlon AuthorsRecord for OpenAI o3 on Toolathlon Original