Benchmarks

10 results across 6 benchmarks

Benchmark results for Gemini 2.5 Pro
BenchmarkScoreSource
GPQA Diamond86.4%storage.googleapis.com for Gemini 2.5 Pro on Graduate-Level Google-Proof Q&A Diamond (opens in a new tab)GPQA Diamond AccuracyReported Technical reportGoogleSelf-reportedGoogle DeepMindRecord for Gemini 2.5 Pro on Graduate-Level Google-Proof Q&A Diamond
GPQA Diamond — Epoch AI harnessConfiguration: Epoch AI harness83.8%epoch.ai for Gemini 2.5 Pro on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness (opens in a new tab)Archived copy (opens in a new tab)GPQA Diamond AccuracyReported Evaluated Benchmark leaderboardEpoch AIIndependentRecord for Gemini 2.5 Pro on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness
GPQA Diamond — Vals AI harnessConfiguration: Vals AI harness: mean of zero-shot and few-shot CoT80.8%vals.ai for Gemini 2.5 Pro on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)GPQA Diamond AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Gemini 2.5 Pro on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness
Humanity's Last Exam Full set — Scale AI evaluation18.2%labs.scale.com for Gemini 2.5 Pro on Humanity's Last Exam Full set — Scale AI evaluation (opens in a new tab)Archived copy (opens in a new tab)Humanity's Last Exam AccuracyReported Benchmark leaderboardScale AIIndependentRecord for Gemini 2.5 Pro on Humanity's Last Exam Full set — Scale AI evaluation
Humanity's Last Exam Text only — Scale AI evaluation18.4%labs.scale.com for Gemini 2.5 Pro on Humanity's Last Exam Text only — Scale AI evaluation (opens in a new tab)Archived copy (opens in a new tab)Humanity's Last Exam AccuracyReported Benchmark leaderboardScale AIIndependentRecord for Gemini 2.5 Pro on Humanity's Last Exam Text only — Scale AI evaluation
MMMU82.0%storage.googleapis.com for Gemini 2.5 Pro on Massive Multi-discipline Multimodal Understanding MMMU (opens in a new tab)MMMU AccuracyReported Technical reportGoogleSelf-reportedGoogle DeepMindRecord for Gemini 2.5 Pro on Massive Multi-discipline Multimodal Understanding MMMU
MMLU-Pro Original — Vals AI harnessConfiguration: Vals AI harness84.1%vals.ai for Gemini 2.5 Pro on MMLU-Pro Original — Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Gemini 2.5 Pro on MMLU-Pro Original — Vals AI harness
MMMU-Pro Vals AI harnessConfiguration: Vals AI harness81.3%vals.ai for Gemini 2.5 Pro on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for Gemini 2.5 Pro on MMMU-Pro Vals AI harness
SWE-bench Verified59.6%storage.googleapis.com for Gemini 2.5 Pro on SWE-bench Verified (opens in a new tab)Resolved TasksReported Technical reportGoogleSelf-reportedGoogle DeepMindRecord for Gemini 2.5 Pro on SWE-bench Verified
SWE-bench Verified — mini-swe-agentConfiguration: mini-swe-agent harness53.6%github.com for Gemini 2.5 Pro on SWE-bench Verified — mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Resolved TasksReported Benchmark repositorySWE-benchIndependentSWE-bench TeamRecord for Gemini 2.5 Pro on SWE-bench Verified — mini-swe-agent