Benchmarks

51 results across 25 benchmarks

Benchmark results for Gemini 3.1 Pro
BenchmarkScoreSource
APEX-Agents OriginalEffort: High35.3%mercor.com for Gemini 3.1 Pro (high) on APEX-Agents Original (opens in a new tab)Archived copy (opens in a new tab)Pass rateReported Benchmark leaderboardMercorIndependentRecord for Gemini 3.1 Pro on APEX-Agents Original
AutomationBench 1.0.6Effort: Low4.6%zapier.com for Gemini 3.1 Pro (low) on AutomationBench 1.0.6 (opens in a new tab)Archived copy (opens in a new tab)Task pass rateReported Benchmark leaderboardZapierIndependentZapier AutomationBench TeamRecord for Gemini 3.1 Pro on AutomationBench 1.0.6
AutomationBench 1.0.6Effort: Medium7.0%zapier.com for Gemini 3.1 Pro (medium) on AutomationBench 1.0.6 (opens in a new tab)Archived copy (opens in a new tab)Task pass rateReported Benchmark leaderboardZapierIndependentZapier AutomationBench TeamRecord for Gemini 3.1 Pro on AutomationBench 1.0.6
AutomationBench 1.0.6Effort: High8.7%zapier.com for Gemini 3.1 Pro (high) on AutomationBench 1.0.6 (opens in a new tab)Archived copy (opens in a new tab)Task pass rateReported Benchmark leaderboardZapierIndependentZapier AutomationBench TeamRecord for Gemini 3.1 Pro on AutomationBench 1.0.6
BrowseComp 130-question subset — Mercor web research agentEffort: HighConfiguration: Mercor web research agent, 100 steps, 3 hours75.6%mercor.com for Gemini 3.1 Pro (high) on BrowseComp 130-question subset — Mercor web research agent (opens in a new tab)Archived copy (opens in a new tab)BrowseComp AccuracyReported Benchmark leaderboardMercorIndependentRecord for Gemini 3.1 Pro on BrowseComp 130-question subset — Mercor web research agent
BrowseCompEffort: High85.9%deepmind.google for Gemini 3.1 Pro (High) on BrowseComp BrowseComp (opens in a new tab)BrowseComp AccuracyReported Model cardGoogle DeepMindSelf-reportedRecord for Gemini 3.1 Pro on BrowseComp BrowseComp
Chartography October 2026 leaderboardEffort: High26.1%surgehq.ai for Gemini 3.1 Pro (High) on Chartography October 2026 leaderboard (opens in a new tab)Archived copy (opens in a new tab)Chartography Pass@1Reported Benchmark leaderboardSurge AIIndependentRecord for Gemini 3.1 Pro on Chartography October 2026 leaderboard
CharXiv Descriptive and reasoning — Mercor single-shotEffort: HighConfiguration: Mercor single-shot agent94.6%mercor.com for Gemini 3.1 Pro (high) on CharXiv Descriptive and reasoning — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)CharXiv AccuracyReported Benchmark leaderboardMercorIndependentRecord for Gemini 3.1 Pro on CharXiv Descriptive and reasoning — Mercor single-shot
DeepSWE 1.1 — Mercor mini-swe-agent, 500 stepsEffort: HighConfiguration: Mercor mini-swe-agent, 500 steps, 2 hours7.7%mercor.com for Gemini 3.1 Pro (high) on DeepSWE 1.1 — Mercor mini-swe-agent, 500 steps (opens in a new tab)Archived copy (opens in a new tab)DeepSWE Pass@1Reported Benchmark leaderboardMercorIndependentRecord for Gemini 3.1 Pro on DeepSWE 1.1 — Mercor mini-swe-agent, 500 steps
Finance Agent Benchmark 2 — October 2026 evaluationEffort: High43.0%vals.ai for Gemini 3.1 Pro (high) on Finance Agent Benchmark 2 — October 2026 evaluation (opens in a new tab)Archived copy (opens in a new tab)Finance Agent AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Gemini 3.1 Pro on Finance Agent Benchmark 2 — October 2026 evaluation
GPQA DiamondEffort: High94.3%deepmind.google for Gemini 3.1 Pro (High) on Graduate-Level Google-Proof Q&A Diamond (opens in a new tab)GPQA Diamond AccuracyReported Model cardGoogle DeepMindSelf-reportedRecord for Gemini 3.1 Pro on Graduate-Level Google-Proof Q&A Diamond
GPQA Diamond — Epoch AI harnessEffort: HighConfiguration: Epoch AI harness94.4%epoch.ai for Gemini 3.1 Pro (high) on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness (opens in a new tab)Archived copy (opens in a new tab)GPQA Diamond AccuracyReported Evaluated Benchmark leaderboardEpoch AIIndependentRecord for Gemini 3.1 Pro on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness
GPQA Diamond — Epoch AI harnessConfiguration: Epoch AI harness94.1%epoch.ai for Gemini 3.1 Pro on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness (opens in a new tab)Archived copy (opens in a new tab)GPQA Diamond AccuracyReported Evaluated Benchmark leaderboardEpoch AIIndependentRecord for Gemini 3.1 Pro on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness
GPQA Diamond — Mercor single-shotEffort: HighConfiguration: Mercor single-shot agent94.2%mercor.com for Gemini 3.1 Pro (high) on Graduate-Level Google-Proof Q&A Diamond — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)GPQA Diamond AccuracyReported Benchmark leaderboardMercorIndependentRecord for Gemini 3.1 Pro on Graduate-Level Google-Proof Q&A Diamond — Mercor single-shot
GPQA Diamond — Vals AI harnessEffort: HighConfiguration: Vals AI harness: mean of zero-shot and few-shot CoT95.5%vals.ai for Gemini 3.1 Pro (high) on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)GPQA Diamond AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Gemini 3.1 Pro on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness
HLAB October 2026 evaluationEffort: High0.0%vals.ai for Gemini 3.1 Pro (high) on Harvey's Legal Agent Benchmark October 2026 evaluation (opens in a new tab)Archived copy (opens in a new tab)HLAB Task Pass RateReported Benchmark leaderboardVals AIIndependentRecord for Gemini 3.1 Pro on Harvey's Legal Agent Benchmark October 2026 evaluation
Humanity's Last Exam Full set — No toolsEffort: HighConfiguration: No tools44.4%deepmind.google for Gemini 3.1 Pro (High) on Humanity's Last Exam Full set — No tools (opens in a new tab)Humanity's Last Exam AccuracyReported Model cardGoogle DeepMindSelf-reportedRecord for Gemini 3.1 Pro on Humanity's Last Exam Full set — No tools
Humanity's Last Exam Full set — Scale AI evaluationEffort: HighReported as: thinking high46.4%labs.scale.com for Gemini 3.1 Pro (thinking high) on Humanity's Last Exam Full set — Scale AI evaluation (opens in a new tab)Archived copy (opens in a new tab)Humanity's Last Exam AccuracyReported Benchmark leaderboardScale AIIndependentRecord for Gemini 3.1 Pro on Humanity's Last Exam Full set — Scale AI evaluation
Humanity's Last Exam Text only — Mercor single-shotEffort: HighConfiguration: Mercor single-shot agent44.1%mercor.com for Gemini 3.1 Pro (high) on Humanity's Last Exam Text only — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)Humanity's Last Exam AccuracyReported Benchmark leaderboardMercorIndependentRecord for Gemini 3.1 Pro on Humanity's Last Exam Text only — Mercor single-shot
Humanity's Last Exam Text only — Scale AI evaluationEffort: HighReported as: thinking high47.3%labs.scale.com for Gemini 3.1 Pro (thinking high) on Humanity's Last Exam Text only — Scale AI evaluation (opens in a new tab)Archived copy (opens in a new tab)Humanity's Last Exam AccuracyReported Benchmark leaderboardScale AIIndependentRecord for Gemini 3.1 Pro on Humanity's Last Exam Text only — Scale AI evaluation
MCP Atlas Public, April 2026 updateEffort: High78.2%labs.scale.com for Gemini 3.1 Pro (high) on MCP Atlas Public, April 2026 update (opens in a new tab)Pass rateReported Benchmark leaderboardScale AIIndependentRecord for Gemini 3.1 Pro on MCP Atlas Public, April 2026 update
MedXpertQA MM — Mercor single-shotEffort: HighConfiguration: Mercor single-shot agent80.2%mercor.com for Gemini 3.1 Pro (high) on MedXpertQA MM — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)AccuracyReported Benchmark leaderboardMercorIndependentRecord for Gemini 3.1 Pro on MedXpertQA MM — Mercor single-shot
MMLU-Pro Original — Mercor single-shotEffort: HighConfiguration: Mercor single-shot agent91.3%mercor.com for Gemini 3.1 Pro (high) on MMLU-Pro Original — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)AccuracyReported Benchmark leaderboardMercorIndependentRecord for Gemini 3.1 Pro on MMLU-Pro Original — Mercor single-shot
MMLU-Pro Original — Vals AI harnessEffort: HighConfiguration: Vals AI harness91.0%vals.ai for Gemini 3.1 Pro (high) on MMLU-Pro Original — Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Gemini 3.1 Pro on MMLU-Pro Original — Vals AI harness
MMMU-Pro 1,730 questions — Mercor single-shotEffort: HighConfiguration: Mercor single-shot agent84.3%mercor.com for Gemini 3.1 Pro (high) on MMMU-Pro 1,730 questions — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Not statedReported Benchmark leaderboardMercorIndependentRecord for Gemini 3.1 Pro on MMMU-Pro 1,730 questions — Mercor single-shot
MMMU-Pro No toolsEffort: HighConfiguration: No tools80.5%deepmind.google for Gemini 3.1 Pro (High) on MMMU-Pro No tools (opens in a new tab)MMMU-Pro accuracySetting: Overall (Standard 10 options + Vision)Reported Model cardGoogle DeepMindSelf-reportedRecord for Gemini 3.1 Pro on MMMU-Pro No tools
MMMU-Pro Vals AI harnessEffort: HighConfiguration: Vals AI harness88.2%vals.ai for Gemini 3.1 Pro (high) on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for Gemini 3.1 Pro on MMMU-Pro Vals AI harness
PostTrainBench 1.1 — OpenCodeConfiguration: OpenCode harness22.0%posttrainbench.com for Gemini 3.1 Pro on PostTrainBench 1.1 — OpenCode (opens in a new tab)Archived copy (opens in a new tab)PostTrainBench Weighted AggregateReported Benchmark leaderboardPostTrainBenchIndependentPostTrainBench TeamRecord for Gemini 3.1 Pro on PostTrainBench 1.1 — OpenCode
PostTrainBench 1.2 — OpenCodeConfiguration: OpenCode harness18.1%posttrainbench.com for Gemini 3.1 Pro on PostTrainBench 1.2 — OpenCode (opens in a new tab)Archived copy (opens in a new tab)PostTrainBench Weighted AggregateReported Benchmark leaderboardPostTrainBenchIndependentPostTrainBench TeamRecord for Gemini 3.1 Pro on PostTrainBench 1.2 — OpenCode
Riemann-bench October 2026 leaderboardEffort: High33.6%surgehq.ai for Gemini 3.1 Pro (High) on Riemann-bench October 2026 leaderboard (opens in a new tab)Archived copy (opens in a new tab)Riemann-bench Pass@1Reported Benchmark leaderboardSurge AIIndependentRecord for Gemini 3.1 Pro on Riemann-bench October 2026 leaderboard
SimpleQA Verified Original — Epoch AI harnessEffort: HighConfiguration: Epoch AI harness73.5%epoch.ai for Gemini 3.1 Pro (high) on SimpleQA Verified Original — Epoch AI harness (opens in a new tab)Archived copy (opens in a new tab)AccuracyReported Evaluated Benchmark leaderboardEpoch AIIndependentRecord for Gemini 3.1 Pro on SimpleQA Verified Original — Epoch AI harness
SWE Atlas Codebase QnA — Mercor mini-swe-agent, 250 stepsEffort: HighConfiguration: Mercor mini-swe-agent, 250 steps, 1 hour18.0%mercor.com for Gemini 3.1 Pro (high) on SWE Atlas Codebase QnA — Mercor mini-swe-agent, 250 steps (opens in a new tab)Archived copy (opens in a new tab)Mean pass@1Reported Benchmark leaderboardMercorIndependentRecord for Gemini 3.1 Pro on SWE Atlas Codebase QnA — Mercor mini-swe-agent, 250 steps
SWE Atlas Codebase QnA — Public; mini-swe-agentConfiguration: mini-swe-agent harness13.5%labs.scale.com for Gemini 3.1 Pro on SWE Atlas Codebase QnA — Public; mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Mean pass@1Reported Benchmark leaderboardScale AIIndependentRecord for Gemini 3.1 Pro on SWE Atlas Codebase QnA — Public; mini-swe-agent
SWE Atlas Refactoring — Gemini CLIConfiguration: Gemini CLI harness33.8%labs.scale.com for Gemini 3.1 Pro on SWE Atlas Refactoring — Gemini CLI (opens in a new tab)Archived copy (opens in a new tab)Mean pass@1Reported Benchmark leaderboardScale AIIndependentRecord for Gemini 3.1 Pro on SWE Atlas Refactoring — Gemini CLI
SWE-bench Multilingual — Mercor mini-swe-agent, 1000 stepsEffort: HighConfiguration: Mercor mini-swe-agent, 1000 steps, 3 hours65.1%mercor.com for Gemini 3.1 Pro (high) on SWE-bench Multilingual — Mercor mini-swe-agent, 1000 steps (opens in a new tab)Archived copy (opens in a new tab)Resolved TasksReported Benchmark leaderboardMercorIndependentRecord for Gemini 3.1 Pro on SWE-bench Multilingual — Mercor mini-swe-agent, 1000 steps
SWE-bench VerifiedEffort: High80.6%deepmind.google for Gemini 3.1 Pro (High) on SWE-bench Verified (opens in a new tab)Resolved TasksReported Model cardGoogle DeepMindSelf-reportedRecord for Gemini 3.1 Pro on SWE-bench Verified
SWE-bench Verified — Mercor mini-swe-agent, 1000 stepsEffort: HighConfiguration: Mercor mini-swe-agent, 1000 steps, 3 hours78.4%mercor.com for Gemini 3.1 Pro (high) on SWE-bench Verified — Mercor mini-swe-agent, 1000 steps (opens in a new tab)Archived copy (opens in a new tab)Resolved TasksReported Benchmark leaderboardMercorIndependentRecord for Gemini 3.1 Pro on SWE-bench Verified — Mercor mini-swe-agent, 1000 steps
SWE-bench Verified — Vals AI mini-swe-agentEffort: HighConfiguration: mini-swe-agent harness78.8%vals.ai for Gemini 3.1 Pro (high) on SWE-bench Verified — Vals AI mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Resolved TasksReported Benchmark leaderboardVals AIIndependentRecord for Gemini 3.1 Pro on SWE-bench Verified — Vals AI mini-swe-agent
SWE-bench Pro Private — Scale AI mini-swe-agentReported as: thinkingConfiguration: mini-swe-agent harness32.2%labs.scale.com for Gemini 3.1 Pro (thinking) on SWE-bench Pro Private — Scale AI mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Public tasks resolvedReported Benchmark leaderboardScale AIIndependentRecord for Gemini 3.1 Pro on SWE-bench Pro Private — Scale AI mini-swe-agent
SWE-bench Pro PublicEffort: High54.2%deepmind.google for Gemini 3.1 Pro (High) on SWE-bench Pro Public (opens in a new tab)Public tasks resolvedReported Model cardGoogle DeepMindSelf-reportedRecord for Gemini 3.1 Pro on SWE-bench Pro Public
SWE-bench Pro Public — Scale AI mini-swe-agentReported as: thinkingConfiguration: mini-swe-agent harness46.1%labs.scale.com for Gemini 3.1 Pro (thinking) on SWE-bench Pro Public — Scale AI mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Public tasks resolvedReported Benchmark leaderboardScale AIIndependentRecord for Gemini 3.1 Pro on SWE-bench Pro Public — Scale AI mini-swe-agent
Terminal-Bench 2.0Effort: High68.5%deepmind.google for Gemini 3.1 Pro (High) on Terminal-Bench 2.0 (opens in a new tab)Terminal-Bench AccuracyReported Model cardGoogle DeepMindSelf-reportedRecord for Gemini 3.1 Pro on Terminal-Bench 2.0
Terminal-Bench 2.0 — Vals AI Terminus 2Effort: HighConfiguration: Terminus-2 harness67.4%vals.ai for Gemini 3.1 Pro (high) on Terminal-Bench 2.0 — Vals AI Terminus 2 (opens in a new tab)Archived copy (opens in a new tab)Terminal-Bench AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Gemini 3.1 Pro on Terminal-Bench 2.0 — Vals AI Terminus 2
Terminal-Bench 2.1 — Mercor Terminus 2Effort: HighConfiguration: Mercor Terminus 2, 1000 steps, 3 hours72.7%mercor.com for Gemini 3.1 Pro (high) on Terminal-Bench 2.1 — Mercor Terminus 2 (opens in a new tab)Archived copy (opens in a new tab)Terminal-Bench AccuracyReported Benchmark leaderboardMercorIndependentRecord for Gemini 3.1 Pro on Terminal-Bench 2.1 — Mercor Terminus 2
Terminal-Bench 2.1 — Vals AI Terminus 2Effort: HighConfiguration: Terminus-2 harness70.8%vals.ai for Gemini 3.1 Pro (high) on Terminal-Bench 2.1 — Vals AI Terminus 2 (opens in a new tab)Archived copy (opens in a new tab)Terminal-Bench AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Gemini 3.1 Pro on Terminal-Bench 2.1 — Vals AI Terminus 2
Terminal-Bench 4.0 — Vals AI mini-swe-agentEffort: HighConfiguration: mini-swe-agent harness2.5%vals.ai for Gemini 3.1 Pro (high) on Terminal-Bench 4.0 — Vals AI mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Terminal-Bench AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Gemini 3.1 Pro on Terminal-Bench 4.0 — Vals AI mini-swe-agent
Toolathlon Original48.8%toolathlon.xyz for Gemini 3.1 Pro on Toolathlon Original (opens in a new tab)Archived copy (opens in a new tab)Tasks completedReported Benchmark leaderboardToolathlonIndependentToolathlon AuthorsRecord for Gemini 3.1 Pro on Toolathlon Original
Toolathlon VerifiedEffort: High61.1%toolathlon.xyz for Gemini 3.1 Pro (high) on Toolathlon Verified (opens in a new tab)Archived copy (opens in a new tab)Tasks completedReported Benchmark leaderboardToolathlonIndependentToolathlon AuthorsRecord for Gemini 3.1 Pro on Toolathlon Verified
Vals Index 2.1Effort: High33.4%vals.ai for Gemini 3.1 Pro (high) on Vals Index 2.1 (opens in a new tab)Archived copy (opens in a new tab)Vals Index GDP-weighted AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Gemini 3.1 Pro on Vals Index 2.1
Vibe Code Bench 1.1 — OpenHands, October 2026 evaluationEffort: HighConfiguration: OpenHands harness32.0%vals.ai for Gemini 3.1 Pro (high) on Vibe Code Bench 1.1 — OpenHands, October 2026 evaluation (opens in a new tab)Archived copy (opens in a new tab)Vibe Code Bench ScoreReported Benchmark leaderboardVals AIIndependentRecord for Gemini 3.1 Pro on Vibe Code Bench 1.1 — OpenHands, October 2026 evaluation