Benchmarks

44 results across 25 benchmarks

Benchmark results for Gemini 3.8 Flash
BenchmarkScoreSource
APEX-Agents OriginalEffort: High64.3%mercor.com for Gemini 3.8 Flash (high) on APEX-Agents Original (opens in a new tab)Archived copy (opens in a new tab)Pass rateReported Benchmark leaderboardMercorIndependentRecord for Gemini 3.8 Flash on APEX-Agents Original
AutomationBench 1.0.6Effort: Medium29.7%zapier.com for Gemini 3.8 Flash (medium) on AutomationBench 1.0.6 (opens in a new tab)Archived copy (opens in a new tab)Task pass rateReported Benchmark leaderboardZapierIndependentZapier AutomationBench TeamRecord for Gemini 3.8 Flash on AutomationBench 1.0.6
AutomationBench 1.0.6Effort: High29.7%zapier.com for Gemini 3.8 Flash (high) on AutomationBench 1.0.6 (opens in a new tab)Archived copy (opens in a new tab)Task pass rateReported Benchmark leaderboardZapierIndependentZapier AutomationBench TeamRecord for Gemini 3.8 Flash on AutomationBench 1.0.6
BrowseComp 130-question subset — Mercor web research agentEffort: HighConfiguration: Mercor web research agent, 100 steps, 3 hours83.1%mercor.com for Gemini 3.8 Flash (high) on BrowseComp 130-question subset — Mercor web research agent (opens in a new tab)Archived copy (opens in a new tab)BrowseComp AccuracyReported Benchmark leaderboardMercorIndependentRecord for Gemini 3.8 Flash on BrowseComp 130-question subset — Mercor web research agent
Chartography October 2026 leaderboardEffort: High40.9%surgehq.ai for Gemini 3.8 Flash (High) on Chartography October 2026 leaderboard (opens in a new tab)Archived copy (opens in a new tab)Chartography Pass@1Reported Benchmark leaderboardSurge AIIndependentRecord for Gemini 3.8 Flash on Chartography October 2026 leaderboard
CharXiv Descriptive and reasoning — Mercor single-shotEffort: HighConfiguration: Mercor single-shot agent95.4%mercor.com for Gemini 3.8 Flash (high) on CharXiv Descriptive and reasoning — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)CharXiv AccuracyReported Benchmark leaderboardMercorIndependentRecord for Gemini 3.8 Flash on CharXiv Descriptive and reasoning — Mercor single-shot
CharXiv Reasoning — No toolsReported as: no toolsConfiguration: No tools86.2%deepmind.google for Gemini 3.8 Flash (no tools) on CharXiv Reasoning — No tools (opens in a new tab)CharXiv AccuracyReported Model cardGoogle DeepMindSelf-reportedRecord for Gemini 3.8 Flash on CharXiv Reasoning — No tools
CursorBench 4.0Effort: Medium37.3%cursor.com for Gemini 3.8 Flash (medium) on CursorBench 4.0 (opens in a new tab)Archived copy (opens in a new tab)CursorBench AccuracyReported Benchmark leaderboardCursorIndependentRecord for Gemini 3.8 Flash on CursorBench 4.0
CursorBench 4.0Effort: High39.6%cursor.com for Gemini 3.8 Flash (high) on CursorBench 4.0 (opens in a new tab)Archived copy (opens in a new tab)CursorBench AccuracyReported Benchmark leaderboardCursorIndependentRecord for Gemini 3.8 Flash on CursorBench 4.0
DeepSWE 1.1Effort: High73.7%deepmind.google for Gemini 3.8 Flash (high) on DeepSWE 1.1 (opens in a new tab)DeepSWE Pass@1Reported Model cardGoogle DeepMindSelf-reportedRecord for Gemini 3.8 Flash on DeepSWE 1.1
DeepSWE 1.1 — Mercor mini-swe-agent, 500 stepsEffort: HighConfiguration: Mercor mini-swe-agent, 500 steps, 2 hours71.4%mercor.com for Gemini 3.8 Flash (high) on DeepSWE 1.1 — Mercor mini-swe-agent, 500 steps (opens in a new tab)Archived copy (opens in a new tab)DeepSWE Pass@1Reported Benchmark leaderboardMercorIndependentRecord for Gemini 3.8 Flash on DeepSWE 1.1 — Mercor mini-swe-agent, 500 steps
Finance Agent Benchmark 2 — October 2026 evaluationEffort: High61.4%vals.ai for Gemini 3.8 Flash (high) on Finance Agent Benchmark 2 — October 2026 evaluation (opens in a new tab)Archived copy (opens in a new tab)Finance Agent AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Gemini 3.8 Flash on Finance Agent Benchmark 2 — October 2026 evaluation
FrontierSWE 2 — October 2026 evaluationConfiguration: Proximus harness, 5 trials, 20-hour budget19.6%frontierswe.com for Gemini 3.8 Flash on FrontierSWE 2 — October 2026 evaluation (opens in a new tab)Archived copy (opens in a new tab)FrontierSWE ScoreReported Benchmark leaderboardProximalIndependentRecord for Gemini 3.8 Flash on FrontierSWE 2 — October 2026 evaluation
GDP.pdf35.0%deepmind.google for Gemini 3.8 Flash on GDP.pdf GDP.pdf (opens in a new tab)GDP.pdf All Pass RateReported Model cardGoogle DeepMindSelf-reportedRecord for Gemini 3.8 Flash on GDP.pdf GDP.pdf
GPQA Diamond — Epoch AI harnessEffort: HighConfiguration: Epoch AI harness95.4%epoch.ai for Gemini 3.8 Flash (high) on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness (opens in a new tab)Archived copy (opens in a new tab)GPQA Diamond AccuracyReported Evaluated Benchmark leaderboardEpoch AIIndependentRecord for Gemini 3.8 Flash on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness
GPQA Diamond — Mercor single-shotEffort: HighConfiguration: Mercor single-shot agent93.9%mercor.com for Gemini 3.8 Flash (high) on Graduate-Level Google-Proof Q&A Diamond — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)GPQA Diamond AccuracyReported Benchmark leaderboardMercorIndependentRecord for Gemini 3.8 Flash on Graduate-Level Google-Proof Q&A Diamond — Mercor single-shot
GPQA Diamond — Vals AI harnessEffort: HighConfiguration: Vals AI harness: mean of zero-shot and few-shot CoT94.4%vals.ai for Gemini 3.8 Flash (high) on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)GPQA Diamond AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Gemini 3.8 Flash on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness
HLAB October 2026 evaluationEffort: High10.0%vals.ai for Gemini 3.8 Flash (high) on Harvey's Legal Agent Benchmark October 2026 evaluation (opens in a new tab)Archived copy (opens in a new tab)HLAB Task Pass RateReported Benchmark leaderboardVals AIIndependentRecord for Gemini 3.8 Flash on Harvey's Legal Agent Benchmark October 2026 evaluation
HLE-Verified Full 1,811-item verified set54.9%deepmind.google for Gemini 3.8 Flash on HLE-Verified Full 1,811-item verified set (opens in a new tab)HLE-Verified AccuracyReported Model cardGoogle DeepMindSelf-reportedRecord for Gemini 3.8 Flash on HLE-Verified Full 1,811-item verified set
Humanity's Last Exam Full set — Scale AI evaluation44.5%labs.scale.com for Gemini 3.8 Flash on Humanity's Last Exam Full set — Scale AI evaluation (opens in a new tab)Archived copy (opens in a new tab)Humanity's Last Exam AccuracyReported Benchmark leaderboardScale AIIndependentRecord for Gemini 3.8 Flash on Humanity's Last Exam Full set — Scale AI evaluation
Humanity's Last Exam Text only — Mercor single-shotEffort: HighConfiguration: Mercor single-shot agent47.0%mercor.com for Gemini 3.8 Flash (high) on Humanity's Last Exam Text only — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)Humanity's Last Exam AccuracyReported Benchmark leaderboardMercorIndependentRecord for Gemini 3.8 Flash on Humanity's Last Exam Text only — Mercor single-shot
Humanity's Last Exam Text only — Scale AI evaluation45.0%labs.scale.com for Gemini 3.8 Flash on Humanity's Last Exam Text only — Scale AI evaluation (opens in a new tab)Archived copy (opens in a new tab)Humanity's Last Exam AccuracyReported Benchmark leaderboardScale AIIndependentRecord for Gemini 3.8 Flash on Humanity's Last Exam Text only — Scale AI evaluation
LVBench 1024 frames — Agentic87.8%deepmind.google for Gemini 3.8 Flash on LVBench 1024 frames — Agentic (opens in a new tab)LVBench AccuracyReported Model cardGoogle DeepMindSelf-reportedRecord for Gemini 3.8 Flash on LVBench 1024 frames — Agentic
LVBench 1024 frames — Static87.1%deepmind.google for Gemini 3.8 Flash on LVBench 1024 frames — Static (opens in a new tab)LVBench AccuracyReported Model cardGoogle DeepMindSelf-reportedRecord for Gemini 3.8 Flash on LVBench 1024 frames — Static
MedXpertQA MM — Mercor single-shotEffort: HighConfiguration: Mercor single-shot agent84.7%mercor.com for Gemini 3.8 Flash (high) on MedXpertQA MM — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)AccuracyReported Benchmark leaderboardMercorIndependentRecord for Gemini 3.8 Flash on MedXpertQA MM — Mercor single-shot
MMLU-Pro Original — Mercor single-shotEffort: HighConfiguration: Mercor single-shot agent90.7%mercor.com for Gemini 3.8 Flash (high) on MMLU-Pro Original — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)AccuracyReported Benchmark leaderboardMercorIndependentRecord for Gemini 3.8 Flash on MMLU-Pro Original — Mercor single-shot
MMLU-Pro Original — Vals AI harnessEffort: HighConfiguration: Vals AI harness90.2%vals.ai for Gemini 3.8 Flash (high) on MMLU-Pro Original — Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Gemini 3.8 Flash on MMLU-Pro Original — Vals AI harness
MMMU-Pro 1,730 questions — Mercor single-shotEffort: HighConfiguration: Mercor single-shot agent85.3%mercor.com for Gemini 3.8 Flash (high) on MMMU-Pro 1,730 questions — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Not statedReported Benchmark leaderboardMercorIndependentRecord for Gemini 3.8 Flash on MMMU-Pro 1,730 questions — Mercor single-shot
MMMU-Pro Vals AI harnessEffort: HighConfiguration: Vals AI harness89.1%vals.ai for Gemini 3.8 Flash (high) on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for Gemini 3.8 Flash on MMMU-Pro Vals AI harness
Riemann-bench October 2026 leaderboardEffort: High51.2%surgehq.ai for Gemini 3.8 Flash (High) on Riemann-bench October 2026 leaderboard (opens in a new tab)Archived copy (opens in a new tab)Riemann-bench Pass@1Reported Benchmark leaderboardSurge AIIndependentRecord for Gemini 3.8 Flash on Riemann-bench October 2026 leaderboard
SimpleQA Verified Original — Epoch AI harnessEffort: HighConfiguration: Epoch AI harness69.7%epoch.ai for Gemini 3.8 Flash (high) on SimpleQA Verified Original — Epoch AI harness (opens in a new tab)Archived copy (opens in a new tab)AccuracyReported Evaluated Benchmark leaderboardEpoch AIIndependentRecord for Gemini 3.8 Flash on SimpleQA Verified Original — Epoch AI harness
SWE Atlas Codebase QnA — Public; mini-swe-agentConfiguration: mini-swe-agent harness47.0%labs.scale.com for Gemini 3.8 Flash on SWE Atlas Codebase QnA — Public; mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Mean pass@1Reported Benchmark leaderboardScale AIIndependentRecord for Gemini 3.8 Flash on SWE Atlas Codebase QnA — Public; mini-swe-agent
SWE Atlas Refactoring — mini-swe-agentConfiguration: mini-swe-agent harness44.8%labs.scale.com for Gemini 3.8 Flash on SWE Atlas Refactoring — mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Mean pass@1Reported Benchmark leaderboardScale AIIndependentRecord for Gemini 3.8 Flash on SWE Atlas Refactoring — mini-swe-agent
SWE-bench Multilingual — Mercor mini-swe-agent, 1000 stepsEffort: HighConfiguration: Mercor mini-swe-agent, 1000 steps, 3 hours85.8%mercor.com for Gemini 3.8 Flash (high) on SWE-bench Multilingual — Mercor mini-swe-agent, 1000 steps (opens in a new tab)Archived copy (opens in a new tab)Resolved TasksReported Benchmark leaderboardMercorIndependentRecord for Gemini 3.8 Flash on SWE-bench Multilingual — Mercor mini-swe-agent, 1000 steps
SWE-bench Verified — Mercor mini-swe-agent, 1000 stepsEffort: HighConfiguration: Mercor mini-swe-agent, 1000 steps, 3 hours80.6%mercor.com for Gemini 3.8 Flash (high) on SWE-bench Verified — Mercor mini-swe-agent, 1000 steps (opens in a new tab)Archived copy (opens in a new tab)Resolved TasksReported Benchmark leaderboardMercorIndependentRecord for Gemini 3.8 Flash on SWE-bench Verified — Mercor mini-swe-agent, 1000 steps
SWE-bench Verified — Vals AI mini-swe-agentEffort: HighConfiguration: mini-swe-agent harness80.0%vals.ai for Gemini 3.8 Flash (high) on SWE-bench Verified — Vals AI mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Resolved TasksReported Benchmark leaderboardVals AIIndependentRecord for Gemini 3.8 Flash on SWE-bench Verified — Vals AI mini-swe-agent
Terminal-Bench 2.189.4%deepmind.google for Gemini 3.8 Flash on Terminal-Bench 2.1 (opens in a new tab)Terminal-Bench AccuracyReported Model cardGoogle DeepMindSelf-reportedRecord for Gemini 3.8 Flash on Terminal-Bench 2.1
Terminal-Bench 2.1 — Vals AI Terminus 2Effort: HighConfiguration: Terminus-2 harness81.3%vals.ai for Gemini 3.8 Flash (high) on Terminal-Bench 2.1 — Vals AI Terminus 2 (opens in a new tab)Archived copy (opens in a new tab)Terminal-Bench AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Gemini 3.8 Flash on Terminal-Bench 2.1 — Vals AI Terminus 2
Terminal-Bench 4.019.1%deepmind.google for Gemini 3.8 Flash on Terminal-Bench 4.0 (opens in a new tab)Terminal-Bench AccuracyReported Model cardGoogle DeepMindSelf-reportedTerminal-Bench TeamRecord for Gemini 3.8 Flash on Terminal-Bench 4.0
Terminal-Bench 4.0 — Mercor mini-swe-agentEffort: HighConfiguration: Mercor mini-swe-agent, 1000 steps, 8 hours12.1%mercor.com for Gemini 3.8 Flash (high) on Terminal-Bench 4.0 — Mercor mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Terminal-Bench AccuracyReported Benchmark leaderboardMercorIndependentRecord for Gemini 3.8 Flash on Terminal-Bench 4.0 — Mercor mini-swe-agent
Terminal-Bench 4.0 — mini-swe-agentEffort: HighConfiguration: mini-swe-agent harness19.1%tbench.ai for Gemini 3.8 Flash (high) on Terminal-Bench 4.0 — mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Terminal-Bench AccuracyReported Benchmark leaderboardTerminal-BenchIndependentTerminal-Bench TeamRecord for Gemini 3.8 Flash on Terminal-Bench 4.0 — mini-swe-agent
Terminal-Bench 4.0 — Vals AI mini-swe-agentEffort: HighConfiguration: mini-swe-agent harness19.2%vals.ai for Gemini 3.8 Flash (high) on Terminal-Bench 4.0 — Vals AI mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Terminal-Bench AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Gemini 3.8 Flash on Terminal-Bench 4.0 — Vals AI mini-swe-agent
Vals Index 2.1Effort: High54.8%vals.ai for Gemini 3.8 Flash (high) on Vals Index 2.1 (opens in a new tab)Archived copy (opens in a new tab)Vals Index GDP-weighted AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Gemini 3.8 Flash on Vals Index 2.1
Vibe Code Bench 1.1 — OpenHands, October 2026 evaluationEffort: HighConfiguration: OpenHands harness78.7%vals.ai for Gemini 3.8 Flash (high) on Vibe Code Bench 1.1 — OpenHands, October 2026 evaluation (opens in a new tab)Archived copy (opens in a new tab)Vibe Code Bench ScoreReported Benchmark leaderboardVals AIIndependentRecord for Gemini 3.8 Flash on Vibe Code Bench 1.1 — OpenHands, October 2026 evaluation