Shared benchmarks

34
Shared benchmarks for Gemini 3.7 Flash and Gemini 3.8 Flash
BenchmarkGemini 3.7 FlashGemini 3.8 Flash
  • 67.8%
  • 64.3%
Evaluation details for APEX-Agents Original

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Gemini 3.7 Flash

Record for Gemini 3.7 Flash on APEX-Agents Original

Gemini 3.8 Flash

Record for Gemini 3.8 Flash on APEX-Agents Original
  • 30.4%
  • 29.7%medium
  • 29.7%high
Evaluation details for AutomationBench 1.0.6

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Gemini 3.7 Flash

Record for Gemini 3.7 Flash on AutomationBench 1.0.6

Gemini 3.8 Flash

Record for Gemini 3.8 Flash on AutomationBench 1.0.6
Record for Gemini 3.8 Flash on AutomationBench 1.0.6
  • 79.2%
  • 83.1%
Evaluation details for BrowseComp 130-question subset — Mercor web research agent

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Gemini 3.7 Flash

Gemini 3.8 Flash

  • 40.4%
  • 40.9%
Evaluation details for Chartography October 2026 leaderboard

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Gemini 3.7 Flash

Record for Gemini 3.7 Flash on Chartography October 2026 leaderboard

Gemini 3.8 Flash

Record for Gemini 3.8 Flash on Chartography October 2026 leaderboard
  • 95.3%
  • 95.4%
Evaluation details for CharXiv Descriptive and reasoning — Mercor single-shot

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

  • 84.5%
  • 86.2%
Evaluation details for CharXiv Reasoning — No tools

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Gemini 3.7 Flash

Record for Gemini 3.7 Flash on CharXiv Reasoning — No tools

Gemini 3.8 Flash

Record for Gemini 3.8 Flash on CharXiv Reasoning — No tools
DeepSWE 1.1Potentially non-equivalent: Evaluator sets differ.
  • 65.0%medium · 1.1 · DataCurve
  • 65.3%high · 1.1 · Google DeepMind
  • 73.7%1.1 · Google DeepMind
Potentially non-equivalent · Evaluation details for DeepSWE 1.1

Evaluator sets differ. Evaluation methodology is not recorded by the registry.

Gemini 3.7 Flash

Record for Gemini 3.7 Flash on DeepSWE 1.1
Record for Gemini 3.7 Flash on DeepSWE 1.1

Gemini 3.8 Flash

Record for Gemini 3.8 Flash on DeepSWE 1.1
  • 65.5%
  • 71.4%
Evaluation details for DeepSWE 1.1 — Mercor mini-swe-agent, 500 steps

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Gemini 3.7 Flash

Gemini 3.8 Flash

  • 59.0%
  • 61.4%
Evaluation details for Finance Agent Benchmark 2 — October 2026 evaluation

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

  • 20.3%
  • 19.6%
Evaluation details for FrontierSWE 2 — October 2026 evaluation

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Gemini 3.7 Flash

Record for Gemini 3.7 Flash on FrontierSWE 2 — October 2026 evaluation

Gemini 3.8 Flash

Record for Gemini 3.8 Flash on FrontierSWE 2 — October 2026 evaluation
  • 34.0%
  • 35.0%
Evaluation details for GDP.pdf

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Gemini 3.7 Flash

Record for Gemini 3.7 Flash on GDP.pdf GDP.pdf

Gemini 3.8 Flash

Record for Gemini 3.8 Flash on GDP.pdf GDP.pdf
  • 94.8%
  • 95.4%
Evaluation details for GPQA Diamond — Epoch AI harness

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Gemini 3.7 Flash

Gemini 3.8 Flash

  • 94.2%
  • 93.9%
Evaluation details for GPQA Diamond — Mercor single-shot

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Gemini 3.7 Flash

Gemini 3.8 Flash

  • 93.9%
  • 94.4%
Evaluation details for GPQA Diamond — Vals AI harness

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Gemini 3.7 Flash

Record for Gemini 3.7 Flash on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness

Gemini 3.8 Flash

Record for Gemini 3.8 Flash on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness
  • 8.8%
  • 10.0%
Evaluation details for HLAB October 2026 evaluation

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Gemini 3.7 Flash

Gemini 3.8 Flash

Record for Gemini 3.8 Flash on Harvey's Legal Agent Benchmark October 2026 evaluation
  • 53.6%
  • 54.9%
Evaluation details for HLE-Verified Full 1,811-item verified set

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Gemini 3.7 Flash

Record for Gemini 3.7 Flash on HLE-Verified Full 1,811-item verified set

Gemini 3.8 Flash

Record for Gemini 3.8 Flash on HLE-Verified Full 1,811-item verified set
  • 46.0%
  • 47.0%
Evaluation details for Humanity's Last Exam Text only — Mercor single-shot

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Gemini 3.7 Flash

Gemini 3.8 Flash

  • 85.4%
  • 87.1%
Evaluation details for LVBench 1024 frames — Static

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Gemini 3.7 Flash

Record for Gemini 3.7 Flash on LVBench 1024 frames — Static

Gemini 3.8 Flash

Record for Gemini 3.8 Flash on LVBench 1024 frames — Static
  • 83.5%
  • 84.7%
Evaluation details for MedXpertQA MM — Mercor single-shot

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Gemini 3.7 Flash

Record for Gemini 3.7 Flash on MedXpertQA MM — Mercor single-shot

Gemini 3.8 Flash

Record for Gemini 3.8 Flash on MedXpertQA MM — Mercor single-shot
  • 90.6%
  • 90.7%
Evaluation details for MMLU-Pro Original — Mercor single-shot

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Gemini 3.7 Flash

Gemini 3.8 Flash

  • 90.1%
  • 90.2%
Evaluation details for MMLU-Pro Original — Vals AI harness

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Gemini 3.7 Flash

Gemini 3.8 Flash

  • 85.2%
  • 85.3%
Evaluation details for MMMU-Pro 1,730 questions — Mercor single-shot

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Gemini 3.7 Flash

Record for Gemini 3.7 Flash on MMMU-Pro 1,730 questions — Mercor single-shot

Gemini 3.8 Flash

Record for Gemini 3.8 Flash on MMMU-Pro 1,730 questions — Mercor single-shot
  • 89.0%
  • 89.1%
Evaluation details for MMMU-Pro Vals AI harness

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Gemini 3.7 Flash

Record for Gemini 3.7 Flash on MMMU-Pro Vals AI harness

Gemini 3.8 Flash

Record for Gemini 3.8 Flash on MMMU-Pro Vals AI harness
  • 39.2%
  • 51.2%
Evaluation details for Riemann-bench October 2026 leaderboard

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Gemini 3.7 Flash

Gemini 3.8 Flash

  • 69.2%
  • 69.7%
Evaluation details for SimpleQA Verified Original — Epoch AI harness

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

  • 82.6%
  • 85.8%
Evaluation details for SWE-bench Multilingual — Mercor mini-swe-agent, 1000 steps

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

  • 81.0%
  • 80.6%
Evaluation details for SWE-bench Verified — Mercor mini-swe-agent, 1000 steps

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Gemini 3.7 Flash

Gemini 3.8 Flash

  • 80.8%
  • 80.0%
Evaluation details for SWE-bench Verified — Vals AI mini-swe-agent

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Gemini 3.7 Flash

Gemini 3.8 Flash

  • 85.8%
  • 89.4%
Evaluation details for Terminal-Bench 2.1

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Gemini 3.7 Flash

Record for Gemini 3.7 Flash on Terminal-Bench 2.1

Gemini 3.8 Flash

Record for Gemini 3.8 Flash on Terminal-Bench 2.1
  • 77.5%
  • 81.3%
Evaluation details for Terminal-Bench 2.1 — Vals AI Terminus 2

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Gemini 3.7 Flash

Record for Gemini 3.7 Flash on Terminal-Bench 2.1 — Vals AI Terminus 2

Gemini 3.8 Flash

Record for Gemini 3.8 Flash on Terminal-Bench 2.1 — Vals AI Terminus 2
  • 11.2%
  • 19.1%
Evaluation details for Terminal-Bench 4.0 — mini-swe-agent

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Gemini 3.7 Flash

Record for Gemini 3.7 Flash on Terminal-Bench 4.0 — mini-swe-agent

Gemini 3.8 Flash

Record for Gemini 3.8 Flash on Terminal-Bench 4.0 — mini-swe-agent
  • 12.1%
  • 19.2%
Evaluation details for Terminal-Bench 4.0 — Vals AI mini-swe-agent

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Gemini 3.7 Flash

Gemini 3.8 Flash

  • 51.3%
  • 54.8%
Evaluation details for Vals Index 2.1

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Gemini 3.7 Flash

Record for Gemini 3.7 Flash on Vals Index 2.1

Gemini 3.8 Flash

Record for Gemini 3.8 Flash on Vals Index 2.1
  • 70.4%
  • 78.7%
Evaluation details for Vibe Code Bench 1.1 — OpenHands, October 2026 evaluation

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.