Shared benchmarks

13
Shared benchmarks for Claude Sonnet 4 and Claude Sonnet 4.5
BenchmarkClaude Sonnet 4Claude Sonnet 4.5
  • 66.7%
  • 73.7%
Evaluation details for GPQA Diamond — Epoch AI harness

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Claude Sonnet 4

Record for Claude Sonnet 4 on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness

Claude Sonnet 4.5

Record for Claude Sonnet 4.5 on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness
  • 69.4%
  • 75.0%thinking
  • 81.6%
Evaluation details for GPQA Diamond — Vals AI harness

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Claude Sonnet 4

Record for Claude Sonnet 4 on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness
Record for Claude Sonnet 4 on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness

Claude Sonnet 4.5

Record for Claude Sonnet 4.5 on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness
  • 5.5%
  • 7.8%Thinking
  • 7.5%
  • 13.7%thinking
Evaluation details for Humanity's Last Exam Full set — Scale AI evaluation

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

  • 5.4%
  • 7.6%Thinking
  • 7.7%
  • 14.1%thinking
Evaluation details for Humanity's Last Exam Text only — Scale AI evaluation

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

  • 79.4%
  • 83.9%thinking
  • 87.4%
Evaluation details for MMLU-Pro Original — Vals AI harness

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Claude Sonnet 4

Record for Claude Sonnet 4 on MMLU-Pro Original — Vals AI harness

Claude Sonnet 4.5

  • 72.4%
  • 74.9%thinking
  • 79.3%
Evaluation details for MMMU-Pro Vals AI harness

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Claude Sonnet 4

Record for Claude Sonnet 4 on MMMU-Pro Vals AI harness
Record for Claude Sonnet 4 on MMMU-Pro Vals AI harness

Claude Sonnet 4.5

Record for Claude Sonnet 4.5 on MMMU-Pro Vals AI harness
  • 41.4%
  • 62.9%
Evaluation details for OSWorld Verified — 100 steps

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Claude Sonnet 4

Record for Claude Sonnet 4 on OSWorld Verified — 100 steps

Claude Sonnet 4.5

Record for Claude Sonnet 4.5 on OSWorld Verified — 100 steps
  • 31.2%
  • 42.9%
Evaluation details for OSWorld Verified — 15 steps

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Claude Sonnet 4

Record for Claude Sonnet 4 on OSWorld Verified — 15 steps

Claude Sonnet 4.5

Record for Claude Sonnet 4.5 on OSWorld Verified — 15 steps
  • 43.9%
  • 58.1%
Evaluation details for OSWorld Verified — 50 steps

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Claude Sonnet 4

Record for Claude Sonnet 4 on OSWorld Verified — 50 steps

Claude Sonnet 4.5

Record for Claude Sonnet 4.5 on OSWorld Verified — 50 steps
  • 72.7%
  • 77.2%
Evaluation details for SWE-bench Verified

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Claude Sonnet 4

Record for Claude Sonnet 4 on SWE-bench Verified

Claude Sonnet 4.5

Record for Claude Sonnet 4.5 on SWE-bench Verified
  • 64.9%
  • 71.4%high
  • 70.6%
Evaluation details for SWE-bench Verified — mini-swe-agent

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Claude Sonnet 4

Record for Claude Sonnet 4 on SWE-bench Verified — mini-swe-agent

Claude Sonnet 4.5

Record for Claude Sonnet 4.5 on SWE-bench Verified — mini-swe-agent
Record for Claude Sonnet 4.5 on SWE-bench Verified — mini-swe-agent
  • 42.7%
  • 43.6%
Evaluation details for SWE-bench Pro Public — Scale AI SWE-Agent

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Claude Sonnet 4

Record for Claude Sonnet 4 on SWE-bench Pro Public — Scale AI SWE-Agent

Claude Sonnet 4.5

Record for Claude Sonnet 4.5 on SWE-bench Pro Public — Scale AI SWE-Agent
  • 29.9%
  • 38.9%
Evaluation details for Toolathlon Original

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Claude Sonnet 4

Record for Claude Sonnet 4 on Toolathlon Original

Claude Sonnet 4.5

Record for Claude Sonnet 4.5 on Toolathlon Original