Shared benchmarks

7
Shared benchmarks for Claude Opus 4 and Claude Opus 4.1
BenchmarkClaude Opus 4Claude Opus 4.1
  • 69.2%
  • 73.2%
Evaluation details for GPQA Diamond — Epoch AI harness

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Claude Opus 4

Record for Claude Opus 4 on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness

Claude Opus 4.1

Record for Claude Opus 4.1 on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness
  • 71.7%
  • 70.0%
  • 76.3%thinking
Evaluation details for GPQA Diamond — Vals AI harness

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Claude Opus 4

Record for Claude Opus 4 on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness

Claude Opus 4.1

Record for Claude Opus 4.1 on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness
Record for Claude Opus 4.1 on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness
  • 6.7%
  • 10.7%Thinking
  • 7.9%
  • 11.5%thinking
Evaluation details for Humanity's Last Exam Full set — Scale AI evaluation

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

  • 6.3%
  • 10.8%Thinking
  • 7.4%
  • 11.3%thinking
Evaluation details for Humanity's Last Exam Text only — Scale AI evaluation

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

  • 86.2%
  • 87.2%
  • 87.9%thinking
Evaluation details for MMLU-Pro Original — Vals AI harness

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Claude Opus 4

Record for Claude Opus 4 on MMLU-Pro Original — Vals AI harness

Claude Opus 4.1

Record for Claude Opus 4.1 on MMLU-Pro Original — Vals AI harness
  • 73.3%
  • 73.7%
  • 77.5%thinking
Evaluation details for MMMU-Pro Vals AI harness

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Claude Opus 4

Record for Claude Opus 4 on MMMU-Pro Vals AI harness

Claude Opus 4.1

Record for Claude Opus 4.1 on MMMU-Pro Vals AI harness
Record for Claude Opus 4.1 on MMMU-Pro Vals AI harness
  • 70.7%
  • 73.3%
Evaluation details for SWE-bench Verified — Epoch AI harness

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Claude Opus 4

Record for Claude Opus 4 on SWE-bench Verified — Epoch AI harness

Claude Opus 4.1

Record for Claude Opus 4.1 on SWE-bench Verified — Epoch AI harness