Shared benchmarks

29
Shared benchmarks for Claude Sonnet 4.6 and Claude Sonnet 5
BenchmarkClaude Sonnet 4.6Claude Sonnet 5
  • 43.0%
  • 54.5%
Evaluation details for APEX-Agents Original

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Claude Sonnet 4.6

Record for Claude Sonnet 4.6 on APEX-Agents Original

Claude Sonnet 5

Record for Claude Sonnet 5 on APEX-Agents Original
  • 2.9%low
  • 3.7%medium
  • 5.0%high
  • 5.3%max
  • 3.7%low
  • 5.8%medium
  • 9.9%high
  • 7.6%xhigh
  • 10.7%max
Evaluation details for AutomationBench 1.0.6

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Claude Sonnet 4.6

Record for Claude Sonnet 4.6 on AutomationBench 1.0.6
Record for Claude Sonnet 4.6 on AutomationBench 1.0.6
Record for Claude Sonnet 4.6 on AutomationBench 1.0.6
Record for Claude Sonnet 4.6 on AutomationBench 1.0.6

Claude Sonnet 5

Record for Claude Sonnet 5 on AutomationBench 1.0.6
Record for Claude Sonnet 5 on AutomationBench 1.0.6
Record for Claude Sonnet 5 on AutomationBench 1.0.6
Record for Claude Sonnet 5 on AutomationBench 1.0.6
Record for Claude Sonnet 5 on AutomationBench 1.0.6
  • 56.7%
  • 68.7%
Evaluation details for BrowseComp 130-question subset — Mercor web research agent

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

  • 90.5%
  • 88.8%
Evaluation details for CharXiv Descriptive and reasoning — Mercor single-shot

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

  • 34.2%
  • 49.6%
Evaluation details for DeepSWE 1.1 — Mercor mini-swe-agent, 500 steps

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Claude Sonnet 4.6

Claude Sonnet 5

  • 51.0%
  • 53.9%
Evaluation details for Finance Agent Benchmark 2 — October 2026 evaluation

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

  • 83.3%medium
  • 83.3%high
  • 78.8%max
  • 90.5%xhigh
  • 80.3%max
Evaluation details for GPQA Diamond — Epoch AI harness

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Claude Sonnet 4.6

Claude Sonnet 5

  • 84.7%
  • 80.6%
Evaluation details for GPQA Diamond — Mercor single-shot

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Claude Sonnet 4.6

Claude Sonnet 5

  • 85.6%
  • 88.9%
Evaluation details for GPQA Diamond — Vals AI harness

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Claude Sonnet 4.6

Record for Claude Sonnet 4.6 on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness

Claude Sonnet 5

Record for Claude Sonnet 5 on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness
  • 5.0%
  • 5.0%
Evaluation details for HLAB October 2026 evaluation

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Claude Sonnet 4.6

Record for Claude Sonnet 4.6 on Harvey's Legal Agent Benchmark October 2026 evaluation

Claude Sonnet 5

Record for Claude Sonnet 5 on Harvey's Legal Agent Benchmark October 2026 evaluation
  • 33.2%
  • 43.2%
Evaluation details for Humanity's Last Exam Full set — No tools

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Claude Sonnet 4.6

Record for Claude Sonnet 4.6 on Humanity's Last Exam Full set — No tools

Claude Sonnet 5

  • 49.0%
  • 36.5%low
  • 47.2%medium
  • 52.8%high
  • 54.6%xhigh
Evaluation details for Humanity's Last Exam Full set — With tools

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Claude Sonnet 4.6

Record for Claude Sonnet 4.6 on Humanity's Last Exam Full set — With tools
  • 29.2%medium
  • 24.4%high
  • 36.3%high
  • 27.3%max
Evaluation details for Humanity's Last Exam Text only — Mercor single-shot

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

  • 61.7%
  • 63.0%
Evaluation details for MedXpertQA MM — Mercor single-shot

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Claude Sonnet 4.6

Claude Sonnet 5

Record for Claude Sonnet 5 on MedXpertQA MM — Mercor single-shot
  • 88.0%
  • 88.5%
Evaluation details for MMLU-Pro Original — Mercor single-shot

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Claude Sonnet 4.6

Claude Sonnet 5

  • 87.3%
  • 87.5%
Evaluation details for MMLU-Pro Original — Vals AI harness

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Claude Sonnet 4.6

Record for Claude Sonnet 4.6 on MMLU-Pro Original — Vals AI harness

Claude Sonnet 5

Record for Claude Sonnet 5 on MMLU-Pro Original — Vals AI harness
  • 74.6%
  • 75.5%
Evaluation details for MMMU-Pro 1,730 questions — Mercor single-shot

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Claude Sonnet 4.6

Record for Claude Sonnet 4.6 on MMMU-Pro 1,730 questions — Mercor single-shot

Claude Sonnet 5

Record for Claude Sonnet 5 on MMMU-Pro 1,730 questions — Mercor single-shot
  • 83.6%
  • 83.0%
Evaluation details for MMMU-Pro Vals AI harness

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Claude Sonnet 4.6

Record for Claude Sonnet 4.6 on MMMU-Pro Vals AI harness

Claude Sonnet 5

Record for Claude Sonnet 5 on MMMU-Pro Vals AI harness
  • 72.5%
  • 81.2%
Evaluation details for OSWorld Verified

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Claude Sonnet 4.6

Record for Claude Sonnet 4.6 on OSWorld Verified

Claude Sonnet 5

Record for Claude Sonnet 5 on OSWorld Verified
  • 35.5%high
  • 32.8%max
  • 32.9%xhigh
  • 33.7%max
Evaluation details for SimpleQA Verified Original — Epoch AI harness

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

  • 35.5%
  • 39.5%
Evaluation details for SWE Atlas Codebase QnA — Mercor mini-swe-agent, 250 steps

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

  • 75.9%
  • 78.3%
Evaluation details for SWE-bench Multilingual

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Claude Sonnet 4.6

Record for Claude Sonnet 4.6 on SWE-bench Multilingual

Claude Sonnet 5

Record for Claude Sonnet 5 on SWE-bench Multilingual
  • 78.1%
  • 89.2%
Evaluation details for SWE-bench Multilingual — Mercor mini-swe-agent, 1000 steps

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

  • 79.6%
  • 85.2%
Evaluation details for SWE-bench Verified

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Claude Sonnet 4.6

Record for Claude Sonnet 4.6 on SWE-bench Verified

Claude Sonnet 5

Record for Claude Sonnet 5 on SWE-bench Verified
  • 79.1%
  • 82.8%
Evaluation details for SWE-bench Verified — Mercor mini-swe-agent, 1000 steps

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

  • 77.4%
  • 79.6%
Evaluation details for SWE-bench Verified — Vals AI mini-swe-agent

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Claude Sonnet 4.6

Claude Sonnet 5

  • 57.3%
  • 74.5%
Evaluation details for Terminal-Bench 2.1 — Vals AI Terminus 2

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.

Claude Sonnet 4.6

Record for Claude Sonnet 4.6 on Terminal-Bench 2.1 — Vals AI Terminus 2

Claude Sonnet 5

Record for Claude Sonnet 5 on Terminal-Bench 2.1 — Vals AI Terminus 2
ToolathlonPotentially non-equivalent: Benchmark versions differ.
  • 44.8%Original · Toolathlon Authors
  • 71.6%Verified · Toolathlon Authors
Potentially non-equivalent · Evaluation details for Toolathlon

Benchmark versions differ. Evaluation methodology is not recorded by the registry.

Claude Sonnet 4.6

Record for Claude Sonnet 4.6 on Toolathlon Original

Claude Sonnet 5

Record for Claude Sonnet 5 on Toolathlon Verified
  • 51.5%
  • 81.3%
Evaluation details for Vibe Code Bench 1.1 — OpenHands, October 2026 evaluation

The recorded benchmark version, metric, and evaluator sets match. Evaluation methodology is not recorded by the registry.