Benchmarks

55 results across 26 benchmarks

Benchmark results for Claude Opus 4.8
BenchmarkScoreSource
APEX-Agents OriginalEffort: Max48.9%mercor.com for Claude Opus 4.8 (max) on APEX-Agents Original (opens in a new tab)Archived copy (opens in a new tab)Pass rateReported Benchmark leaderboardMercorIndependentRecord for Claude Opus 4.8 on APEX-Agents Original
AutomationBench 1.0.6Effort: Low13.6%zapier.com for Claude Opus 4.8 (low) on AutomationBench 1.0.6 (opens in a new tab)Archived copy (opens in a new tab)Task pass rateReported Benchmark leaderboardZapierIndependentZapier AutomationBench TeamRecord for Claude Opus 4.8 on AutomationBench 1.0.6
AutomationBench 1.0.6Effort: Medium14.0%zapier.com for Claude Opus 4.8 (medium) on AutomationBench 1.0.6 (opens in a new tab)Archived copy (opens in a new tab)Task pass rateReported Benchmark leaderboardZapierIndependentZapier AutomationBench TeamRecord for Claude Opus 4.8 on AutomationBench 1.0.6
AutomationBench 1.0.6Effort: High16.1%zapier.com for Claude Opus 4.8 (high) on AutomationBench 1.0.6 (opens in a new tab)Archived copy (opens in a new tab)Task pass rateReported Benchmark leaderboardZapierIndependentZapier AutomationBench TeamRecord for Claude Opus 4.8 on AutomationBench 1.0.6
AutomationBench 1.0.6Effort: xHigh16.9%zapier.com for Claude Opus 4.8 (xhigh) on AutomationBench 1.0.6 (opens in a new tab)Archived copy (opens in a new tab)Task pass rateReported Benchmark leaderboardZapierIndependentZapier AutomationBench TeamRecord for Claude Opus 4.8 on AutomationBench 1.0.6
AutomationBench 1.0.6Effort: Max16.4%zapier.com for Claude Opus 4.8 (max) on AutomationBench 1.0.6 (opens in a new tab)Archived copy (opens in a new tab)Task pass rateReported Benchmark leaderboardZapierIndependentZapier AutomationBench TeamRecord for Claude Opus 4.8 on AutomationBench 1.0.6
BrowseComp 130-question subset — Mercor web research agentEffort: MaxConfiguration: Mercor web research agent, 100 steps, 3 hours73.7%mercor.com for Claude Opus 4.8 (max) on BrowseComp 130-question subset — Mercor web research agent (opens in a new tab)Archived copy (opens in a new tab)BrowseComp AccuracyReported Benchmark leaderboardMercorIndependentRecord for Claude Opus 4.8 on BrowseComp 130-question subset — Mercor web research agent
Chartography October 2026 leaderboardEffort: MaxReported as: Adaptive/Max15.9%surgehq.ai for Claude Opus 4.8 (Adaptive/Max) on Chartography October 2026 leaderboard (opens in a new tab)Archived copy (opens in a new tab)Chartography Pass@1Reported Benchmark leaderboardSurge AIIndependentRecord for Claude Opus 4.8 on Chartography October 2026 leaderboard
CharXiv Descriptive and reasoning — Mercor single-shotEffort: MaxConfiguration: Mercor single-shot agent93.2%mercor.com for Claude Opus 4.8 (max) on CharXiv Descriptive and reasoning — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)CharXiv AccuracyReported Benchmark leaderboardMercorIndependentRecord for Claude Opus 4.8 on CharXiv Descriptive and reasoning — Mercor single-shot
DeepSWE 1.1Effort: Max59.0%deepswe.datacurve.ai for Claude Opus 4.8 (max) on DeepSWE 1.1 (opens in a new tab)DeepSWE Pass@1Reported Benchmark leaderboardDatacurveIndependentRecord for Claude Opus 4.8 on DeepSWE 1.1
DeepSWE 1.1 — Mercor mini-swe-agent, 500 stepsEffort: MaxConfiguration: Mercor mini-swe-agent, 500 steps, 2 hours59.6%mercor.com for Claude Opus 4.8 (max) on DeepSWE 1.1 — Mercor mini-swe-agent, 500 steps (opens in a new tab)Archived copy (opens in a new tab)DeepSWE Pass@1Reported Benchmark leaderboardMercorIndependentRecord for Claude Opus 4.8 on DeepSWE 1.1 — Mercor mini-swe-agent, 500 steps
Finance Agent Benchmark 2 — October 2026 evaluation53.9%vals.ai for Claude Opus 4.8 on Finance Agent Benchmark 2 — October 2026 evaluation (opens in a new tab)Archived copy (opens in a new tab)Finance Agent AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Claude Opus 4.8 on Finance Agent Benchmark 2 — October 2026 evaluation
GPQA DiamondEffort: MaxReported as: adaptive thinking, max93.6%www-cdn.anthropic.com for Claude Opus 4.8 (adaptive thinking, max) on Graduate-Level Google-Proof Q&A Diamond (opens in a new tab)GPQA Diamond AccuracyReported System cardAnthropicSelf-reportedRecord for Claude Opus 4.8 on Graduate-Level Google-Proof Q&A Diamond
GPQA Diamond — Epoch AI harnessEffort: NoneConfiguration: Epoch AI harness85.4%epoch.ai for Claude Opus 4.8 (none) on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness (opens in a new tab)Archived copy (opens in a new tab)GPQA Diamond AccuracyReported Evaluated Benchmark leaderboardEpoch AIIndependentRecord for Claude Opus 4.8 on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness
GPQA Diamond — Epoch AI harnessEffort: LowConfiguration: Epoch AI harness88.4%epoch.ai for Claude Opus 4.8 (low) on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness (opens in a new tab)Archived copy (opens in a new tab)GPQA Diamond AccuracyReported Evaluated Benchmark leaderboardEpoch AIIndependentRecord for Claude Opus 4.8 on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness
GPQA Diamond — Epoch AI harnessEffort: MaxConfiguration: Epoch AI harness91.0%epoch.ai for Claude Opus 4.8 (max) on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness (opens in a new tab)Archived copy (opens in a new tab)GPQA Diamond AccuracyReported Evaluated Benchmark leaderboardEpoch AIIndependentRecord for Claude Opus 4.8 on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness
GPQA Diamond — Mercor single-shotEffort: MaxConfiguration: Mercor single-shot agent92.0%mercor.com for Claude Opus 4.8 (max) on Graduate-Level Google-Proof Q&A Diamond — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)GPQA Diamond AccuracyReported Benchmark leaderboardMercorIndependentRecord for Claude Opus 4.8 on Graduate-Level Google-Proof Q&A Diamond — Mercor single-shot
GPQA Diamond — Vals AI harnessConfiguration: Vals AI harness: mean of zero-shot and few-shot CoT92.4%vals.ai for Claude Opus 4.8 on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)GPQA Diamond AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Claude Opus 4.8 on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness
HLAB October 2026 evaluation9.6%vals.ai for Claude Opus 4.8 on Harvey's Legal Agent Benchmark October 2026 evaluation (opens in a new tab)Archived copy (opens in a new tab)HLAB Task Pass RateReported Benchmark leaderboardVals AIIndependentRecord for Claude Opus 4.8 on Harvey's Legal Agent Benchmark October 2026 evaluation
Humanity's Last Exam Full set — No toolsEffort: MaxReported as: adaptive thinking, maxConfiguration: No tools49.8%www-cdn.anthropic.com for Claude Opus 4.8 (adaptive thinking, max) on Humanity's Last Exam Full set — No tools (opens in a new tab)Humanity's Last Exam AccuracyReported System cardAnthropicSelf-reportedRecord for Claude Opus 4.8 on Humanity's Last Exam Full set — No tools
Humanity's Last Exam Full set — With toolsEffort: MaxReported as: adaptive thinking, maxConfiguration: With tools57.9%www-cdn.anthropic.com for Claude Opus 4.8 (adaptive thinking, max) on Humanity's Last Exam Full set — With tools (opens in a new tab)Humanity's Last Exam AccuracyReported System cardAnthropicSelf-reportedRecord for Claude Opus 4.8 on Humanity's Last Exam Full set — With tools
Humanity's Last Exam Text only — Mercor single-shotEffort: HighConfiguration: Mercor single-shot agent44.8%mercor.com for Claude Opus 4.8 (high) on Humanity's Last Exam Text only — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)Humanity's Last Exam AccuracyReported Benchmark leaderboardMercorIndependentRecord for Claude Opus 4.8 on Humanity's Last Exam Text only — Mercor single-shot
Humanity's Last Exam Text only — Mercor single-shotEffort: MaxConfiguration: Mercor single-shot agent37.1%mercor.com for Claude Opus 4.8 (max) on Humanity's Last Exam Text only — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)Humanity's Last Exam AccuracyReported Benchmark leaderboardMercorIndependentRecord for Claude Opus 4.8 on Humanity's Last Exam Text only — Mercor single-shot
MCP Atlas Public, April 2026 updateEffort: Max82.2%labs.scale.com for Claude Opus 4.8 (max) on MCP Atlas Public, April 2026 update (opens in a new tab)Pass rateReported Benchmark leaderboardScale AIIndependentRecord for Claude Opus 4.8 on MCP Atlas Public, April 2026 update
MedXpertQA MM — Mercor single-shotEffort: MaxConfiguration: Mercor single-shot agent72.5%mercor.com for Claude Opus 4.8 (max) on MedXpertQA MM — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)AccuracyReported Benchmark leaderboardMercorIndependentRecord for Claude Opus 4.8 on MedXpertQA MM — Mercor single-shot
MMLU-Pro Original — Mercor single-shotEffort: MaxConfiguration: Mercor single-shot agent90.0%mercor.com for Claude Opus 4.8 (max) on MMLU-Pro Original — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)AccuracyReported Benchmark leaderboardMercorIndependentRecord for Claude Opus 4.8 on MMLU-Pro Original — Mercor single-shot
MMLU-Pro Original — Vals AI harnessConfiguration: Vals AI harness89.6%vals.ai for Claude Opus 4.8 on MMLU-Pro Original — Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Claude Opus 4.8 on MMLU-Pro Original — Vals AI harness
MMMU-Pro 1,730 questions — Mercor single-shotEffort: MaxConfiguration: Mercor single-shot agent79.5%mercor.com for Claude Opus 4.8 (max) on MMMU-Pro 1,730 questions — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Not statedReported Benchmark leaderboardMercorIndependentRecord for Claude Opus 4.8 on MMMU-Pro 1,730 questions — Mercor single-shot
MMMU-Pro Vals AI harnessConfiguration: Vals AI harness86.6%vals.ai for Claude Opus 4.8 on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for Claude Opus 4.8 on MMMU-Pro Vals AI harness
OSWorld VerifiedEffort: MaxReported as: adaptive thinking, max83.4%www-cdn.anthropic.com for Claude Opus 4.8 (adaptive thinking, max) on OSWorld Verified (opens in a new tab)OSWorld-Verified success rateReported System cardAnthropicSelf-reportedRecord for Claude Opus 4.8 on OSWorld Verified
PostTrainBench 1.1 — Claude CodeEffort: HighConfiguration: Claude Code harness33.8%posttrainbench.com for Claude Opus 4.8 (High) on PostTrainBench 1.1 — Claude Code (opens in a new tab)Archived copy (opens in a new tab)PostTrainBench Weighted AggregateReported Benchmark leaderboardPostTrainBenchIndependentPostTrainBench TeamRecord for Claude Opus 4.8 on PostTrainBench 1.1 — Claude Code
PostTrainBench 1.1 — Claude CodeEffort: MaxConfiguration: Claude Code harness32.9%posttrainbench.com for Claude Opus 4.8 (Max) on PostTrainBench 1.1 — Claude Code (opens in a new tab)Archived copy (opens in a new tab)PostTrainBench Weighted AggregateReported Benchmark leaderboardPostTrainBenchIndependentPostTrainBench TeamRecord for Claude Opus 4.8 on PostTrainBench 1.1 — Claude Code
PostTrainBench 1.2 — Claude CodeEffort: HighConfiguration: Claude Code harness30.7%posttrainbench.com for Claude Opus 4.8 (High) on PostTrainBench 1.2 — Claude Code (opens in a new tab)Archived copy (opens in a new tab)PostTrainBench Weighted AggregateReported Benchmark leaderboardPostTrainBenchIndependentPostTrainBench TeamRecord for Claude Opus 4.8 on PostTrainBench 1.2 — Claude Code
PostTrainBench 1.2 — Claude CodeEffort: MaxConfiguration: Claude Code harness31.0%posttrainbench.com for Claude Opus 4.8 (Max) on PostTrainBench 1.2 — Claude Code (opens in a new tab)Archived copy (opens in a new tab)PostTrainBench Weighted AggregateReported Benchmark leaderboardPostTrainBenchIndependentPostTrainBench TeamRecord for Claude Opus 4.8 on PostTrainBench 1.2 — Claude Code
Riemann-bench October 2026 leaderboardEffort: MaxReported as: Adaptive/Max47.2%surgehq.ai for Claude Opus 4.8 (Adaptive/Max) on Riemann-bench October 2026 leaderboard (opens in a new tab)Archived copy (opens in a new tab)Riemann-bench Pass@1Reported Benchmark leaderboardSurge AIIndependentRecord for Claude Opus 4.8 on Riemann-bench October 2026 leaderboard
SimpleQA Verified Original — Epoch AI harnessEffort: MaxConfiguration: Epoch AI harness53.0%epoch.ai for Claude Opus 4.8 (max) on SimpleQA Verified Original — Epoch AI harness (opens in a new tab)Archived copy (opens in a new tab)AccuracyReported Evaluated Benchmark leaderboardEpoch AIIndependentRecord for Claude Opus 4.8 on SimpleQA Verified Original — Epoch AI harness
SWE Atlas Codebase QnA — Mercor mini-swe-agent, 250 stepsEffort: MaxConfiguration: Mercor mini-swe-agent, 250 steps, 1 hour44.4%mercor.com for Claude Opus 4.8 (max) on SWE Atlas Codebase QnA — Mercor mini-swe-agent, 250 steps (opens in a new tab)Archived copy (opens in a new tab)Mean pass@1Reported Benchmark leaderboardMercorIndependentRecord for Claude Opus 4.8 on SWE Atlas Codebase QnA — Mercor mini-swe-agent, 250 steps
SWE Atlas Codebase QnA — Public; Claude CodeEffort: xHighConfiguration: Claude Code harness57.3%labs.scale.com for Claude Opus 4.8 (xhigh) on SWE Atlas Codebase QnA — Public; Claude Code (opens in a new tab)Archived copy (opens in a new tab)Mean pass@1Reported Benchmark leaderboardScale AIIndependentRecord for Claude Opus 4.8 on SWE Atlas Codebase QnA — Public; Claude Code
SWE Atlas Refactoring — Claude CodeConfiguration: Claude Code harness46.7%labs.scale.com for Claude Opus 4.8 on SWE Atlas Refactoring — Claude Code (opens in a new tab)Archived copy (opens in a new tab)Mean pass@1Reported Benchmark leaderboardScale AIIndependentRecord for Claude Opus 4.8 on SWE Atlas Refactoring — Claude Code
SWE-bench MultilingualEffort: MaxReported as: adaptive thinking, max84.4%www-cdn.anthropic.com for Claude Opus 4.8 (adaptive thinking, max) on SWE-bench Multilingual (opens in a new tab)Resolved TasksReported System cardAnthropicSelf-reportedRecord for Claude Opus 4.8 on SWE-bench Multilingual
SWE-bench Multilingual — Mercor mini-swe-agent, 1000 stepsEffort: MaxConfiguration: Mercor mini-swe-agent, 1000 steps, 3 hours86.6%mercor.com for Claude Opus 4.8 (max) on SWE-bench Multilingual — Mercor mini-swe-agent, 1000 steps (opens in a new tab)Archived copy (opens in a new tab)Resolved TasksReported Benchmark leaderboardMercorIndependentRecord for Claude Opus 4.8 on SWE-bench Multilingual — Mercor mini-swe-agent, 1000 steps
SWE-bench VerifiedEffort: MaxReported as: adaptive thinking, max88.6%www-cdn.anthropic.com for Claude Opus 4.8 (adaptive thinking, max) on SWE-bench Verified (opens in a new tab)Resolved TasksReported System cardAnthropicSelf-reportedRecord for Claude Opus 4.8 on SWE-bench Verified
SWE-bench Verified — Mercor mini-swe-agent, 1000 stepsEffort: MaxConfiguration: Mercor mini-swe-agent, 1000 steps, 3 hours88.5%mercor.com for Claude Opus 4.8 (max) on SWE-bench Verified — Mercor mini-swe-agent, 1000 steps (opens in a new tab)Archived copy (opens in a new tab)Resolved TasksReported Benchmark leaderboardMercorIndependentRecord for Claude Opus 4.8 on SWE-bench Verified — Mercor mini-swe-agent, 1000 steps
SWE-bench Verified — Vals AI mini-swe-agentConfiguration: mini-swe-agent harness88.6%vals.ai for Claude Opus 4.8 on SWE-bench Verified — Vals AI mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Resolved TasksReported Benchmark leaderboardVals AIIndependentRecord for Claude Opus 4.8 on SWE-bench Verified — Vals AI mini-swe-agent
SWE-bench Pro PublicEffort: MaxReported as: adaptive thinking, max69.2%www-cdn.anthropic.com for Claude Opus 4.8 (adaptive thinking, max) on SWE-bench Pro Public (opens in a new tab)Public tasks resolvedReported System cardAnthropicSelf-reportedRecord for Claude Opus 4.8 on SWE-bench Pro Public
Terminal-Bench 2.0 — Vals AI Terminus 2Configuration: Terminus-2 harness70.0%vals.ai for Claude Opus 4.8 on Terminal-Bench 2.0 — Vals AI Terminus 2 (opens in a new tab)Archived copy (opens in a new tab)Terminal-Bench AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Claude Opus 4.8 on Terminal-Bench 2.0 — Vals AI Terminus 2
Terminal-Bench 2.1Effort: MaxReported as: adaptive thinking, max74.6%www-cdn.anthropic.com for Claude Opus 4.8 (adaptive thinking, max) on Terminal-Bench 2.1 (opens in a new tab)Terminal-Bench AccuracyReported System cardAnthropicSelf-reportedRecord for Claude Opus 4.8 on Terminal-Bench 2.1
Terminal-Bench 2.1 — Vals AI Terminus 2Configuration: Terminus-2 harness71.9%vals.ai for Claude Opus 4.8 on Terminal-Bench 2.1 — Vals AI Terminus 2 (opens in a new tab)Archived copy (opens in a new tab)Terminal-Bench AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Claude Opus 4.8 on Terminal-Bench 2.1 — Vals AI Terminus 2
Terminal-Bench 4.0 — Claude CodeEffort: MaxConfiguration: Claude Code harness23.6%tbench.ai for Claude Opus 4.8 (max) on Terminal-Bench 4.0 — Claude Code (opens in a new tab)Archived copy (opens in a new tab)Terminal-Bench AccuracyReported Benchmark leaderboardTerminal-BenchIndependentTerminal-Bench TeamRecord for Claude Opus 4.8 on Terminal-Bench 4.0 — Claude Code
Terminal-Bench 4.0 — Vals AI mini-swe-agentConfiguration: mini-swe-agent harness23.2%vals.ai for Claude Opus 4.8 on Terminal-Bench 4.0 — Vals AI mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Terminal-Bench AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Claude Opus 4.8 on Terminal-Bench 4.0 — Vals AI mini-swe-agent