GPQA Diamond Effort: None Reported as: No extended thinking 68.0% anthropic.com for Claude 3.7 Sonnet (No extended thinking) on Graduate-Level Google-Proof Q&A Diamond (opens in a new tab) GPQA Diamond Accuracy Reported February 24, 2025 Launch post Anthropic Self-reported Record for Claude 3.7 Sonnet on Graduate-Level Google-Proof Q&A Diamond GPQA Diamond Reported as: 64K extended thinking 78.2% anthropic.com for Claude 3.7 Sonnet (64K extended thinking) on Graduate-Level Google-Proof Q&A Diamond (opens in a new tab) GPQA Diamond Accuracy Reported February 24, 2025 Launch post Anthropic Self-reported Record for Claude 3.7 Sonnet on Graduate-Level Google-Proof Q&A Diamond GPQA Diamond — Epoch AI harness Configuration: Epoch AI harness 66.0% epoch.ai for Claude 3.7 Sonnet on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness (opens in a new tab) Archived copy (opens in a new tab) GPQA Diamond Accuracy Reported October 7, 2026 Evaluated February 24, 2025 at 7:14 PM UTC Benchmark leaderboard Epoch AI Independent Record for Claude 3.7 Sonnet on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness GPQA Diamond — Vals AI harness Configuration: Vals AI harness: mean of zero-shot and few-shot CoT 67.4% vals.ai for Claude 3.7 Sonnet on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness (opens in a new tab) Archived copy (opens in a new tab) GPQA Diamond Accuracy Reported September 1, 2026 Benchmark leaderboard Vals AI Independent Record for Claude 3.7 Sonnet on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness GPQA Diamond — Vals AI harness Reported as: thinking Configuration: Vals AI harness: mean of zero-shot and few-shot CoT 75.3% vals.ai for Claude 3.7 Sonnet (thinking) on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness (opens in a new tab) Archived copy (opens in a new tab) GPQA Diamond Accuracy Reported September 1, 2026 Benchmark leaderboard Vals AI Independent Record for Claude 3.7 Sonnet on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness Humanity's Last Exam Full set — Scale AI evaluation Reported as: Thinking 8.0% labs.scale.com for Claude 3.7 Sonnet (Thinking) on Humanity's Last Exam Full set — Scale AI evaluation (opens in a new tab) Archived copy (opens in a new tab) Humanity's Last Exam Accuracy Reported October 3, 2026 Benchmark leaderboard Scale AI Independent Record for Claude 3.7 Sonnet on Humanity's Last Exam Full set — Scale AI evaluation Humanity's Last Exam Text only — Scale AI evaluation Reported as: Thinking 7.9% labs.scale.com for Claude 3.7 Sonnet (Thinking) on Humanity's Last Exam Text only — Scale AI evaluation (opens in a new tab) Archived copy (opens in a new tab) Humanity's Last Exam Accuracy Reported October 7, 2026 Benchmark leaderboard Scale AI Independent Record for Claude 3.7 Sonnet on Humanity's Last Exam Text only — Scale AI evaluation MMLU-Pro Original — Vals AI harness Configuration: Vals AI harness 80.7% vals.ai for Claude 3.7 Sonnet on MMLU-Pro Original — Vals AI harness (opens in a new tab) Archived copy (opens in a new tab) Accuracy Reported September 1, 2026 Benchmark leaderboard Vals AI Independent Record for Claude 3.7 Sonnet on MMLU-Pro Original — Vals AI harness MMLU-Pro Original — Vals AI harness Reported as: thinking Configuration: Vals AI harness 82.7% vals.ai for Claude 3.7 Sonnet (thinking) on MMLU-Pro Original — Vals AI harness (opens in a new tab) Archived copy (opens in a new tab) Accuracy Reported September 1, 2026 Benchmark leaderboard Vals AI Independent Record for Claude 3.7 Sonnet on MMLU-Pro Original — Vals AI harness MMMU-Pro Vals AI harness Configuration: Vals AI harness 71.5% vals.ai for Claude 3.7 Sonnet on MMMU-Pro Vals AI harness (opens in a new tab) Archived copy (opens in a new tab) MMMU-Pro accuracy Setting: Standard (4 options) Reported September 1, 2026 Benchmark leaderboard Vals AI Independent Record for Claude 3.7 Sonnet on MMMU-Pro Vals AI harness MMMU-Pro Vals AI harness Reported as: thinking Configuration: Vals AI harness 75.1% vals.ai for Claude 3.7 Sonnet (thinking) on MMMU-Pro Vals AI harness (opens in a new tab) Archived copy (opens in a new tab) MMMU-Pro accuracy Setting: Standard (4 options) Reported September 1, 2026 Benchmark leaderboard Vals AI Independent Record for Claude 3.7 Sonnet on MMMU-Pro Vals AI harness OSWorld Verified — 100 steps Configuration: 100-step limit 35.6% os-world.github.io for Claude 3.7 Sonnet on OSWorld Verified — 100 steps (opens in a new tab) Archived copy (opens in a new tab) OSWorld-Verified success rate Reported July 28, 2025 Benchmark leaderboard OSWorld Independent XLang Lab Record for Claude 3.7 Sonnet on OSWorld Verified — 100 steps OSWorld Verified — 15 steps Configuration: 15-step limit 27.1% os-world.github.io for Claude 3.7 Sonnet on OSWorld Verified — 15 steps (opens in a new tab) Archived copy (opens in a new tab) OSWorld-Verified success rate Reported July 28, 2025 Benchmark leaderboard OSWorld Independent XLang Lab Record for Claude 3.7 Sonnet on OSWorld Verified — 15 steps OSWorld Verified — 50 steps Configuration: 50-step limit 35.8% os-world.github.io for Claude 3.7 Sonnet on OSWorld Verified — 50 steps (opens in a new tab) Archived copy (opens in a new tab) OSWorld-Verified success rate Reported July 28, 2025 Benchmark leaderboard OSWorld Independent XLang Lab Record for Claude 3.7 Sonnet on OSWorld Verified — 50 steps SWE-bench Verified Effort: None Reported as: No extended thinking 62.3% anthropic.com for Claude 3.7 Sonnet (No extended thinking) on SWE-bench Verified (opens in a new tab) Resolved Tasks Reported February 24, 2025 Launch post Anthropic Self-reported Record for Claude 3.7 Sonnet on SWE-bench Verified SWE-bench Verified — Epoch AI harness Configuration: Epoch AI harness 61.0% epoch.ai for Claude 3.7 Sonnet on SWE-bench Verified — Epoch AI harness (opens in a new tab) Archived copy (opens in a new tab) Resolved Tasks Reported October 7, 2026 Evaluated February 4, 2026 at 6:05 PM UTC Benchmark leaderboard Epoch AI Independent Record for Claude 3.7 Sonnet on SWE-bench Verified — Epoch AI harness SWE-bench Verified — mini-swe-agent Configuration: mini-swe-agent harness 52.8% github.com for Claude 3.7 Sonnet on SWE-bench Verified — mini-swe-agent (opens in a new tab) Archived copy (opens in a new tab) Resolved Tasks Reported July 20, 2025 Benchmark repository SWE-bench Independent SWE-bench Team Record for Claude 3.7 Sonnet on SWE-bench Verified — mini-swe-agent