GPQA Diamond Effort: None Reported as: standard (no extended thinking) 70.0% anthropic.com for Claude Sonnet 4 (standard (no extended thinking)) on Graduate-Level Google-Proof Q&A Diamond (opens in a new tab) GPQA Diamond Accuracy Reported May 22, 2025 Launch post Anthropic Self-reported Record for Claude Sonnet 4 on Graduate-Level Google-Proof Q&A Diamond GPQA Diamond — Epoch AI harness Configuration: Epoch AI harness 66.7% epoch.ai for Claude Sonnet 4 on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness (opens in a new tab) Archived copy (opens in a new tab) GPQA Diamond Accuracy Reported October 7, 2026 Evaluated May 22, 2025 at 5:26 PM UTC Benchmark leaderboard Epoch AI Independent Record for Claude Sonnet 4 on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness GPQA Diamond — Vals AI harness Configuration: Vals AI harness: mean of zero-shot and few-shot CoT 69.4% vals.ai for Claude Sonnet 4 on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness (opens in a new tab) Archived copy (opens in a new tab) GPQA Diamond Accuracy Reported September 1, 2026 Benchmark leaderboard Vals AI Independent Record for Claude Sonnet 4 on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness GPQA Diamond — Vals AI harness Reported as: thinking Configuration: Vals AI harness: mean of zero-shot and few-shot CoT 75.0% vals.ai for Claude Sonnet 4 (thinking) on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness (opens in a new tab) Archived copy (opens in a new tab) GPQA Diamond Accuracy Reported September 1, 2026 Benchmark leaderboard Vals AI Independent Record for Claude Sonnet 4 on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness Humanity's Last Exam Full set — Scale AI evaluation 5.5% labs.scale.com for Claude Sonnet 4 on Humanity's Last Exam Full set — Scale AI evaluation (opens in a new tab) Archived copy (opens in a new tab) Humanity's Last Exam Accuracy Reported October 3, 2026 Benchmark leaderboard Scale AI Independent Record for Claude Sonnet 4 on Humanity's Last Exam Full set — Scale AI evaluation Humanity's Last Exam Full set — Scale AI evaluation Reported as: Thinking 7.8% labs.scale.com for Claude Sonnet 4 (Thinking) on Humanity's Last Exam Full set — Scale AI evaluation (opens in a new tab) Archived copy (opens in a new tab) Humanity's Last Exam Accuracy Reported October 3, 2026 Benchmark leaderboard Scale AI Independent Record for Claude Sonnet 4 on Humanity's Last Exam Full set — Scale AI evaluation Humanity's Last Exam Text only — Scale AI evaluation 5.4% labs.scale.com for Claude Sonnet 4 on Humanity's Last Exam Text only — Scale AI evaluation (opens in a new tab) Archived copy (opens in a new tab) Humanity's Last Exam Accuracy Reported October 7, 2026 Benchmark leaderboard Scale AI Independent Record for Claude Sonnet 4 on Humanity's Last Exam Text only — Scale AI evaluation Humanity's Last Exam Text only — Scale AI evaluation Reported as: Thinking 7.6% labs.scale.com for Claude Sonnet 4 (Thinking) on Humanity's Last Exam Text only — Scale AI evaluation (opens in a new tab) Archived copy (opens in a new tab) Humanity's Last Exam Accuracy Reported October 7, 2026 Benchmark leaderboard Scale AI Independent Record for Claude Sonnet 4 on Humanity's Last Exam Text only — Scale AI evaluation MMMU Effort: None Reported as: standard (no extended thinking) 72.6% anthropic.com for Claude Sonnet 4 (standard (no extended thinking)) on Massive Multi-discipline Multimodal Understanding MMMU (opens in a new tab) MMMU Accuracy Reported May 22, 2025 Launch post Anthropic Self-reported Record for Claude Sonnet 4 on Massive Multi-discipline Multimodal Understanding MMMU MMLU-Pro Original — Vals AI harness Configuration: Vals AI harness 79.4% vals.ai for Claude Sonnet 4 on MMLU-Pro Original — Vals AI harness (opens in a new tab) Archived copy (opens in a new tab) Accuracy Reported September 1, 2026 Benchmark leaderboard Vals AI Independent Record for Claude Sonnet 4 on MMLU-Pro Original — Vals AI harness MMLU-Pro Original — Vals AI harness Reported as: thinking Configuration: Vals AI harness 83.9% vals.ai for Claude Sonnet 4 (thinking) on MMLU-Pro Original — Vals AI harness (opens in a new tab) Archived copy (opens in a new tab) Accuracy Reported September 1, 2026 Benchmark leaderboard Vals AI Independent Record for Claude Sonnet 4 on MMLU-Pro Original — Vals AI harness MMMU-Pro Vals AI harness Configuration: Vals AI harness 72.4% vals.ai for Claude Sonnet 4 on MMMU-Pro Vals AI harness (opens in a new tab) Archived copy (opens in a new tab) MMMU-Pro accuracy Setting: Standard (4 options) Reported September 1, 2026 Benchmark leaderboard Vals AI Independent Record for Claude Sonnet 4 on MMMU-Pro Vals AI harness MMMU-Pro Vals AI harness Reported as: thinking Configuration: Vals AI harness 74.9% vals.ai for Claude Sonnet 4 (thinking) on MMMU-Pro Vals AI harness (opens in a new tab) Archived copy (opens in a new tab) MMMU-Pro accuracy Setting: Standard (4 options) Reported September 1, 2026 Benchmark leaderboard Vals AI Independent Record for Claude Sonnet 4 on MMMU-Pro Vals AI harness OSWorld Verified — 100 steps Configuration: 100-step limit 41.4% os-world.github.io for Claude Sonnet 4 on OSWorld Verified — 100 steps (opens in a new tab) Archived copy (opens in a new tab) OSWorld-Verified success rate Reported July 28, 2025 Benchmark leaderboard OSWorld Independent XLang Lab Record for Claude Sonnet 4 on OSWorld Verified — 100 steps OSWorld Verified — 15 steps Configuration: 15-step limit 31.2% os-world.github.io for Claude Sonnet 4 on OSWorld Verified — 15 steps (opens in a new tab) Archived copy (opens in a new tab) OSWorld-Verified success rate Reported July 28, 2025 Benchmark leaderboard OSWorld Independent XLang Lab Record for Claude Sonnet 4 on OSWorld Verified — 15 steps OSWorld Verified — 50 steps Configuration: 50-step limit 43.9% os-world.github.io for Claude Sonnet 4 on OSWorld Verified — 50 steps (opens in a new tab) Archived copy (opens in a new tab) OSWorld-Verified success rate Reported July 28, 2025 Benchmark leaderboard OSWorld Independent XLang Lab Record for Claude Sonnet 4 on OSWorld Verified — 50 steps SWE-bench Verified Effort: None Reported as: standard (no extended thinking) 72.7% anthropic.com for Claude Sonnet 4 (standard (no extended thinking)) on SWE-bench Verified (opens in a new tab) Resolved Tasks Reported May 22, 2025 Launch post Anthropic Self-reported Record for Claude Sonnet 4 on SWE-bench Verified SWE-bench Verified — mini-swe-agent Configuration: mini-swe-agent harness 64.9% github.com for Claude Sonnet 4 on SWE-bench Verified — mini-swe-agent (opens in a new tab) Archived copy (opens in a new tab) Resolved Tasks Reported July 26, 2025 Benchmark repository SWE-bench Independent SWE-bench Team Record for Claude Sonnet 4 on SWE-bench Verified — mini-swe-agent SWE-bench Pro Private — Scale AI SWE-Agent Configuration: SWE-Agent harness, 250 turns, uncapped cost 9.1% labs.scale.com for Claude Sonnet 4 on SWE-bench Pro Private — Scale AI SWE-Agent (opens in a new tab) Archived copy (opens in a new tab) Public tasks resolved Reported October 7, 2026 Benchmark leaderboard Scale AI Independent Record for Claude Sonnet 4 on SWE-bench Pro Private — Scale AI SWE-Agent SWE-bench Pro Public — Scale AI SWE-Agent Configuration: SWE-Agent harness, 250 turns, uncapped cost 42.7% labs.scale.com for Claude Sonnet 4 on SWE-bench Pro Public — Scale AI SWE-Agent (opens in a new tab) Archived copy (opens in a new tab) Public tasks resolved Reported October 7, 2026 Benchmark leaderboard Scale AI Independent Record for Claude Sonnet 4 on SWE-bench Pro Public — Scale AI SWE-Agent Toolathlon Original 29.9% toolathlon.xyz for Claude Sonnet 4 on Toolathlon Original (opens in a new tab) Archived copy (opens in a new tab) Tasks completed Reported October 28, 2025 Benchmark leaderboard Toolathlon Independent Toolathlon Authors Record for Claude Sonnet 4 on Toolathlon Original