Benchmarks

54 results across 16 benchmarks

Benchmark results for DeepSeek-V4-Pro
BenchmarkScoreSource
BrowseComp 130-question subset — Mercor web research agentEffort: MaxConfiguration: Mercor web research agent, 100 steps, 3 hours62.3%mercor.com for DeepSeek-V4-Pro (max) on BrowseComp 130-question subset — Mercor web research agent (opens in a new tab)Archived copy (opens in a new tab)BrowseComp AccuracyReported Benchmark leaderboardMercorIndependentRecord for DeepSeek-V4-Pro on BrowseComp 130-question subset — Mercor web research agent
BrowseCompEffort: High80.4%huggingface.co for DeepSeek-V4-Pro (high) on BrowseComp BrowseComp (opens in a new tab)BrowseComp AccuracyReported Model cardDeepSeekSelf-reportedDeepSeek AIRecord for DeepSeek-V4-Pro on BrowseComp BrowseComp
BrowseCompEffort: Max83.4%huggingface.co for DeepSeek-V4-Pro (max) on BrowseComp BrowseComp (opens in a new tab)BrowseComp AccuracyReported Model cardDeepSeekSelf-reportedDeepSeek AIRecord for DeepSeek-V4-Pro on BrowseComp BrowseComp
DeepSWE 1.1Effort: Max63.0%deepswe.datacurve.ai for DeepSeek-V4-Pro (max) on DeepSWE 1.1 (opens in a new tab)DeepSWE Pass@1Reported Benchmark leaderboardDatacurveIndependentRecord for DeepSeek-V4-Pro on DeepSWE 1.1
DeepSWE 1.1 — Mercor mini-swe-agent, 500 stepsEffort: MaxConfiguration: Mercor mini-swe-agent, 500 steps, 2 hours9.1%mercor.com for DeepSeek-V4-Pro (max) on DeepSWE 1.1 — Mercor mini-swe-agent, 500 steps (opens in a new tab)Archived copy (opens in a new tab)DeepSWE Pass@1Reported Benchmark leaderboardMercorIndependentRecord for DeepSeek-V4-Pro on DeepSWE 1.1 — Mercor mini-swe-agent, 500 steps
Finance Agent Benchmark 2 — October 2026 evaluationEffort: Max44.1%vals.ai for DeepSeek-V4-Pro (max) on Finance Agent Benchmark 2 — October 2026 evaluation (opens in a new tab)Archived copy (opens in a new tab)Finance Agent AccuracyReported Benchmark leaderboardVals AIIndependentRecord for DeepSeek-V4-Pro on Finance Agent Benchmark 2 — October 2026 evaluation
GPQA DiamondEffort: NoneReported as: non-thinking72.9%huggingface.co for DeepSeek-V4-Pro (non-thinking) on Graduate-Level Google-Proof Q&A Diamond (opens in a new tab)GPQA Diamond AccuracyReported Model cardDeepSeekSelf-reportedDeepSeek AIRecord for DeepSeek-V4-Pro on Graduate-Level Google-Proof Q&A Diamond
GPQA DiamondEffort: High89.1%huggingface.co for DeepSeek-V4-Pro (high) on Graduate-Level Google-Proof Q&A Diamond (opens in a new tab)GPQA Diamond AccuracyReported Model cardDeepSeekSelf-reportedDeepSeek AIRecord for DeepSeek-V4-Pro on Graduate-Level Google-Proof Q&A Diamond
GPQA DiamondEffort: Max90.1%huggingface.co for DeepSeek-V4-Pro (max) on Graduate-Level Google-Proof Q&A Diamond (opens in a new tab)GPQA Diamond AccuracyReported Model cardDeepSeekSelf-reportedDeepSeek AIRecord for DeepSeek-V4-Pro on Graduate-Level Google-Proof Q&A Diamond
GPQA Diamond — Epoch AI harnessEffort: NoneConfiguration: Epoch AI harness73.2%epoch.ai for DeepSeek-V4-Pro (none) on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness (opens in a new tab)Archived copy (opens in a new tab)GPQA Diamond AccuracyReported Evaluated Benchmark leaderboardEpoch AIIndependentRecord for DeepSeek-V4-Pro on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness
GPQA Diamond — Epoch AI harnessEffort: HighConfiguration: Epoch AI harness90.9%epoch.ai for DeepSeek-V4-Pro (high) on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness (opens in a new tab)Archived copy (opens in a new tab)GPQA Diamond AccuracyReported Evaluated Benchmark leaderboardEpoch AIIndependentRecord for DeepSeek-V4-Pro on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness
GPQA Diamond — Epoch AI harnessEffort: MaxConfiguration: Epoch AI harness89.6%epoch.ai for DeepSeek-V4-Pro (max) on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness (opens in a new tab)Archived copy (opens in a new tab)GPQA Diamond AccuracyReported Evaluated Benchmark leaderboardEpoch AIIndependentRecord for DeepSeek-V4-Pro on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness
GPQA Diamond — Mercor single-shotEffort: MaxConfiguration: Mercor single-shot agent89.4%mercor.com for DeepSeek-V4-Pro (max) on Graduate-Level Google-Proof Q&A Diamond — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)GPQA Diamond AccuracyReported Benchmark leaderboardMercorIndependentRecord for DeepSeek-V4-Pro on Graduate-Level Google-Proof Q&A Diamond — Mercor single-shot
GPQA Diamond — Vals AI harnessEffort: MaxConfiguration: Vals AI harness: mean of zero-shot and few-shot CoT89.4%vals.ai for DeepSeek-V4-Pro (max) on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)GPQA Diamond AccuracyReported Benchmark leaderboardVals AIIndependentRecord for DeepSeek-V4-Pro on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness
HLAB October 2026 evaluationEffort: Max3.8%vals.ai for DeepSeek-V4-Pro (max) on Harvey's Legal Agent Benchmark October 2026 evaluation (opens in a new tab)Archived copy (opens in a new tab)HLAB Task Pass RateReported Benchmark leaderboardVals AIIndependentRecord for DeepSeek-V4-Pro on Harvey's Legal Agent Benchmark October 2026 evaluation
Humanity's Last Exam Full set — No toolsEffort: MaxConfiguration: No tools37.7%huggingface.co for DeepSeek-V4-Pro (max) on Humanity's Last Exam Full set — No tools (opens in a new tab)Humanity's Last Exam AccuracyReported Model cardDeepSeekSelf-reportedDeepSeek AIRecord for DeepSeek-V4-Pro on Humanity's Last Exam Full set — No tools
Humanity's Last Exam Full set — With toolsEffort: MaxConfiguration: With tools48.2%huggingface.co for DeepSeek-V4-Pro (max) on Humanity's Last Exam Full set — With tools (opens in a new tab)Humanity's Last Exam AccuracyReported Model cardDeepSeekSelf-reportedDeepSeek AIRecord for DeepSeek-V4-Pro on Humanity's Last Exam Full set — With tools
Humanity's Last Exam Text only — Mercor single-shotEffort: MaxConfiguration: Mercor single-shot agent34.4%mercor.com for DeepSeek-V4-Pro (max) on Humanity's Last Exam Text only — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)Humanity's Last Exam AccuracyReported Benchmark leaderboardMercorIndependentRecord for DeepSeek-V4-Pro on Humanity's Last Exam Text only — Mercor single-shot
MMLU-Pro OriginalEffort: NoneReported as: non-thinking82.9%huggingface.co for DeepSeek-V4-Pro (non-thinking) on MMLU-Pro Original (opens in a new tab)AccuracyReported Model cardDeepSeekSelf-reportedDeepSeek AIRecord for DeepSeek-V4-Pro on MMLU-Pro Original
MMLU-Pro OriginalEffort: High87.1%huggingface.co for DeepSeek-V4-Pro (high) on MMLU-Pro Original (opens in a new tab)AccuracyReported Model cardDeepSeekSelf-reportedDeepSeek AIRecord for DeepSeek-V4-Pro on MMLU-Pro Original
MMLU-Pro OriginalEffort: Max87.5%huggingface.co for DeepSeek-V4-Pro (max) on MMLU-Pro Original (opens in a new tab)AccuracyReported Model cardDeepSeekSelf-reportedDeepSeek AIRecord for DeepSeek-V4-Pro on MMLU-Pro Original
MMLU-Pro Original — Mercor single-shotEffort: MaxConfiguration: Mercor single-shot agent86.5%mercor.com for DeepSeek-V4-Pro (max) on MMLU-Pro Original — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)AccuracyReported Benchmark leaderboardMercorIndependentRecord for DeepSeek-V4-Pro on MMLU-Pro Original — Mercor single-shot
MMLU-Pro Original — Vals AI harnessEffort: MaxConfiguration: Vals AI harness87.2%vals.ai for DeepSeek-V4-Pro (max) on MMLU-Pro Original — Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)AccuracyReported Benchmark leaderboardVals AIIndependentRecord for DeepSeek-V4-Pro on MMLU-Pro Original — Vals AI harness
Riemann-bench October 2026 leaderboardEffort: Max38.4%surgehq.ai for DeepSeek-V4-Pro (Max) on Riemann-bench October 2026 leaderboard (opens in a new tab)Archived copy (opens in a new tab)Riemann-bench Pass@1Reported Benchmark leaderboardSurge AIIndependentRecord for DeepSeek-V4-Pro on Riemann-bench October 2026 leaderboard
SimpleQA Verified OriginalEffort: NoneReported as: non-thinking45.0%huggingface.co for DeepSeek-V4-Pro (non-thinking) on SimpleQA Verified Original (opens in a new tab)AccuracyReported Model cardDeepSeekSelf-reportedDeepSeek AIRecord for DeepSeek-V4-Pro on SimpleQA Verified Original
SimpleQA Verified OriginalEffort: High46.2%huggingface.co for DeepSeek-V4-Pro (high) on SimpleQA Verified Original (opens in a new tab)AccuracyReported Model cardDeepSeekSelf-reportedDeepSeek AIRecord for DeepSeek-V4-Pro on SimpleQA Verified Original
SimpleQA Verified OriginalEffort: Max57.9%huggingface.co for DeepSeek-V4-Pro (max) on SimpleQA Verified Original (opens in a new tab)AccuracyReported Model cardDeepSeekSelf-reportedDeepSeek AIRecord for DeepSeek-V4-Pro on SimpleQA Verified Original
SimpleQA Verified Original — Epoch AI harnessEffort: MaxConfiguration: Epoch AI harness47.0%epoch.ai for DeepSeek-V4-Pro (max) on SimpleQA Verified Original — Epoch AI harness (opens in a new tab)Archived copy (opens in a new tab)AccuracyReported Evaluated Benchmark leaderboardEpoch AIIndependentRecord for DeepSeek-V4-Pro on SimpleQA Verified Original — Epoch AI harness
SWE Atlas Codebase QnA — Mercor mini-swe-agent, 250 stepsEffort: MaxConfiguration: Mercor mini-swe-agent, 250 steps, 1 hour25.0%mercor.com for DeepSeek-V4-Pro (max) on SWE Atlas Codebase QnA — Mercor mini-swe-agent, 250 steps (opens in a new tab)Archived copy (opens in a new tab)Mean pass@1Reported Benchmark leaderboardMercorIndependentRecord for DeepSeek-V4-Pro on SWE Atlas Codebase QnA — Mercor mini-swe-agent, 250 steps
SWE Atlas Codebase QnA — Public; mini-swe-agentConfiguration: mini-swe-agent harness27.2%labs.scale.com for DeepSeek-V4-Pro on SWE Atlas Codebase QnA — Public; mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Mean pass@1Reported Benchmark leaderboardScale AIIndependentRecord for DeepSeek-V4-Pro on SWE Atlas Codebase QnA — Public; mini-swe-agent
SWE-bench MultilingualEffort: NoneReported as: non-thinking69.8%huggingface.co for DeepSeek-V4-Pro (non-thinking) on SWE-bench Multilingual (opens in a new tab)Resolved TasksReported Model cardDeepSeekSelf-reportedDeepSeek AIRecord for DeepSeek-V4-Pro on SWE-bench Multilingual
SWE-bench MultilingualEffort: High74.1%huggingface.co for DeepSeek-V4-Pro (high) on SWE-bench Multilingual (opens in a new tab)Resolved TasksReported Model cardDeepSeekSelf-reportedDeepSeek AIRecord for DeepSeek-V4-Pro on SWE-bench Multilingual
SWE-bench MultilingualEffort: Max76.2%huggingface.co for DeepSeek-V4-Pro (max) on SWE-bench Multilingual (opens in a new tab)Resolved TasksReported Model cardDeepSeekSelf-reportedDeepSeek AIRecord for DeepSeek-V4-Pro on SWE-bench Multilingual
SWE-bench VerifiedEffort: NoneReported as: non-thinking73.6%huggingface.co for DeepSeek-V4-Pro (non-thinking) on SWE-bench Verified (opens in a new tab)Resolved TasksReported Model cardDeepSeekSelf-reportedDeepSeek AIRecord for DeepSeek-V4-Pro on SWE-bench Verified
SWE-bench VerifiedEffort: High79.4%huggingface.co for DeepSeek-V4-Pro (high) on SWE-bench Verified (opens in a new tab)Resolved TasksReported Model cardDeepSeekSelf-reportedDeepSeek AIRecord for DeepSeek-V4-Pro on SWE-bench Verified
SWE-bench VerifiedEffort: Max80.6%huggingface.co for DeepSeek-V4-Pro (max) on SWE-bench Verified (opens in a new tab)Resolved TasksReported Model cardDeepSeekSelf-reportedDeepSeek AIRecord for DeepSeek-V4-Pro on SWE-bench Verified
SWE-bench Verified — Epoch AI harnessEffort: MaxConfiguration: Epoch AI harness77.6%epoch.ai for DeepSeek-V4-Pro (max) on SWE-bench Verified — Epoch AI harness (opens in a new tab)Archived copy (opens in a new tab)Resolved TasksReported Evaluated Benchmark leaderboardEpoch AIIndependentRecord for DeepSeek-V4-Pro on SWE-bench Verified — Epoch AI harness
SWE-bench Verified — Mercor mini-swe-agent, 1000 stepsEffort: MaxConfiguration: Mercor mini-swe-agent, 1000 steps, 3 hours75.4%mercor.com for DeepSeek-V4-Pro (max) on SWE-bench Verified — Mercor mini-swe-agent, 1000 steps (opens in a new tab)Archived copy (opens in a new tab)Resolved TasksReported Benchmark leaderboardMercorIndependentRecord for DeepSeek-V4-Pro on SWE-bench Verified — Mercor mini-swe-agent, 1000 steps
SWE-bench Verified — Vals AI mini-swe-agentEffort: MaxConfiguration: mini-swe-agent harness77.4%vals.ai for DeepSeek-V4-Pro (max) on SWE-bench Verified — Vals AI mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Resolved TasksReported Benchmark leaderboardVals AIIndependentRecord for DeepSeek-V4-Pro on SWE-bench Verified — Vals AI mini-swe-agent
SWE-bench Pro PublicEffort: Max55.4%huggingface.co for DeepSeek-V4-Pro (max) on SWE-bench Pro Public (opens in a new tab)Public tasks resolvedReported Model cardDeepSeekSelf-reportedDeepSeek AIRecord for DeepSeek-V4-Pro on SWE-bench Pro Public
Terminal-Bench 2.0Effort: NoneReported as: non-thinking59.1%huggingface.co for DeepSeek-V4-Pro (non-thinking) on Terminal-Bench 2.0 (opens in a new tab)Terminal-Bench AccuracyReported Model cardDeepSeekSelf-reportedDeepSeek AIRecord for DeepSeek-V4-Pro on Terminal-Bench 2.0
Terminal-Bench 2.0Effort: High63.3%huggingface.co for DeepSeek-V4-Pro (high) on Terminal-Bench 2.0 (opens in a new tab)Terminal-Bench AccuracyReported Model cardDeepSeekSelf-reportedDeepSeek AIRecord for DeepSeek-V4-Pro on Terminal-Bench 2.0
Terminal-Bench 2.0Effort: Max67.9%huggingface.co for DeepSeek-V4-Pro (max) on Terminal-Bench 2.0 (opens in a new tab)Terminal-Bench AccuracyReported Model cardDeepSeekSelf-reportedDeepSeek AIRecord for DeepSeek-V4-Pro on Terminal-Bench 2.0
Terminal-Bench 2.0 — Vals AI Terminus 2Configuration: Terminus-2 harness56.2%vals.ai for DeepSeek-V4-Pro on Terminal-Bench 2.0 — Vals AI Terminus 2 (opens in a new tab)Archived copy (opens in a new tab)Terminal-Bench AccuracyReported Benchmark leaderboardVals AIIndependentRecord for DeepSeek-V4-Pro on Terminal-Bench 2.0 — Vals AI Terminus 2
Terminal-Bench 2.1 — Mercor Terminus 2Effort: MaxConfiguration: Mercor Terminus 2, 1000 steps, 3 hours66.7%mercor.com for DeepSeek-V4-Pro (max) on Terminal-Bench 2.1 — Mercor Terminus 2 (opens in a new tab)Archived copy (opens in a new tab)Terminal-Bench AccuracyReported Benchmark leaderboardMercorIndependentRecord for DeepSeek-V4-Pro on Terminal-Bench 2.1 — Mercor Terminus 2
Terminal-Bench 2.1 — Vals AI Terminus 2Configuration: Terminus-2 harness50.2%vals.ai for DeepSeek-V4-Pro on Terminal-Bench 2.1 — Vals AI Terminus 2 (opens in a new tab)Archived copy (opens in a new tab)Terminal-Bench AccuracyReported Benchmark leaderboardVals AIIndependentRecord for DeepSeek-V4-Pro on Terminal-Bench 2.1 — Vals AI Terminus 2
Terminal-Bench 4.0 — Vals AI mini-swe-agentEffort: MaxConfiguration: mini-swe-agent harness11.1%vals.ai for DeepSeek-V4-Pro (max) on Terminal-Bench 4.0 — Vals AI mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Terminal-Bench AccuracyReported Benchmark leaderboardVals AIIndependentRecord for DeepSeek-V4-Pro on Terminal-Bench 4.0 — Vals AI mini-swe-agent
Toolathlon OriginalEffort: NoneReported as: non-thinking46.3%huggingface.co for DeepSeek-V4-Pro (non-thinking) on Toolathlon Original (opens in a new tab)Tasks completedReported Model cardDeepSeekSelf-reportedDeepSeek AIRecord for DeepSeek-V4-Pro on Toolathlon Original
Toolathlon OriginalEffort: High49.0%huggingface.co for DeepSeek-V4-Pro (high) on Toolathlon Original (opens in a new tab)Tasks completedReported Model cardDeepSeekSelf-reportedDeepSeek AIRecord for DeepSeek-V4-Pro on Toolathlon Original
Toolathlon OriginalEffort: Max51.8%huggingface.co for DeepSeek-V4-Pro (max) on Toolathlon Original (opens in a new tab)Tasks completedReported Model cardDeepSeekSelf-reportedDeepSeek AIRecord for DeepSeek-V4-Pro on Toolathlon Original