Benchmarks

22 results across 12 benchmarks

Benchmark results for DeepSeek-V3.2
BenchmarkScoreSource
APEX-Agents Original21.3%mercor.com for DeepSeek-V3.2 on APEX-Agents Original (opens in a new tab)Archived copy (opens in a new tab)Pass rateReported Benchmark leaderboardMercorIndependentRecord for DeepSeek-V3.2 on APEX-Agents Original
BrowseComp 130-question subset — Mercor web research agentConfiguration: Mercor web research agent, 100 steps, 3 hours38.1%mercor.com for DeepSeek-V3.2 on BrowseComp 130-question subset — Mercor web research agent (opens in a new tab)Archived copy (opens in a new tab)BrowseComp AccuracyReported Benchmark leaderboardMercorIndependentRecord for DeepSeek-V3.2 on BrowseComp 130-question subset — Mercor web research agent
DeepSWE 1.1 — Mercor mini-swe-agent, 500 stepsConfiguration: Mercor mini-swe-agent, 500 steps, 2 hours0.6%mercor.com for DeepSeek-V3.2 on DeepSWE 1.1 — Mercor mini-swe-agent, 500 steps (opens in a new tab)Archived copy (opens in a new tab)DeepSWE Pass@1Reported Benchmark leaderboardMercorIndependentRecord for DeepSeek-V3.2 on DeepSWE 1.1 — Mercor mini-swe-agent, 500 steps
GPQA Diamond — Mercor single-shotConfiguration: Mercor single-shot agent85.2%mercor.com for DeepSeek-V3.2 on Graduate-Level Google-Proof Q&A Diamond — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)GPQA Diamond AccuracyReported Benchmark leaderboardMercorIndependentRecord for DeepSeek-V3.2 on Graduate-Level Google-Proof Q&A Diamond — Mercor single-shot
GPQA Diamond — Vals AI harnessEffort: NoneConfiguration: Vals AI harness: mean of zero-shot and few-shot CoT76.3%vals.ai for DeepSeek-V3.2 (none) on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)GPQA Diamond AccuracyReported Benchmark leaderboardVals AIIndependentRecord for DeepSeek-V3.2 on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness
GPQA Diamond — Vals AI harnessEffort: HighConfiguration: Vals AI harness: mean of zero-shot and few-shot CoT80.3%vals.ai for DeepSeek-V3.2 (high) on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)GPQA Diamond AccuracyReported Benchmark leaderboardVals AIIndependentRecord for DeepSeek-V3.2 on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness
Humanity's Last Exam Text only — Mercor single-shotConfiguration: Mercor single-shot agent22.3%mercor.com for DeepSeek-V3.2 on Humanity's Last Exam Text only — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)Humanity's Last Exam AccuracyReported Benchmark leaderboardMercorIndependentRecord for DeepSeek-V3.2 on Humanity's Last Exam Text only — Mercor single-shot
MMLU-Pro Original — Mercor single-shotConfiguration: Mercor single-shot agent76.5%mercor.com for DeepSeek-V3.2 on MMLU-Pro Original — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)AccuracyReported Benchmark leaderboardMercorIndependentRecord for DeepSeek-V3.2 on MMLU-Pro Original — Mercor single-shot
MMLU-Pro Original — Vals AI harnessConfiguration: Vals AI harness83.1%vals.ai for DeepSeek-V3.2 on MMLU-Pro Original — Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)AccuracyReported Benchmark leaderboardVals AIIndependentRecord for DeepSeek-V3.2 on MMLU-Pro Original — Vals AI harness
MMLU-Pro Original — Vals AI harnessReported as: thinkingConfiguration: Vals AI harness84.9%vals.ai for DeepSeek-V3.2 (thinking) on MMLU-Pro Original — Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)AccuracyReported Benchmark leaderboardVals AIIndependentRecord for DeepSeek-V3.2 on MMLU-Pro Original — Vals AI harness
Riemann-bench October 2026 leaderboardEffort: High8.0%surgehq.ai for DeepSeek-V3.2 (High) on Riemann-bench October 2026 leaderboard (opens in a new tab)Archived copy (opens in a new tab)Riemann-bench Pass@1Reported Benchmark leaderboardSurge AIIndependentRecord for DeepSeek-V3.2 on Riemann-bench October 2026 leaderboard
SWE Atlas Codebase QnA — Mercor mini-swe-agent, 250 stepsConfiguration: Mercor mini-swe-agent, 250 steps, 1 hour5.9%mercor.com for DeepSeek-V3.2 on SWE Atlas Codebase QnA — Mercor mini-swe-agent, 250 steps (opens in a new tab)Archived copy (opens in a new tab)Mean pass@1Reported Benchmark leaderboardMercorIndependentRecord for DeepSeek-V3.2 on SWE Atlas Codebase QnA — Mercor mini-swe-agent, 250 steps
SWE-bench Multilingual — Mercor mini-swe-agent, 1000 stepsConfiguration: Mercor mini-swe-agent, 1000 steps, 3 hours60.7%mercor.com for DeepSeek-V3.2 on SWE-bench Multilingual — Mercor mini-swe-agent, 1000 steps (opens in a new tab)Archived copy (opens in a new tab)Resolved TasksReported Benchmark leaderboardMercorIndependentRecord for DeepSeek-V3.2 on SWE-bench Multilingual — Mercor mini-swe-agent, 1000 steps
SWE-bench Multilingual — mini-swe-agentConfiguration: mini-swe-agent harness59.0%github.com for DeepSeek-V3.2 on SWE-bench Multilingual — mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Resolved TasksReported Benchmark repositorySWE-benchIndependentSWE-bench TeamRecord for DeepSeek-V3.2 on SWE-bench Multilingual — mini-swe-agent
SWE-bench Verified — Mercor mini-swe-agent, 1000 stepsConfiguration: Mercor mini-swe-agent, 1000 steps, 3 hours66.0%mercor.com for DeepSeek-V3.2 on SWE-bench Verified — Mercor mini-swe-agent, 1000 steps (opens in a new tab)Archived copy (opens in a new tab)Resolved TasksReported Benchmark leaderboardMercorIndependentRecord for DeepSeek-V3.2 on SWE-bench Verified — Mercor mini-swe-agent, 1000 steps
SWE-bench Verified — mini-swe-agentEffort: HighConfiguration: mini-swe-agent harness70.0%github.com for DeepSeek-V3.2 (high) on SWE-bench Verified — mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Resolved TasksReported Benchmark repositorySWE-benchIndependentSWE-bench TeamRecord for DeepSeek-V3.2 on SWE-bench Verified — mini-swe-agent
SWE-bench Verified — Vals AI mini-swe-agentEffort: HighConfiguration: mini-swe-agent harness67.6%vals.ai for DeepSeek-V3.2 (high) on SWE-bench Verified — Vals AI mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Resolved TasksReported Benchmark leaderboardVals AIIndependentRecord for DeepSeek-V3.2 on SWE-bench Verified — Vals AI mini-swe-agent
SWE-bench Pro Public — Scale AI SWE-AgentConfiguration: SWE-Agent harness, 250 turns, uncapped cost15.6%labs.scale.com for DeepSeek-V3.2 on SWE-bench Pro Public — Scale AI SWE-Agent (opens in a new tab)Archived copy (opens in a new tab)Public tasks resolvedReported Benchmark leaderboardScale AIIndependentRecord for DeepSeek-V3.2 on SWE-bench Pro Public — Scale AI SWE-Agent
Terminal-Bench 2.0 — Vals AI Terminus 2Effort: NoneConfiguration: Terminus-2 harness34.8%vals.ai for DeepSeek-V3.2 (none) on Terminal-Bench 2.0 — Vals AI Terminus 2 (opens in a new tab)Archived copy (opens in a new tab)Terminal-Bench AccuracyReported Benchmark leaderboardVals AIIndependentRecord for DeepSeek-V3.2 on Terminal-Bench 2.0 — Vals AI Terminus 2
Terminal-Bench 2.0 — Vals AI Terminus 2Effort: HighConfiguration: Terminus-2 harness36.0%vals.ai for DeepSeek-V3.2 (high) on Terminal-Bench 2.0 — Vals AI Terminus 2 (opens in a new tab)Archived copy (opens in a new tab)Terminal-Bench AccuracyReported Benchmark leaderboardVals AIIndependentRecord for DeepSeek-V3.2 on Terminal-Bench 2.0 — Vals AI Terminus 2
Terminal-Bench 2.1 — Mercor Terminus 2Configuration: Mercor Terminus 2, 1000 steps, 3 hours44.6%mercor.com for DeepSeek-V3.2 on Terminal-Bench 2.1 — Mercor Terminus 2 (opens in a new tab)Archived copy (opens in a new tab)Terminal-Bench AccuracyReported Benchmark leaderboardMercorIndependentRecord for DeepSeek-V3.2 on Terminal-Bench 2.1 — Mercor Terminus 2
Vibe Code Bench 1.1 — OpenHands, October 2026 evaluationEffort: HighConfiguration: OpenHands harness5.1%vals.ai for DeepSeek-V3.2 (high) on Vibe Code Bench 1.1 — OpenHands, October 2026 evaluation (opens in a new tab)Archived copy (opens in a new tab)Vibe Code Bench ScoreReported Benchmark leaderboardVals AIIndependentRecord for DeepSeek-V3.2 on Vibe Code Bench 1.1 — OpenHands, October 2026 evaluation