Benchmarks

42 results across 28 benchmarks

Benchmark results for Qwen3.8-Max
BenchmarkScoreSource
APEX-Agents OriginalEffort: xHigh63.3%mercor.com for Qwen3.8-Max (xhigh) on APEX-Agents Original (opens in a new tab)Archived copy (opens in a new tab)Pass rateReported Benchmark leaderboardMercorIndependentRecord for Qwen3.8-Max on APEX-Agents Original
BrowseComp 130-question subset — Mercor web research agentEffort: xHighConfiguration: Mercor web research agent, 100 steps, 3 hours67.9%mercor.com for Qwen3.8-Max (xhigh) on BrowseComp 130-question subset — Mercor web research agent (opens in a new tab)Archived copy (opens in a new tab)BrowseComp AccuracyReported Benchmark leaderboardMercorIndependentRecord for Qwen3.8-Max on BrowseComp 130-question subset — Mercor web research agent
Chartography October 2026 leaderboardEffort: xHigh29.1%surgehq.ai for Qwen3.8-Max (xHigh) on Chartography October 2026 leaderboard (opens in a new tab)Archived copy (opens in a new tab)Chartography Pass@1Reported Benchmark leaderboardSurge AIIndependentRecord for Qwen3.8-Max on Chartography October 2026 leaderboard
CharXiv Descriptive and reasoning — Mercor single-shotEffort: xHighConfiguration: Mercor single-shot agent94.1%mercor.com for Qwen3.8-Max (xhigh) on CharXiv Descriptive and reasoning — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)CharXiv AccuracyReported Benchmark leaderboardMercorIndependentRecord for Qwen3.8-Max on CharXiv Descriptive and reasoning — Mercor single-shot
DeepSWE 1.1Effort: xHigh57.0%deepswe.datacurve.ai for Qwen3.8-Max (xhigh) on DeepSWE 1.1 (opens in a new tab)DeepSWE Pass@1Reported Benchmark leaderboardDatacurveIndependentRecord for Qwen3.8-Max on DeepSWE 1.1
DeepSWE 1.156.6%qwen.ai for Qwen3.8-Max on DeepSWE 1.1 (opens in a new tab)DeepSWE Pass@1Reported Launch postQwen TeamSelf-reportedRecord for Qwen3.8-Max on DeepSWE 1.1
DeepSWE 1.1 — Mercor mini-swe-agent, 500 stepsEffort: xHighConfiguration: Mercor mini-swe-agent, 500 steps, 2 hours53.1%mercor.com for Qwen3.8-Max (xhigh) on DeepSWE 1.1 — Mercor mini-swe-agent, 500 steps (opens in a new tab)Archived copy (opens in a new tab)DeepSWE Pass@1Reported Benchmark leaderboardMercorIndependentRecord for Qwen3.8-Max on DeepSWE 1.1 — Mercor mini-swe-agent, 500 steps
Finance Agent Benchmark 2 — October 2026 evaluation50.6%vals.ai for Qwen3.8-Max on Finance Agent Benchmark 2 — October 2026 evaluation (opens in a new tab)Archived copy (opens in a new tab)Finance Agent AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Qwen3.8-Max on Finance Agent Benchmark 2 — October 2026 evaluation
FrontierSWE 2 — October 2026 evaluationConfiguration: Proximus harness, 5 trials, 20-hour budget15.8%frontierswe.com for Qwen3.8-Max on FrontierSWE 2 — October 2026 evaluation (opens in a new tab)Archived copy (opens in a new tab)FrontierSWE ScoreReported Benchmark leaderboardProximalIndependentRecord for Qwen3.8-Max on FrontierSWE 2 — October 2026 evaluation
GPQA Diamond92.6%huggingface.co for Qwen3.8-Max on Graduate-Level Google-Proof Q&A Diamond (opens in a new tab)GPQA Diamond AccuracyReported Model cardQwenSelf-reportedQwen TeamRecord for Qwen3.8-Max on Graduate-Level Google-Proof Q&A Diamond
GPQA Diamond — Epoch AI harnessEffort: xHighConfiguration: Epoch AI harness92.7%epoch.ai for Qwen3.8-Max (xhigh) on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness (opens in a new tab)Archived copy (opens in a new tab)GPQA Diamond AccuracyReported Evaluated Benchmark leaderboardEpoch AIIndependentRecord for Qwen3.8-Max on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness
GPQA Diamond — Mercor single-shotEffort: xHighConfiguration: Mercor single-shot agent92.5%mercor.com for Qwen3.8-Max (xhigh) on Graduate-Level Google-Proof Q&A Diamond — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)GPQA Diamond AccuracyReported Benchmark leaderboardMercorIndependentRecord for Qwen3.8-Max on Graduate-Level Google-Proof Q&A Diamond — Mercor single-shot
GPQA Diamond — Vals AI harnessConfiguration: Vals AI harness: mean of zero-shot and few-shot CoT93.7%vals.ai for Qwen3.8-Max on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)GPQA Diamond AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Qwen3.8-Max on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness
HLAB October 2026 evaluation10.4%vals.ai for Qwen3.8-Max on Harvey's Legal Agent Benchmark October 2026 evaluation (opens in a new tab)Archived copy (opens in a new tab)HLAB Task Pass RateReported Benchmark leaderboardVals AIIndependentRecord for Qwen3.8-Max on Harvey's Legal Agent Benchmark October 2026 evaluation
HealthBench60.2%huggingface.co for Qwen3.8-Max on HealthBench HealthBench (opens in a new tab)HealthBench ScoreReported Model cardQwenSelf-reportedQwen TeamRecord for Qwen3.8-Max on HealthBench HealthBench
Humanity's Last Exam Full set — No toolsConfiguration: No tools43.6%huggingface.co for Qwen3.8-Max on Humanity's Last Exam Full set — No tools (opens in a new tab)Humanity's Last Exam AccuracyReported Model cardQwenSelf-reportedQwen TeamRecord for Qwen3.8-Max on Humanity's Last Exam Full set — No tools
Humanity's Last Exam Full set — With toolsConfiguration: With tools56.2%huggingface.co for Qwen3.8-Max on Humanity's Last Exam Full set — With tools (opens in a new tab)Humanity's Last Exam AccuracyReported Model cardQwenSelf-reportedQwen TeamRecord for Qwen3.8-Max on Humanity's Last Exam Full set — With tools
Humanity's Last Exam Text only — Mercor single-shotEffort: xHighConfiguration: Mercor single-shot agent42.2%mercor.com for Qwen3.8-Max (xhigh) on Humanity's Last Exam Text only — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)Humanity's Last Exam AccuracyReported Benchmark leaderboardMercorIndependentRecord for Qwen3.8-Max on Humanity's Last Exam Text only — Mercor single-shot
IFBench Original82.8%huggingface.co for Qwen3.8-Max on IFBench Original (opens in a new tab)AccuracyReported Model cardQwenSelf-reportedQwen TeamRecord for Qwen3.8-Max on IFBench Original
LongBench v266.3%qwen.ai for Qwen3.8-Max on LongBench v2 (opens in a new tab)Multiple-choice accuracyReported Launch postQwen TeamSelf-reportedRecord for Qwen3.8-Max on LongBench v2
MedXpertQA MM80.4%qwen.ai for Qwen3.8-Max on MedXpertQA MM (opens in a new tab)AccuracyReported Launch postQwen TeamSelf-reportedRecord for Qwen3.8-Max on MedXpertQA MM
MedXpertQA MM — Mercor single-shotEffort: xHighConfiguration: Mercor single-shot agent78.3%mercor.com for Qwen3.8-Max (xhigh) on MedXpertQA MM — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)AccuracyReported Benchmark leaderboardMercorIndependentRecord for Qwen3.8-Max on MedXpertQA MM — Mercor single-shot
MMLU-Pro Original — Mercor single-shotEffort: xHighConfiguration: Mercor single-shot agent88.8%mercor.com for Qwen3.8-Max (xhigh) on MMLU-Pro Original — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)AccuracyReported Benchmark leaderboardMercorIndependentRecord for Qwen3.8-Max on MMLU-Pro Original — Mercor single-shot
MMLU-Pro Original — Vals AI harnessConfiguration: Vals AI harness88.6%vals.ai for Qwen3.8-Max on MMLU-Pro Original — Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Qwen3.8-Max on MMLU-Pro Original — Vals AI harness
MMMU-Pro 1,730 questions — Mercor single-shotEffort: xHighConfiguration: Mercor single-shot agent82.8%mercor.com for Qwen3.8-Max (xhigh) on MMMU-Pro 1,730 questions — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Not statedReported Benchmark leaderboardMercorIndependentRecord for Qwen3.8-Max on MMMU-Pro 1,730 questions — Mercor single-shot
MMMU-Pro Vals AI harnessConfiguration: Vals AI harness88.0%vals.ai for Qwen3.8-Max on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for Qwen3.8-Max on MMMU-Pro Vals AI harness
MTVQA Test56.6%qwen.ai for Qwen3.8-Max on MTVQA Test (opens in a new tab)Mean answer accuracyReported Launch postQwen TeamSelf-reportedRecord for Qwen3.8-Max on MTVQA Test
OSWorld Verified86.1%qwen.ai for Qwen3.8-Max on OSWorld Verified (opens in a new tab)OSWorld-Verified success rateReported Launch postQwen TeamSelf-reportedRecord for Qwen3.8-Max on OSWorld Verified
PaperBench Code-Dev93.0%qwen.ai for Qwen3.8-Max on PaperBench Code-Dev (opens in a new tab)Average replication scoreReported Launch postQwen TeamSelf-reportedRecord for Qwen3.8-Max on PaperBench Code-Dev
PerceptionBench Original63.5%qwen.ai for Qwen3.8-Max on PerceptionBench Original (opens in a new tab)Answer accuracyReported Launch postQwen TeamSelf-reportedRecord for Qwen3.8-Max on PerceptionBench Original
RealWorldQA Original88.0%qwen.ai for Qwen3.8-Max on RealWorldQA Original (opens in a new tab)Answer accuracyReported Launch postQwen TeamSelf-reportedRecord for Qwen3.8-Max on RealWorldQA Original
Riemann-bench October 2026 leaderboardEffort: xHigh15.2%surgehq.ai for Qwen3.8-Max (xHigh) on Riemann-bench October 2026 leaderboard (opens in a new tab)Archived copy (opens in a new tab)Riemann-bench Pass@1Reported Benchmark leaderboardSurge AIIndependentRecord for Qwen3.8-Max on Riemann-bench October 2026 leaderboard
SimpleQA Verified Original — Epoch AI harnessEffort: xHighConfiguration: Epoch AI harness45.8%epoch.ai for Qwen3.8-Max (xhigh) on SimpleQA Verified Original — Epoch AI harness (opens in a new tab)Archived copy (opens in a new tab)AccuracyReported Evaluated Benchmark leaderboardEpoch AIIndependentRecord for Qwen3.8-Max on SimpleQA Verified Original — Epoch AI harness
SWE Atlas Codebase QnA — Mercor mini-swe-agent, 250 stepsEffort: xHighConfiguration: Mercor mini-swe-agent, 250 steps, 1 hour50.5%mercor.com for Qwen3.8-Max (xhigh) on SWE Atlas Codebase QnA — Mercor mini-swe-agent, 250 steps (opens in a new tab)Archived copy (opens in a new tab)Mean pass@1Reported Benchmark leaderboardMercorIndependentRecord for Qwen3.8-Max on SWE Atlas Codebase QnA — Mercor mini-swe-agent, 250 steps
SWE-bench Multilingual — Mercor mini-swe-agent, 1000 stepsEffort: xHighConfiguration: Mercor mini-swe-agent, 1000 steps, 3 hours88.3%mercor.com for Qwen3.8-Max (xhigh) on SWE-bench Multilingual — Mercor mini-swe-agent, 1000 steps (opens in a new tab)Archived copy (opens in a new tab)Resolved TasksReported Benchmark leaderboardMercorIndependentRecord for Qwen3.8-Max on SWE-bench Multilingual — Mercor mini-swe-agent, 1000 steps
SWE-bench Verified — Mercor mini-swe-agent, 1000 stepsEffort: xHighConfiguration: Mercor mini-swe-agent, 1000 steps, 3 hours83.6%mercor.com for Qwen3.8-Max (xhigh) on SWE-bench Verified — Mercor mini-swe-agent, 1000 steps (opens in a new tab)Archived copy (opens in a new tab)Resolved TasksReported Benchmark leaderboardMercorIndependentRecord for Qwen3.8-Max on SWE-bench Verified — Mercor mini-swe-agent, 1000 steps
SWE-bench Verified — Vals AI mini-swe-agentConfiguration: mini-swe-agent harness85.6%vals.ai for Qwen3.8-Max on SWE-bench Verified — Vals AI mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Resolved TasksReported Benchmark leaderboardVals AIIndependentRecord for Qwen3.8-Max on SWE-bench Verified — Vals AI mini-swe-agent
Terminal-Bench 2.186.6%huggingface.co for Qwen3.8-Max on Terminal-Bench 2.1 (opens in a new tab)Terminal-Bench AccuracyReported Model cardQwenSelf-reportedQwen TeamRecord for Qwen3.8-Max on Terminal-Bench 2.1
Terminal-Bench 2.1 — Vals AI Terminus 2Configuration: Terminus-2 harness67.4%vals.ai for Qwen3.8-Max on Terminal-Bench 2.1 — Vals AI Terminus 2 (opens in a new tab)Archived copy (opens in a new tab)Terminal-Bench AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Qwen3.8-Max on Terminal-Bench 2.1 — Vals AI Terminus 2
Terminal-Bench 4.0 — Vals AI mini-swe-agentConfiguration: mini-swe-agent harness34.3%vals.ai for Qwen3.8-Max on Terminal-Bench 4.0 — Vals AI mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Terminal-Bench AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Qwen3.8-Max on Terminal-Bench 4.0 — Vals AI mini-swe-agent
Vals Index 2.148.3%vals.ai for Qwen3.8-Max on Vals Index 2.1 (opens in a new tab)Archived copy (opens in a new tab)Vals Index GDP-weighted AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Qwen3.8-Max on Vals Index 2.1
Vibe Code Bench 1.1 — OpenHands, October 2026 evaluationConfiguration: OpenHands harness64.7%vals.ai for Qwen3.8-Max on Vibe Code Bench 1.1 — OpenHands, October 2026 evaluation (opens in a new tab)Archived copy (opens in a new tab)Vibe Code Bench ScoreReported Benchmark leaderboardVals AIIndependentRecord for Qwen3.8-Max on Vibe Code Bench 1.1 — OpenHands, October 2026 evaluation