Benchmark Registry

MMLU-Pro

MMLU-Pro results reported for 91 models across 1 version. Updated 2026-10-07.

Report an issue

Versions

1 version, plus 2 variants and configurations · Metric: Accuracy

VersionReleasedModelsResults
MMLU-Pro OriginalLatest
2 variants of Original
VariantConfigurationReleasedModelsResults
MMLU-Pro Original — Mercor single-shotMercor single-shot agentJune 3, 20243840
MMLU-Pro Original — Vals AI harnessVals AI harnessJune 3, 20247277
June 3, 20241924

Recently reported results: MMLU-Pro Original

Latest version: MMLU-Pro Original

ModelScoreSource
MAI-Thinking-185.0%microsoft.aiAccuracyReported Technical reportMicrosoft AISelf-reportedMicrosoftRecord
Gemma 4 31B Instruct (thinking mode)85.2%huggingface.coAccuracyReported Model cardGoogleSelf-reportedGoogle DeepMindRecord
Gemma 4 E2B Instruct (thinking mode)60.0%huggingface.coAccuracyReported Model cardGoogleSelf-reportedGoogle DeepMindRecord
Gemma 4 26B A4B Instruct (thinking mode)82.6%huggingface.coAccuracyReported Model cardGoogleSelf-reportedGoogle DeepMindRecord
Gemma 4 E4B Instruct (thinking mode)69.4%huggingface.coAccuracyReported Model cardGoogleSelf-reportedGoogle DeepMindRecord