Versions

1 version · Metric: Multiple-choice accuracy

Versions of MMMLU
VersionReleasedModelsResults
MMMLU OriginalLatestSeptember 13, 202455

Recently reported results: MMMLU Original

Latest version: MMMLU Original

Recently reported results for MMMLU Original
ModelScoreSource
Gemma 4 12B Instruct (thinking mode)83.4%arxiv.org for Gemma 4 12B Instruct (thinking mode) on MMMLU Original (opens in a new tab)Multiple-choice accuracyReported Technical reportGoogle DeepMindSelf-reportedRecord for Gemma 4 12B Instruct on MMMLU Original
Qwen3.7-Plus89.0%qwen.ai for Qwen3.7-Plus on MMMLU Original (opens in a new tab)Multiple-choice accuracyReported Launch postQwen TeamSelf-reportedRecord for Qwen3.7-Plus on MMMLU Original
Qwen3.7-Max90.3%qwen.ai for Qwen3.7-Max on MMMLU Original (opens in a new tab)Multiple-choice accuracyReported Launch postQwen TeamSelf-reportedRecord for Qwen3.7-Max on MMMLU Original
Claude Opus 4.7 (adaptive thinking, max)91.5%www-cdn.anthropic.com for Claude Opus 4.7 (adaptive thinking, max) on MMMLU Original (opens in a new tab)Multiple-choice accuracyReported System cardAnthropicSelf-reportedRecord for Claude Opus 4.7 on MMMLU Original
Qwen3.6-Plus89.5%qwen.ai for Qwen3.6-Plus on MMMLU Original (opens in a new tab)Multiple-choice accuracyReported Launch postQwen TeamSelf-reportedRecord for Qwen3.6-Plus on MMMLU Original