Versions

5 versions · Metric: MMMU-Pro accuracy

Versions of MMMU-Pro
VersionReleasedModelsResults
MMMU-Pro With toolsConfiguration: With toolsMarch 5, 202666
MMMU-Pro With PythonConfiguration: With PythonDecember 11, 202533
MMMU-Pro 1,730 questions — Mercor single-shotConfiguration: Mercor single-shot agentSeptember 4, 20243234
MMMU-Pro No toolsConfiguration: No toolsSeptember 4, 20241718
MMMU-Pro Vals AI harnessConfiguration: Vals AI harnessSeptember 4, 20245560

Settings

Each score is labelled with the setting its source states.

Recently reported results: MMMU-Pro With tools

Most recent configuration: MMMU-Pro With tools

Recently reported results for MMMU-Pro With tools
ModelScoreSource
Kimi K3 (max)83.4%huggingface.co for Kimi K3 (max) on MMMU-Pro With tools (opens in a new tab)MMMU-Pro accuracySetting: Not statedReported Model cardMoonshot AISelf-reportedRecord for Kimi K3 on MMMU-Pro With tools
GPT-5.6 Terra82.0%openai.com for GPT-5.6 Terra on MMMU-Pro With tools (opens in a new tab)MMMU-Pro accuracySetting: Not statedReported Launch postOpenAISelf-reportedRecord for GPT-5.6 Terra on MMMU-Pro With tools
GPT-5.6 Luna79.5%openai.com for GPT-5.6 Luna on MMMU-Pro With tools (opens in a new tab)MMMU-Pro accuracySetting: Not statedReported Launch postOpenAISelf-reportedRecord for GPT-5.6 Luna on MMMU-Pro With tools
GPT-5.6 Sol84.6%openai.com for GPT-5.6 Sol on MMMU-Pro With tools (opens in a new tab)MMMU-Pro accuracySetting: Not statedReported Launch postOpenAISelf-reportedRecord for GPT-5.6 Sol on MMMU-Pro With tools
GPT-5.5 (xhigh)83.2%openai.com for GPT-5.5 (xhigh) on MMMU-Pro With tools (opens in a new tab)MMMU-Pro accuracySetting: Not statedReported Launch postOpenAISelf-reportedRecord for GPT-5.5 on MMMU-Pro With tools