Results

60 results

MMMU-Pro Vals AI harness results
ProviderModelScoreSourceRegistry No.
OpenAIGPT-4.1 (high)72.4%vals.ai for GPT-4.1 (high) on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for GPT-4.1 on MMMU-Pro Vals AI harness10002
OpenAIOpenAI o3 (high)80.4%vals.ai for OpenAI o3 (high) on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for OpenAI o3 on MMMU-Pro Vals AI harness10003
OpenAIOpenAI o4-mini (high)79.7%vals.ai for OpenAI o4-mini (high) on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for OpenAI o4-mini on MMMU-Pro Vals AI harness10004
OpenAIGPT-5.4 (xhigh)87.5%vals.ai for GPT-5.4 (xhigh) on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for GPT-5.4 on MMMU-Pro Vals AI harness10007
OpenAIGPT-5.5 (xhigh)88.3%vals.ai for GPT-5.5 (xhigh) on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for GPT-5.5 on MMMU-Pro Vals AI harness10008
OpenAIGPT-5.6 Luna (max)85.0%vals.ai for GPT-5.6 Luna (max) on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for GPT-5.6 Luna on MMMU-Pro Vals AI harness10009
OpenAIGPT-5.6 Sol (max)88.8%vals.ai for GPT-5.6 Sol (max) on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for GPT-5.6 Sol on MMMU-Pro Vals AI harness10010
OpenAIGPT-5.6 Terra (max)86.5%vals.ai for GPT-5.6 Terra (max) on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for GPT-5.6 Terra on MMMU-Pro Vals AI harness10011
OpenAIGPT-5.2 (xhigh)86.7%vals.ai for GPT-5.2 (xhigh) on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for GPT-5.2 on MMMU-Pro Vals AI harness10012
OpenAIGPT-4.1 mini (high)70.5%vals.ai for GPT-4.1 mini (high) on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for GPT-4.1 mini on MMMU-Pro Vals AI harness10016
OpenAIGPT-4.1 nano (high)55.1%vals.ai for GPT-4.1 nano (high) on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for GPT-4.1 nano on MMMU-Pro Vals AI harness10017
OpenAIGPT-5 (high)81.5%vals.ai for GPT-5 (high) on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for GPT-5 on MMMU-Pro Vals AI harness10018
OpenAIGPT-5 mini (high)78.9%vals.ai for GPT-5 mini (high) on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for GPT-5 mini on MMMU-Pro Vals AI harness10019
OpenAIGPT-5 nano (high)70.9%vals.ai for GPT-5 nano (high) on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for GPT-5 nano on MMMU-Pro Vals AI harness10020
OpenAIGPT-5.4 mini (xhigh)79.2%vals.ai for GPT-5.4 mini (xhigh) on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for GPT-5.4 mini on MMMU-Pro Vals AI harness10021
OpenAIGPT-5.4 nano (high)73.6%vals.ai for GPT-5.4 nano (high) on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for GPT-5.4 nano on MMMU-Pro Vals AI harness10022
Moonshot AIKimi K2.5 (thinking)84.3%vals.ai for Kimi K2.5 (thinking) on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for Kimi K2.5 on MMMU-Pro Vals AI harness120001
Moonshot AIKimi K2.686.3%vals.ai for Kimi K2.6 on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for Kimi K2.6 on MMMU-Pro Vals AI harness120002
Moonshot AIKimi K388.2%vals.ai for Kimi K3 on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for Kimi K3 on MMMU-Pro Vals AI harness120003
Alibaba (Tongyi)Qwen3.6-Plus84.2%vals.ai for Qwen3.6-Plus on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for Qwen3.6-Plus on MMMU-Pro Vals AI harness130004
Alibaba (Tongyi)Qwen3.8-Max88.0%vals.ai for Qwen3.8-Max on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for Qwen3.8-Max on MMMU-Pro Vals AI harness130008
MiniMaxMiniMax M381.2%vals.ai for MiniMax M3 on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for MiniMax M3 on MMMU-Pro Vals AI harness140003
Z.aiGLM-5.3-Flash (max)86.0%vals.ai for GLM-5.3-Flash (max) on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for GLM-5.3-Flash on MMMU-Pro Vals AI harness150006
Thinking Machines LabInkling (0.99)78.5%vals.ai for Inkling (0.99) on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for Inkling on MMMU-Pro Vals AI harness160001
AnthropicClaude 3.7 Sonnet71.5%vals.ai for Claude 3.7 Sonnet on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for Claude 3.7 Sonnet on MMMU-Pro Vals AI harness20001
AnthropicClaude 3.7 Sonnet (thinking)75.1%vals.ai for Claude 3.7 Sonnet (thinking) on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for Claude 3.7 Sonnet on MMMU-Pro Vals AI harness20001
AnthropicClaude Opus 473.3%vals.ai for Claude Opus 4 on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for Claude Opus 4 on MMMU-Pro Vals AI harness20002
AnthropicClaude Sonnet 4 (thinking)74.9%vals.ai for Claude Sonnet 4 (thinking) on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for Claude Sonnet 4 on MMMU-Pro Vals AI harness20003
AnthropicClaude Sonnet 472.4%vals.ai for Claude Sonnet 4 on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for Claude Sonnet 4 on MMMU-Pro Vals AI harness20003
AnthropicClaude Fable 5.190.6%vals.ai for Claude Fable 5.1 on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for Claude Fable 5.1 on MMMU-Pro Vals AI harness20004
AnthropicClaude Sonnet 4.5 (thinking)79.3%vals.ai for Claude Sonnet 4.5 (thinking) on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for Claude Sonnet 4.5 on MMMU-Pro Vals AI harness20005
AnthropicClaude Haiku 4.5 (thinking)46.1%vals.ai for Claude Haiku 4.5 (thinking) on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for Claude Haiku 4.5 on MMMU-Pro Vals AI harness20006
AnthropicClaude Opus 4.581.1%vals.ai for Claude Opus 4.5 on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for Claude Opus 4.5 on MMMU-Pro Vals AI harness20007
AnthropicClaude Opus 4.5 (thinking)82.9%vals.ai for Claude Opus 4.5 (thinking) on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for Claude Opus 4.5 on MMMU-Pro Vals AI harness20007
AnthropicClaude Opus 4.6 (thinking)83.9%vals.ai for Claude Opus 4.6 (thinking) on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for Claude Opus 4.6 on MMMU-Pro Vals AI harness20008
AnthropicClaude Sonnet 4.683.6%vals.ai for Claude Sonnet 4.6 on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for Claude Sonnet 4.6 on MMMU-Pro Vals AI harness20009
AnthropicClaude Opus 4.785.5%vals.ai for Claude Opus 4.7 on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for Claude Opus 4.7 on MMMU-Pro Vals AI harness20010
AnthropicClaude Opus 4.886.6%vals.ai for Claude Opus 4.8 on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for Claude Opus 4.8 on MMMU-Pro Vals AI harness20011
AnthropicClaude Fable 589.3%vals.ai for Claude Fable 5 on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for Claude Fable 5 on MMMU-Pro Vals AI harness20012
AnthropicClaude Sonnet 583.0%vals.ai for Claude Sonnet 5 on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for Claude Sonnet 5 on MMMU-Pro Vals AI harness20013
AnthropicClaude Opus 589.9%vals.ai for Claude Opus 5 on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for Claude Opus 5 on MMMU-Pro Vals AI harness20014
AnthropicClaude Opus 4.1 (thinking)77.5%vals.ai for Claude Opus 4.1 (thinking) on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for Claude Opus 4.1 on MMMU-Pro Vals AI harness20017
AnthropicClaude Opus 4.173.7%vals.ai for Claude Opus 4.1 on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for Claude Opus 4.1 on MMMU-Pro Vals AI harness20017
Google (DeepMind)Gemini 2.0 Flash69.8%vals.ai for Gemini 2.0 Flash on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for Gemini 2.0 Flash on MMMU-Pro Vals AI harness30001
Google (DeepMind)Gemini 2.5 Pro81.3%vals.ai for Gemini 2.5 Pro on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for Gemini 2.5 Pro on MMMU-Pro Vals AI harness30002
Google (DeepMind)Gemini 2.5 Flash68.0%vals.ai for Gemini 2.5 Flash on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for Gemini 2.5 Flash on MMMU-Pro Vals AI harness30003
Google (DeepMind)Gemini 2.5 Flash (thinking)71.9%vals.ai for Gemini 2.5 Flash (thinking) on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for Gemini 2.5 Flash on MMMU-Pro Vals AI harness30003
Google (DeepMind)Gemini 3.8 Flash (high)89.1%vals.ai for Gemini 3.8 Flash (high) on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for Gemini 3.8 Flash on MMMU-Pro Vals AI harness30004
Google (DeepMind)Gemini 3.1 Pro (high)88.2%vals.ai for Gemini 3.1 Pro (high) on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for Gemini 3.1 Pro on MMMU-Pro Vals AI harness30005
Google (DeepMind)Gemini 3.5 Flash (high)88.3%vals.ai for Gemini 3.5 Flash (high) on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for Gemini 3.5 Flash on MMMU-Pro Vals AI harness30006