Benchmark Registry

Mistral AI

Benchmark results for 5 Mistral AI models from official publications. Updated 2026-10-07.

Established
Not available
Latest model
Mistral Medium 3.5 (May 2026)

Benchmarks

42 results

ModelBenchmarkScoreSourceRegistry No.
Mistral Large 3Chartography October 2026 leaderboard9.0%surgehq.aiArchived copyChartography Pass@1Reported Benchmark leaderboardSurge AIIndependentRecord90001
Mistral Small 4 (high)Vibe Code Bench 1.1 — OpenHands, October 2026 evaluation0.0%vals.aiArchived copyVibe Code Bench ScoreReported Benchmark leaderboardVals AIIndependentRecord90004
Mistral Medium 3.5 (high)HLAB October 2026 evaluation0.4%vals.aiArchived copyHLAB Task Pass RateReported Benchmark leaderboardVals AIIndependentRecord90005
Mistral Medium 3.5 (high)Vibe Code Bench 1.1 — OpenHands, October 2026 evaluation2.9%vals.aiArchived copyVibe Code Bench ScoreReported Benchmark leaderboardVals AIIndependentRecord90005
Mistral Medium 3.5 (high)Finance Agent Benchmark 2 — October 2026 evaluation32.1%vals.aiArchived copyFinance Agent AccuracyReported Benchmark leaderboardVals AIIndependentRecord90005
Mistral Medium 3.5 (high)Terminal-Bench 2.1 — Vals AI Terminus 239.0%vals.aiArchived copyTerminal-Bench AccuracyReported Benchmark leaderboardVals AIIndependentRecord90005
Mistral Large 3Arena Text Text Overall — 2026-09-13 snapshot1413.0arena.aiArena EloReported Benchmark leaderboardArena IntelligenceIndependentArenaRecord90001
Mistral Large 3SWE-bench Verified — Vals AI mini-swe-agent41.4%vals.aiArchived copyResolved TasksReported Benchmark leaderboardVals AIIndependentRecord90001
Mistral Large 3MMLU-Pro Original — Vals AI harness79.8%vals.aiArchived copyAccuracyReported Benchmark leaderboardVals AIIndependentRecord90001
Mistral Large 3GPQA Diamond — Vals AI harness68.4%vals.aiArchived copyGPQA Diamond AccuracyReported Benchmark leaderboardVals AIIndependentRecord90001
Mistral Large 3MMMU-Pro Vals AI harness66.2%vals.aiArchived copyMMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord90001
Devstral 2SWE-bench Verified — Vals AI mini-swe-agent62.8%vals.aiArchived copyResolved TasksReported Benchmark leaderboardVals AIIndependentRecord90002
Mistral Medium 3.5 (high)SWE-bench Verified — Vals AI mini-swe-agent66.4%vals.aiArchived copyResolved TasksReported Benchmark leaderboardVals AIIndependentRecord90005
Mistral Medium 3.5 (high)MMLU-Pro Original — Vals AI harness75.3%vals.aiArchived copyAccuracyReported Benchmark leaderboardVals AIIndependentRecord90005
Mistral Medium 3.5 (high)GPQA Diamond — Vals AI harness34.8%vals.aiArchived copyGPQA Diamond AccuracyReported Benchmark leaderboardVals AIIndependentRecord90005
Mistral Large 3Terminal-Bench 2.0 — Vals AI Terminus 29.0%vals.aiArchived copyTerminal-Bench AccuracyReported Benchmark leaderboardVals AIIndependentRecord90001
Mistral Small 4 (high)Terminal-Bench 2.0 — Vals AI Terminus 216.9%vals.aiArchived copyTerminal-Bench AccuracyReported Benchmark leaderboardVals AIIndependentRecord90004
Mistral Medium 3.5 (high)Terminal-Bench 2.0 — Vals AI Terminus 230.3%vals.aiArchived copyTerminal-Bench AccuracyReported Benchmark leaderboardVals AIIndependentRecord90005
Mistral Medium 3.5 (high)τ³-bench τ³ — Retail76.1%huggingface.coTask success rateReported Model cardMistral AISelf-reportedRecord90005
Mistral Medium 3.5 (high)τ³-bench τ³ — Telecom91.4%huggingface.coTask success rateReported Model cardMistral AISelf-reportedRecord90005
Mistral Medium 3.5 (high)SWE-bench Verified77.6%huggingface.coResolved TasksReported Model cardMistral AISelf-reportedRecord90005
Mistral Medium 3.5 (high)τ³-bench τ³ — Banking13.4%huggingface.coTask success rateReported Model cardMistral AISelf-reportedRecord90005
Mistral Medium 3.5 (high)BrowseComp48.6%huggingface.coBrowseComp AccuracyReported Model cardMistral AISelf-reportedRecord90005
Mistral Medium 3.5 (high)BeyondAIME Original66.9%mistral.aiExact-answer accuracyReported Launch postMistral AISelf-reportedRecord90005
Mistral Medium 3.5 (high)τ³-bench τ³ — Airline72.0%huggingface.coTask success rateReported Model cardMistral AISelf-reportedRecord90005
Mistral Medium 3.5 (high)IFBench Original68.0%huggingface.coAccuracyReported Model cardMistral AISelf-reportedRecord90005
Mistral Small 4 (high)IFBench Original48.0%huggingface.coAccuracyReported Model cardMistral AISelf-reportedRecord90004
Mistral Small 4 (none)GPQA Diamond59.1%huggingface.coGPQA Diamond AccuracyReported Model cardMistral AISelf-reportedRecord90004
Mistral Small 4 (high)MMLU-Pro Original78.0%huggingface.coAccuracyReported Model cardMistral AISelf-reportedRecord90004
Mistral Small 4 (none)IFBench Original35.7%huggingface.coAccuracyReported Model cardMistral AISelf-reportedRecord90004
Mistral Small 4 (none)MMLU-Pro Original73.5%huggingface.coAccuracyReported Model cardMistral AISelf-reportedRecord90004
Mistral Small 4 (high)MMMU-Pro No tools60.0%huggingface.coMMMU-Pro accuracySetting: Not statedReported Model cardMistral AISelf-reportedRecord90004
Mistral Small 4 (high)GPQA Diamond71.2%huggingface.coGPQA Diamond AccuracyReported Model cardMistral AISelf-reportedRecord90004
Mistral Small 4 (none)MMMU-Pro No tools46.3%huggingface.coMMMU-Pro accuracySetting: Not statedReported Model cardMistral AISelf-reportedRecord90004
Devstral 2SWE-bench Multilingual61.3%huggingface.coResolved TasksReported Model cardMistral AISelf-reportedRecord90002
Devstral 2SWE-bench Verified — mini-swe-agent53.8%github.comArchived copyResolved TasksReported Benchmark repositorySWE-benchIndependentSWE-bench TeamRecord90002
Devstral 2Terminal-Bench 2.032.6%huggingface.coTerminal-Bench AccuracyReported Model cardMistral AISelf-reportedRecord90002
Devstral 2SWE-bench Verified72.2%huggingface.coResolved TasksReported Model cardMistral AISelf-reportedRecord90002
Devstral Small 2SWE-bench Verified68.0%huggingface.coResolved TasksReported Model cardMistral AISelf-reportedRecord90003
Devstral Small 2SWE-bench Multilingual55.7%huggingface.coResolved TasksReported Model cardMistral AISelf-reportedRecord90003
Devstral Small 2SWE-bench Verified — mini-swe-agent56.4%github.comArchived copyResolved TasksReported Benchmark repositorySWE-benchIndependentSWE-bench TeamRecord90003
Devstral Small 2Terminal-Bench 2.022.5%huggingface.coTerminal-Bench AccuracyReported Model cardMistral AISelf-reportedRecord90003

Mistral AI models