Benchmarks

16 results across 11 benchmarks

Benchmark results for Mistral Medium 3.5
BenchmarkScoreSource
BeyondAIME OriginalEffort: High66.9%mistral.ai for Mistral Medium 3.5 (high) on BeyondAIME Original (opens in a new tab)Exact-answer accuracyReported Launch postMistral AISelf-reportedRecord for Mistral Medium 3.5 on BeyondAIME Original
BrowseCompEffort: High48.6%huggingface.co for Mistral Medium 3.5 (high) on BrowseComp BrowseComp (opens in a new tab)BrowseComp AccuracyReported Model cardMistral AISelf-reportedRecord for Mistral Medium 3.5 on BrowseComp BrowseComp
Finance Agent Benchmark 2 — October 2026 evaluationEffort: High32.1%vals.ai for Mistral Medium 3.5 (high) on Finance Agent Benchmark 2 — October 2026 evaluation (opens in a new tab)Archived copy (opens in a new tab)Finance Agent AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Mistral Medium 3.5 on Finance Agent Benchmark 2 — October 2026 evaluation
GPQA Diamond — Vals AI harnessEffort: HighConfiguration: Vals AI harness: mean of zero-shot and few-shot CoT34.8%vals.ai for Mistral Medium 3.5 (high) on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)GPQA Diamond AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Mistral Medium 3.5 on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness
HLAB October 2026 evaluationEffort: High0.4%vals.ai for Mistral Medium 3.5 (high) on Harvey's Legal Agent Benchmark October 2026 evaluation (opens in a new tab)Archived copy (opens in a new tab)HLAB Task Pass RateReported Benchmark leaderboardVals AIIndependentRecord for Mistral Medium 3.5 on Harvey's Legal Agent Benchmark October 2026 evaluation
IFBench OriginalEffort: High68.0%huggingface.co for Mistral Medium 3.5 (high) on IFBench Original (opens in a new tab)AccuracyReported Model cardMistral AISelf-reportedRecord for Mistral Medium 3.5 on IFBench Original
MMLU-Pro Original — Vals AI harnessEffort: HighConfiguration: Vals AI harness75.3%vals.ai for Mistral Medium 3.5 (high) on MMLU-Pro Original — Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Mistral Medium 3.5 on MMLU-Pro Original — Vals AI harness
SWE-bench VerifiedEffort: High77.6%huggingface.co for Mistral Medium 3.5 (high) on SWE-bench Verified (opens in a new tab)Resolved TasksReported Model cardMistral AISelf-reportedRecord for Mistral Medium 3.5 on SWE-bench Verified
SWE-bench Verified — Vals AI mini-swe-agentEffort: HighConfiguration: mini-swe-agent harness66.4%vals.ai for Mistral Medium 3.5 (high) on SWE-bench Verified — Vals AI mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Resolved TasksReported Benchmark leaderboardVals AIIndependentRecord for Mistral Medium 3.5 on SWE-bench Verified — Vals AI mini-swe-agent
Terminal-Bench 2.0 — Vals AI Terminus 2Effort: HighConfiguration: Terminus-2 harness30.3%vals.ai for Mistral Medium 3.5 (high) on Terminal-Bench 2.0 — Vals AI Terminus 2 (opens in a new tab)Archived copy (opens in a new tab)Terminal-Bench AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Mistral Medium 3.5 on Terminal-Bench 2.0 — Vals AI Terminus 2
Terminal-Bench 2.1 — Vals AI Terminus 2Effort: HighConfiguration: Terminus-2 harness39.0%vals.ai for Mistral Medium 3.5 (high) on Terminal-Bench 2.1 — Vals AI Terminus 2 (opens in a new tab)Archived copy (opens in a new tab)Terminal-Bench AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Mistral Medium 3.5 on Terminal-Bench 2.1 — Vals AI Terminus 2
Vibe Code Bench 1.1 — OpenHands, October 2026 evaluationEffort: HighConfiguration: OpenHands harness2.9%vals.ai for Mistral Medium 3.5 (high) on Vibe Code Bench 1.1 — OpenHands, October 2026 evaluation (opens in a new tab)Archived copy (opens in a new tab)Vibe Code Bench ScoreReported Benchmark leaderboardVals AIIndependentRecord for Mistral Medium 3.5 on Vibe Code Bench 1.1 — OpenHands, October 2026 evaluation
τ³-bench τ³ — AirlineEffort: High72.0%huggingface.co for Mistral Medium 3.5 (high) on τ³-bench τ³ — Airline (opens in a new tab)Task success rateReported Model cardMistral AISelf-reportedRecord for Mistral Medium 3.5 on τ³-bench τ³ — Airline
τ³-bench τ³ — BankingEffort: High13.4%huggingface.co for Mistral Medium 3.5 (high) on τ³-bench τ³ — Banking (opens in a new tab)Task success rateReported Model cardMistral AISelf-reportedRecord for Mistral Medium 3.5 on τ³-bench τ³ — Banking
τ³-bench τ³ — RetailEffort: High76.1%huggingface.co for Mistral Medium 3.5 (high) on τ³-bench τ³ — Retail (opens in a new tab)Task success rateReported Model cardMistral AISelf-reportedRecord for Mistral Medium 3.5 on τ³-bench τ³ — Retail
τ³-bench τ³ — TelecomEffort: High91.4%huggingface.co for Mistral Medium 3.5 (high) on τ³-bench τ³ — Telecom (opens in a new tab)Task success rateReported Model cardMistral AISelf-reportedRecord for Mistral Medium 3.5 on τ³-bench τ³ — Telecom