Benchmarks

9 results across 9 benchmarks

Benchmark results for Gemma 4 12B Instruct
BenchmarkScoreSource
BIG-Bench Extra Hard Original — micro averageReported as: thinking mode53.0%huggingface.co for Gemma 4 12B Instruct (thinking mode) on BIG-Bench Extra Hard Original — micro average (opens in a new tab)Micro-average accuracyReported Model cardGoogleSelf-reportedGoogle DeepMindRecord for Gemma 4 12B Instruct on BIG-Bench Extra Hard Original — micro average
GPQA DiamondReported as: thinking mode78.8%huggingface.co for Gemma 4 12B Instruct (thinking mode) on Graduate-Level Google-Proof Q&A Diamond (opens in a new tab)GPQA Diamond AccuracyReported Model cardGoogleSelf-reportedGoogle DeepMindRecord for Gemma 4 12B Instruct on Graduate-Level Google-Proof Q&A Diamond
IFBench OriginalReported as: thinking mode74.0%arxiv.org for Gemma 4 12B Instruct (thinking mode) on IFBench Original (opens in a new tab)AccuracyReported Technical reportGoogle DeepMindSelf-reportedRecord for Gemma 4 12B Instruct on IFBench Original
IFEval OriginalReported as: thinking mode97.2%huggingface.co for Gemma 4 12B Instruct (thinking mode) on IFEval Original (opens in a new tab)AccuracyReported Model cardGoogleSelf-reportedGoogle DeepMindRecord for Gemma 4 12B Instruct on IFEval Original
MATH-Vision OriginalReported as: thinking mode79.7%huggingface.co for Gemma 4 12B Instruct (thinking mode) on MATH-Vision Original (opens in a new tab)AccuracyReported Model cardGoogleSelf-reportedGoogle DeepMindRecord for Gemma 4 12B Instruct on MATH-Vision Original
MedXpertQA MMReported as: thinking mode48.7%huggingface.co for Gemma 4 12B Instruct (thinking mode) on MedXpertQA MM (opens in a new tab)AccuracyReported Model cardGoogleSelf-reportedGoogle DeepMindRecord for Gemma 4 12B Instruct on MedXpertQA MM
MMLU-Pro OriginalReported as: thinking mode77.2%huggingface.co for Gemma 4 12B Instruct (thinking mode) on MMLU-Pro Original (opens in a new tab)AccuracyReported Model cardGoogleSelf-reportedGoogle DeepMindRecord for Gemma 4 12B Instruct on MMLU-Pro Original
MMMLU OriginalReported as: thinking mode83.4%arxiv.org for Gemma 4 12B Instruct (thinking mode) on MMMLU Original (opens in a new tab)Multiple-choice accuracyReported Technical reportGoogle DeepMindSelf-reportedRecord for Gemma 4 12B Instruct on MMMLU Original
SciCode OriginalReported as: thinking mode38.0%arxiv.org for Gemma 4 12B Instruct (thinking mode) on SciCode Original (opens in a new tab)AccuracyReported Technical reportGoogle DeepMindSelf-reportedRecord for Gemma 4 12B Instruct on SciCode Original