Benchmarks

36 results across 18 benchmarks

Benchmark results for Muse Spark
BenchmarkScoreSource
CharXiv Reasoning — No toolsReported as: thinkingConfiguration: No tools86.4%ai.meta.com for Muse Spark (thinking) on CharXiv Reasoning — No tools (opens in a new tab)CharXiv AccuracyReported Evaluation methodologyMetaSelf-reportedRecord for Muse Spark on CharXiv Reasoning — No tools
CharXiv Reasoning — Python enabledReported as: thinkingConfiguration: With Python88.9%ai.meta.com for Muse Spark (thinking) on CharXiv Reasoning — Python enabled (opens in a new tab)CharXiv AccuracyReported Evaluation methodologyMetaSelf-reportedRecord for Muse Spark on CharXiv Reasoning — Python enabled
CyberGym OriginalEffort: High43.5%ai.meta.com for Muse Spark (high) on CyberGym Original (opens in a new tab)Vulnerability reproduction successReported Evaluation reportMetaSelf-reportedRecord for Muse Spark on CyberGym Original
CyberGym OriginalReported as: thinking43.5%ai.meta.com for Muse Spark (thinking) on CyberGym Original (opens in a new tab)Vulnerability reproduction successReported Safety reportMetaSelf-reportedRecord for Muse Spark on CyberGym Original
DeepSearchQA 900-prompt setReported as: thinking74.8%ai.meta.com for Muse Spark (thinking) on DeepSearchQA 900-prompt set (opens in a new tab)F1 scoreReported Evaluation methodologyMetaSelf-reportedRecord for Muse Spark on DeepSearchQA 900-prompt set
ERQA No toolsReported as: thinkingConfiguration: No tools64.7%ai.meta.com for Muse Spark (thinking) on Embodied Reasoning Question Answer No tools (opens in a new tab)Multiple-choice accuracyReported Evaluation methodologyMetaSelf-reportedRecord for Muse Spark on Embodied Reasoning Question Answer No tools
ERQA With PythonReported as: thinkingConfiguration: With Python67.0%ai.meta.com for Muse Spark (thinking) on Embodied Reasoning Question Answer With Python (opens in a new tab)Multiple-choice accuracyReported Evaluation methodologyMetaSelf-reportedRecord for Muse Spark on Embodied Reasoning Question Answer With Python
FrontierScience ResearchReported as: contemplating38.3%ai.meta.com for Muse Spark (contemplating) on FrontierScience Research (opens in a new tab)Research accuracy (at least 7/10 rubric points)Reported Launch postMetaSelf-reportedRecord for Muse Spark on FrontierScience Research
GPQA DiamondReported as: thinking89.5%ai.meta.com for Muse Spark (thinking) on Graduate-Level Google-Proof Q&A Diamond (opens in a new tab)GPQA Diamond AccuracyReported Evaluation methodologyMetaSelf-reportedRecord for Muse Spark on Graduate-Level Google-Proof Q&A Diamond
GPQA Diamond — Epoch AI harnessConfiguration: Epoch AI harness89.8%epoch.ai for Muse Spark on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness (opens in a new tab)Archived copy (opens in a new tab)GPQA Diamond AccuracyReported Evaluated Benchmark leaderboardEpoch AIIndependentRecord for Muse Spark on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness
GPQA Diamond — Vals AI harnessConfiguration: Vals AI harness: mean of zero-shot and few-shot CoT89.6%vals.ai for Muse Spark on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)GPQA Diamond AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Muse Spark on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness
HealthBench HealthBench HardReported as: thinking42.8%ai.meta.com for Muse Spark (thinking) on HealthBench HealthBench Hard (opens in a new tab)HealthBench ScoreReported Evaluation methodologyMetaSelf-reportedRecord for Muse Spark on HealthBench HealthBench Hard
Humanity's Last Exam Full set — No toolsReported as: contemplatingConfiguration: No tools50.2%ai.meta.com for Muse Spark (contemplating) on Humanity's Last Exam Full set — No tools (opens in a new tab)Humanity's Last Exam AccuracyReported Launch postMetaSelf-reportedRecord for Muse Spark on Humanity's Last Exam Full set — No tools
Humanity's Last Exam Full set — No toolsReported as: thinkingConfiguration: No tools42.8%ai.meta.com for Muse Spark (thinking) on Humanity's Last Exam Full set — No tools (opens in a new tab)Humanity's Last Exam AccuracyReported Evaluation methodologyMetaSelf-reportedRecord for Muse Spark on Humanity's Last Exam Full set — No tools
Humanity's Last Exam Full set — Scale AI evaluation40.6%labs.scale.com for Muse Spark on Humanity's Last Exam Full set — Scale AI evaluation (opens in a new tab)Archived copy (opens in a new tab)Humanity's Last Exam AccuracyReported Benchmark leaderboardScale AIIndependentRecord for Muse Spark on Humanity's Last Exam Full set — Scale AI evaluation
Humanity's Last Exam Full set — With toolsReported as: contemplatingConfiguration: With tools58.4%ai.meta.com for Muse Spark (contemplating) on Humanity's Last Exam Full set — With tools (opens in a new tab)Humanity's Last Exam AccuracyReported Launch postMetaSelf-reportedRecord for Muse Spark on Humanity's Last Exam Full set — With tools
Humanity's Last Exam Full set — With toolsReported as: thinkingConfiguration: With tools50.4%ai.meta.com for Muse Spark (thinking) on Humanity's Last Exam Full set — With tools (opens in a new tab)Humanity's Last Exam AccuracyReported Evaluation methodologyMetaSelf-reportedRecord for Muse Spark on Humanity's Last Exam Full set — With tools
Humanity's Last Exam Text only — Scale AI evaluation40.9%labs.scale.com for Muse Spark on Humanity's Last Exam Text only — Scale AI evaluation (opens in a new tab)Archived copy (opens in a new tab)Humanity's Last Exam AccuracyReported Benchmark leaderboardScale AIIndependentRecord for Muse Spark on Humanity's Last Exam Text only — Scale AI evaluation
International Physics Olympiad 2025 TheoryReported as: contemplating82.6%ai.meta.com for Muse Spark (contemplating) on International Physics Olympiad 2025 Theory (opens in a new tab)Percentage of theoretical-exam rubric pointsReported Launch postMetaSelf-reportedRecord for Muse Spark on International Physics Olympiad 2025 Theory
MCP Atlas Public, April 2026 update82.2%labs.scale.com for Muse Spark on MCP Atlas Public, April 2026 update (opens in a new tab)Pass rateReported Benchmark leaderboardScale AIIndependentRecord for Muse Spark on MCP Atlas Public, April 2026 update
MedXpertQA MMReported as: thinking78.4%ai.meta.com for Muse Spark (thinking) on MedXpertQA MM (opens in a new tab)AccuracyReported Evaluation methodologyMetaSelf-reportedRecord for Muse Spark on MedXpertQA MM
MedXpertQA TextReported as: thinking52.6%ai.meta.com for Muse Spark (thinking) on MedXpertQA Text (opens in a new tab)Text multiple-choice accuracyReported Evaluation methodologyMetaSelf-reportedRecord for Muse Spark on MedXpertQA Text
MMLU-Pro Original — Vals AI harnessConfiguration: Vals AI harness87.3%vals.ai for Muse Spark on MMLU-Pro Original — Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Muse Spark on MMLU-Pro Original — Vals AI harness
MMMU-Pro No toolsReported as: thinkingConfiguration: No tools80.4%ai.meta.com for Muse Spark (thinking) on MMMU-Pro No tools (opens in a new tab)MMMU-Pro accuracySetting: Standard (10 options)Reported Evaluation methodologyMetaSelf-reportedRecord for Muse Spark on MMMU-Pro No tools
MMMU-Pro Vals AI harnessConfiguration: Vals AI harness87.4%vals.ai for Muse Spark on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for Muse Spark on MMMU-Pro Vals AI harness
MMMU-Pro With PythonReported as: thinkingConfiguration: With Python81.3%ai.meta.com for Muse Spark (thinking) on MMMU-Pro With Python (opens in a new tab)MMMU-Pro accuracySetting: Standard (10 options)Reported Evaluation methodologyMetaSelf-reportedRecord for Muse Spark on MMMU-Pro With Python
ScreenSpot-Pro No toolsReported as: thinkingConfiguration: No tools72.2%ai.meta.com for Muse Spark (thinking) on ScreenSpot-Pro No tools (opens in a new tab)Point-in-box accuracyReported Evaluation methodologyMetaSelf-reportedRecord for Muse Spark on ScreenSpot-Pro No tools
ScreenSpot-Pro With PythonReported as: thinkingConfiguration: With Python84.1%ai.meta.com for Muse Spark (thinking) on ScreenSpot-Pro With Python (opens in a new tab)Point-in-box accuracyReported Evaluation methodologyMetaSelf-reportedRecord for Muse Spark on ScreenSpot-Pro With Python
SWE-bench Verified — Muse Spark corrected testsReported as: thinkingConfiguration: Muse Spark corrected tests77.4%ai.meta.com for Muse Spark (thinking) on SWE-bench Verified — Muse Spark corrected tests (opens in a new tab)Resolved TasksReported Evaluation methodologyMetaSelf-reportedRecord for Muse Spark on SWE-bench Verified — Muse Spark corrected tests
SWE-bench Verified — Vals AI mini-swe-agentConfiguration: mini-swe-agent harness74.4%vals.ai for Muse Spark on SWE-bench Verified — Vals AI mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Resolved TasksReported Benchmark leaderboardVals AIIndependentRecord for Muse Spark on SWE-bench Verified — Vals AI mini-swe-agent
SWE-bench Pro Private — Scale AI mini-swe-agentConfiguration: mini-swe-agent harness44.7%labs.scale.com for Muse Spark on SWE-bench Pro Private — Scale AI mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Public tasks resolvedReported Benchmark leaderboardScale AIIndependentRecord for Muse Spark on SWE-bench Pro Private — Scale AI mini-swe-agent
SWE-bench Pro PublicReported as: thinking52.4%ai.meta.com for Muse Spark (thinking) on SWE-bench Pro Public (opens in a new tab)Public tasks resolvedReported Evaluation methodologyMetaSelf-reportedRecord for Muse Spark on SWE-bench Pro Public
SWE-bench Pro Public — Scale AI mini-swe-agentConfiguration: mini-swe-agent harness55.0%labs.scale.com for Muse Spark on SWE-bench Pro Public — Scale AI mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Public tasks resolvedReported Benchmark leaderboardScale AIIndependentRecord for Muse Spark on SWE-bench Pro Public — Scale AI mini-swe-agent
Terminal-Bench 2.0Reported as: thinking59.0%ai.meta.com for Muse Spark (thinking) on Terminal-Bench 2.0 (opens in a new tab)Terminal-Bench AccuracyReported Evaluation methodologyMetaSelf-reportedRecord for Muse Spark on Terminal-Bench 2.0
Terminal-Bench 2.0 — Vals AI Terminus 2Configuration: Terminus-2 harness59.6%vals.ai for Muse Spark on Terminal-Bench 2.0 — Vals AI Terminus 2 (opens in a new tab)Archived copy (opens in a new tab)Terminal-Bench AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Muse Spark on Terminal-Bench 2.0 — Vals AI Terminus 2
Vibe Code Bench 1.1 — OpenHands, October 2026 evaluationConfiguration: OpenHands harness19.7%vals.ai for Muse Spark on Vibe Code Bench 1.1 — OpenHands, October 2026 evaluation (opens in a new tab)Archived copy (opens in a new tab)Vibe Code Bench ScoreReported Benchmark leaderboardVals AIIndependentRecord for Muse Spark on Vibe Code Bench 1.1 — OpenHands, October 2026 evaluation