Benchmarks

29 results across 23 benchmarks

Benchmark results for Muse Spark 1.2
BenchmarkScoreSource
APEX-Agents OriginalEffort: xHigh36.4%mercor.com for Muse Spark 1.2 (xhigh) on APEX-Agents Original (opens in a new tab)Archived copy (opens in a new tab)Pass rateReported Benchmark leaderboardMercorIndependentRecord for Muse Spark 1.2 on APEX-Agents Original
BrowseComp 130-question subset — Mercor web research agentEffort: xHighConfiguration: Mercor web research agent, 100 steps, 3 hours75.2%mercor.com for Muse Spark 1.2 (xhigh) on BrowseComp 130-question subset — Mercor web research agent (opens in a new tab)Archived copy (opens in a new tab)BrowseComp AccuracyReported Benchmark leaderboardMercorIndependentRecord for Muse Spark 1.2 on BrowseComp 130-question subset — Mercor web research agent
Chartography October 2026 leaderboardEffort: xHigh32.1%surgehq.ai for Muse Spark 1.2 (xHigh) on Chartography October 2026 leaderboard (opens in a new tab)Archived copy (opens in a new tab)Chartography Pass@1Reported Benchmark leaderboardSurge AIIndependentRecord for Muse Spark 1.2 on Chartography October 2026 leaderboard
CharXiv Descriptive and reasoning — Mercor single-shotEffort: xHighConfiguration: Mercor single-shot agent95.3%mercor.com for Muse Spark 1.2 (xhigh) on CharXiv Descriptive and reasoning — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)CharXiv AccuracyReported Benchmark leaderboardMercorIndependentRecord for Muse Spark 1.2 on CharXiv Descriptive and reasoning — Mercor single-shot
DeepSearchQA 900-prompt setEffort: xHigh85.9%dev.meta.ai for Muse Spark 1.2 (xhigh) on DeepSearchQA 900-prompt set (opens in a new tab)F1 scoreReported Model pageMetaSelf-reportedRecord for Muse Spark 1.2 on DeepSearchQA 900-prompt set
DeepSWE 1.1 — mini-swe-agentEffort: xHighConfiguration: mini-swe-agent harness55.0%research.meta.ai for Muse Spark 1.2 (xhigh) on DeepSWE 1.1 — mini-swe-agent (opens in a new tab)DeepSWE Pass@1Reported Launch postMetaSelf-reportedDataCurveRecord for Muse Spark 1.2 on DeepSWE 1.1 — mini-swe-agent
DeepSWE 1.1 — Muse CodeEffort: xHighConfiguration: Muse Code harness59.3%research.meta.ai for Muse Spark 1.2 (xhigh) on DeepSWE 1.1 — Muse Code (opens in a new tab)DeepSWE Pass@1Reported Launch postMetaSelf-reportedRecord for Muse Spark 1.2 on DeepSWE 1.1 — Muse Code
Finance Agent Benchmark 2 — October 2026 evaluationEffort: xHigh60.6%vals.ai for Muse Spark 1.2 (xhigh) on Finance Agent Benchmark 2 — October 2026 evaluation (opens in a new tab)Archived copy (opens in a new tab)Finance Agent AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Muse Spark 1.2 on Finance Agent Benchmark 2 — October 2026 evaluation
FrontierSWE 2 — October 2026 evaluationConfiguration: Proximus harness, 5 trials, 20-hour budget12.0%frontierswe.com for Muse Spark 1.2 on FrontierSWE 2 — October 2026 evaluation (opens in a new tab)Archived copy (opens in a new tab)FrontierSWE ScoreReported Benchmark leaderboardProximalIndependentRecord for Muse Spark 1.2 on FrontierSWE 2 — October 2026 evaluation
HLAB October 2026 evaluationEffort: xHigh25.4%vals.ai for Muse Spark 1.2 (xhigh) on Harvey's Legal Agent Benchmark October 2026 evaluation (opens in a new tab)Archived copy (opens in a new tab)HLAB Task Pass RateReported Benchmark leaderboardVals AIIndependentRecord for Muse Spark 1.2 on Harvey's Legal Agent Benchmark October 2026 evaluation
JobBench Main 65 tasksEffort: xHigh61.6%dev.meta.ai for Muse Spark 1.2 (xhigh) on JobBench Main 65 tasks (opens in a new tab)Mean rubric scoreReported Model pageMetaSelf-reportedRecord for Muse Spark 1.2 on JobBench Main 65 tasks
MedXpertQA MM — Mercor single-shotEffort: xHighConfiguration: Mercor single-shot agent75.7%mercor.com for Muse Spark 1.2 (xhigh) on MedXpertQA MM — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)AccuracyReported Benchmark leaderboardMercorIndependentRecord for Muse Spark 1.2 on MedXpertQA MM — Mercor single-shot
MMLU-Pro Original — Vals AI harnessEffort: xHighConfiguration: Vals AI harness88.3%vals.ai for Muse Spark 1.2 (xhigh) on MMLU-Pro Original — Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Muse Spark 1.2 on MMLU-Pro Original — Vals AI harness
MMMU-Pro 1,730 questions — Mercor single-shotEffort: xHighConfiguration: Mercor single-shot agent80.7%mercor.com for Muse Spark 1.2 (xhigh) on MMMU-Pro 1,730 questions — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Not statedReported Benchmark leaderboardMercorIndependentRecord for Muse Spark 1.2 on MMMU-Pro 1,730 questions — Mercor single-shot
MMMU-Pro Vals AI harnessEffort: xHighConfiguration: Vals AI harness86.1%vals.ai for Muse Spark 1.2 (xhigh) on MMMU-Pro Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Standard (4 options)Reported Benchmark leaderboardVals AIIndependentRecord for Muse Spark 1.2 on MMMU-Pro Vals AI harness
MRCR v2 8-needle 256K–512K (o200k_base)Effort: xHigh66.3%dev.meta.ai for Muse Spark 1.2 (xhigh) on MRCR v2 8-needle 256K–512K (o200k_base) (opens in a new tab)Mean sequence-matcher ratioReported Model pageMetaSelf-reportedRecord for Muse Spark 1.2 on MRCR v2 8-needle 256K–512K (o200k_base)
MRCR v2 8-needle 512K–1M (o200k_base)Effort: xHigh55.5%dev.meta.ai for Muse Spark 1.2 (xhigh) on MRCR v2 8-needle 512K–1M (o200k_base) (opens in a new tab)Mean sequence-matcher ratioReported Model pageMetaSelf-reportedRecord for Muse Spark 1.2 on MRCR v2 8-needle 512K–1M (o200k_base)
OSWorld 2.0 v2026.06.24 — BinaryEffort: xHigh17.9%dev.meta.ai for Muse Spark 1.2 (xhigh) on OSWorld 2.0 v2026.06.24 — Binary (opens in a new tab)OSWorld Strict ScoreReported Model pageMetaSelf-reportedRecord for Muse Spark 1.2 on OSWorld 2.0 v2026.06.24 — Binary
OSWorld 2.0 v2026.06.24 — PartialEffort: xHigh47.6%dev.meta.ai for Muse Spark 1.2 (xhigh) on OSWorld 2.0 v2026.06.24 — Partial (opens in a new tab)OSWorld Partial ScoreReported Model pageMetaSelf-reportedRecord for Muse Spark 1.2 on OSWorld 2.0 v2026.06.24 — Partial
Riemann-bench October 2026 leaderboardEffort: xHigh23.2%surgehq.ai for Muse Spark 1.2 (xHigh) on Riemann-bench October 2026 leaderboard (opens in a new tab)Archived copy (opens in a new tab)Riemann-bench Pass@1Reported Benchmark leaderboardSurge AIIndependentRecord for Muse Spark 1.2 on Riemann-bench October 2026 leaderboard
SimpleQA Verified Original — Epoch AI harnessEffort: xHighConfiguration: Epoch AI harness60.3%epoch.ai for Muse Spark 1.2 (xhigh) on SimpleQA Verified Original — Epoch AI harness (opens in a new tab)Archived copy (opens in a new tab)AccuracyReported Evaluated Benchmark leaderboardEpoch AIIndependentRecord for Muse Spark 1.2 on SimpleQA Verified Original — Epoch AI harness
SWE Atlas Codebase QnA — Public; mini-swe-agentEffort: xHighConfiguration: mini-swe-agent harness46.2%dev.meta.ai for Muse Spark 1.2 (xhigh) on SWE Atlas Codebase QnA — Public; mini-swe-agent (opens in a new tab)Mean pass@1Reported Model pageMetaSelf-reportedRecord for Muse Spark 1.2 on SWE Atlas Codebase QnA — Public; mini-swe-agent
SWE-bench Verified — Vals AI mini-swe-agentEffort: xHighConfiguration: mini-swe-agent harness86.6%vals.ai for Muse Spark 1.2 (xhigh) on SWE-bench Verified — Vals AI mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Resolved TasksReported Benchmark leaderboardVals AIIndependentRecord for Muse Spark 1.2 on SWE-bench Verified — Vals AI mini-swe-agent
Terminal-Bench 2.1 — Muse CodeEffort: xHighConfiguration: Muse Code harness82.9%research.meta.ai for Muse Spark 1.2 (xhigh) on Terminal-Bench 2.1 — Muse Code (opens in a new tab)Terminal-Bench AccuracyReported Launch postMetaSelf-reportedRecord for Muse Spark 1.2 on Terminal-Bench 2.1 — Muse Code
Terminal-Bench 2.1 — Vals AI Terminus 2Effort: xHighConfiguration: Terminus-2 harness69.7%vals.ai for Muse Spark 1.2 (xhigh) on Terminal-Bench 2.1 — Vals AI Terminus 2 (opens in a new tab)Archived copy (opens in a new tab)Terminal-Bench AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Muse Spark 1.2 on Terminal-Bench 2.1 — Vals AI Terminus 2
Terminal-Bench 4.0 — Vals AI mini-swe-agentEffort: xHighConfiguration: mini-swe-agent harness6.1%vals.ai for Muse Spark 1.2 (xhigh) on Terminal-Bench 4.0 — Vals AI mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Terminal-Bench AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Muse Spark 1.2 on Terminal-Bench 4.0 — Vals AI mini-swe-agent
Toolathlon VerifiedEffort: xHigh75.9%toolathlon.xyz for Muse Spark 1.2 (xhigh) on Toolathlon Verified (opens in a new tab)Archived copy (opens in a new tab)Tasks completedReported Benchmark leaderboardToolathlonIndependentToolathlon AuthorsRecord for Muse Spark 1.2 on Toolathlon Verified
Vals Index 2.1Effort: xHigh49.3%vals.ai for Muse Spark 1.2 (xhigh) on Vals Index 2.1 (opens in a new tab)Archived copy (opens in a new tab)Vals Index GDP-weighted AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Muse Spark 1.2 on Vals Index 2.1
Vibe Code Bench 1.1 — OpenHands, October 2026 evaluationEffort: xHighConfiguration: OpenHands harness79.1%vals.ai for Muse Spark 1.2 (xhigh) on Vibe Code Bench 1.1 — OpenHands, October 2026 evaluation (opens in a new tab)Archived copy (opens in a new tab)Vibe Code Bench ScoreReported Benchmark leaderboardVals AIIndependentRecord for Muse Spark 1.2 on Vibe Code Bench 1.1 — OpenHands, October 2026 evaluation