Benchmarks

30 results across 20 benchmarks

Benchmark results for Muse Spark 1.3
BenchmarkScoreSource
APEX-Agents OriginalEffort: xHigh58.6%mercor.com for Muse Spark 1.3 (xhigh) on APEX-Agents Original (opens in a new tab)Archived copy (opens in a new tab)Pass rateReported Benchmark leaderboardMercorIndependentRecord for Muse Spark 1.3 on APEX-Agents Original
APEX-Agents OriginalEffort: Max47.6%mercor.com for Muse Spark 1.3 (max) on APEX-Agents Original (opens in a new tab)Archived copy (opens in a new tab)Pass rateReported Benchmark leaderboardMercorIndependentRecord for Muse Spark 1.3 on APEX-Agents Original
AutomationBench 1.0.6Effort: Max20.7%zapier.com for Muse Spark 1.3 (max) on AutomationBench 1.0.6 (opens in a new tab)Archived copy (opens in a new tab)Task pass rateReported Benchmark leaderboardZapierIndependentZapier AutomationBench TeamRecord for Muse Spark 1.3 on AutomationBench 1.0.6
Chartography October 2026 leaderboardEffort: xHigh27.6%surgehq.ai for Muse Spark 1.3 (xHigh) on Chartography October 2026 leaderboard (opens in a new tab)Archived copy (opens in a new tab)Chartography Pass@1Reported Benchmark leaderboardSurge AIIndependentRecord for Muse Spark 1.3 on Chartography October 2026 leaderboard
CharXiv Descriptive and reasoning — Mercor single-shotEffort: MaxConfiguration: Mercor single-shot agent93.9%mercor.com for Muse Spark 1.3 (max) on CharXiv Descriptive and reasoning — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)CharXiv AccuracyReported Benchmark leaderboardMercorIndependentRecord for Muse Spark 1.3 on CharXiv Descriptive and reasoning — Mercor single-shot
CursorBench 4.0Effort: Low29.3%cursor.com for Muse Spark 1.3 (low) on CursorBench 4.0 (opens in a new tab)Archived copy (opens in a new tab)CursorBench AccuracyReported Benchmark leaderboardCursorIndependentRecord for Muse Spark 1.3 on CursorBench 4.0
CursorBench 4.0Effort: Medium32.6%cursor.com for Muse Spark 1.3 (medium) on CursorBench 4.0 (opens in a new tab)Archived copy (opens in a new tab)CursorBench AccuracyReported Benchmark leaderboardCursorIndependentRecord for Muse Spark 1.3 on CursorBench 4.0
CursorBench 4.0Effort: High33.4%cursor.com for Muse Spark 1.3 (high) on CursorBench 4.0 (opens in a new tab)Archived copy (opens in a new tab)CursorBench AccuracyReported Benchmark leaderboardCursorIndependentRecord for Muse Spark 1.3 on CursorBench 4.0
CursorBench 4.0Effort: xHigh37.5%cursor.com for Muse Spark 1.3 (xhigh) on CursorBench 4.0 (opens in a new tab)Archived copy (opens in a new tab)CursorBench AccuracyReported Benchmark leaderboardCursorIndependentRecord for Muse Spark 1.3 on CursorBench 4.0
CursorBench 4.0Effort: Max41.6%cursor.com for Muse Spark 1.3 (max) on CursorBench 4.0 (opens in a new tab)Archived copy (opens in a new tab)CursorBench AccuracyReported Benchmark leaderboardCursorIndependentRecord for Muse Spark 1.3 on CursorBench 4.0
CWE-bench 1 — OpenCodeEffort: HighConfiguration: OpenCode harness55.0%cwe-bench.com for Muse Spark 1.3 (high) on CWE-bench 1 — OpenCode (opens in a new tab)Archived copy (opens in a new tab)CWE-bench Programmatic Pass@1Reported Benchmark leaderboardCollinear AIIndependentRecord for Muse Spark 1.3 on CWE-bench 1 — OpenCode
DeepSearchQA 900-prompt setEffort: Max90.3%dev.meta.ai for Muse Spark 1.3 (max) on DeepSearchQA 900-prompt set (opens in a new tab)F1 scoreReported Model pageMetaSelf-reportedRecord for Muse Spark 1.3 on DeepSearchQA 900-prompt set
DeepSWE 1.1 — mini-swe-agentEffort: MaxConfiguration: mini-swe-agent harness75.4%dev.meta.ai for Muse Spark 1.3 (max) on DeepSWE 1.1 — mini-swe-agent (opens in a new tab)DeepSWE Pass@1Reported Model pageMetaSelf-reportedRecord for Muse Spark 1.3 on DeepSWE 1.1 — mini-swe-agent
Finance Agent Benchmark 2 — October 2026 evaluationEffort: xHigh58.9%vals.ai for Muse Spark 1.3 (xhigh) on Finance Agent Benchmark 2 — October 2026 evaluation (opens in a new tab)Archived copy (opens in a new tab)Finance Agent AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Muse Spark 1.3 on Finance Agent Benchmark 2 — October 2026 evaluation
GPQA Diamond — Mercor single-shotEffort: MaxConfiguration: Mercor single-shot agent91.3%mercor.com for Muse Spark 1.3 (max) on Graduate-Level Google-Proof Q&A Diamond — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)GPQA Diamond AccuracyReported Benchmark leaderboardMercorIndependentRecord for Muse Spark 1.3 on Graduate-Level Google-Proof Q&A Diamond — Mercor single-shot
HLAB October 2026 evaluationEffort: xHigh22.9%vals.ai for Muse Spark 1.3 (xhigh) on Harvey's Legal Agent Benchmark October 2026 evaluation (opens in a new tab)Archived copy (opens in a new tab)HLAB Task Pass RateReported Benchmark leaderboardVals AIIndependentRecord for Muse Spark 1.3 on Harvey's Legal Agent Benchmark October 2026 evaluation
JobBench Main 65 tasksEffort: Max64.9%dev.meta.ai for Muse Spark 1.3 (max) on JobBench Main 65 tasks (opens in a new tab)Mean rubric scoreReported Model pageMetaSelf-reportedRecord for Muse Spark 1.3 on JobBench Main 65 tasks
MedXpertQA MM — Mercor single-shotEffort: MaxConfiguration: Mercor single-shot agent76.3%mercor.com for Muse Spark 1.3 (max) on MedXpertQA MM — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)AccuracyReported Benchmark leaderboardMercorIndependentRecord for Muse Spark 1.3 on MedXpertQA MM — Mercor single-shot
MRCR v2 8-needle 256K–512K (o200k_base)Effort: Max98.5%dev.meta.ai for Muse Spark 1.3 (max) on MRCR v2 8-needle 256K–512K (o200k_base) (opens in a new tab)Mean sequence-matcher ratioReported Model pageMetaSelf-reportedRecord for Muse Spark 1.3 on MRCR v2 8-needle 256K–512K (o200k_base)
MRCR v2 8-needle 512K–1M (o200k_base)Effort: Max98.1%dev.meta.ai for Muse Spark 1.3 (max) on MRCR v2 8-needle 512K–1M (o200k_base) (opens in a new tab)Mean sequence-matcher ratioReported Model pageMetaSelf-reportedRecord for Muse Spark 1.3 on MRCR v2 8-needle 512K–1M (o200k_base)
OSWorld 2.0 v2026.08.08 — BinaryEffort: Max32.0%dev.meta.ai for Muse Spark 1.3 (max) on OSWorld 2.0 v2026.08.08 — Binary (opens in a new tab)OSWorld Strict ScoreReported Model pageMetaSelf-reportedRecord for Muse Spark 1.3 on OSWorld 2.0 v2026.08.08 — Binary
OSWorld 2.0 v2026.08.08 — PartialEffort: Max66.9%dev.meta.ai for Muse Spark 1.3 (max) on OSWorld 2.0 v2026.08.08 — Partial (opens in a new tab)OSWorld Partial ScoreReported Model pageMetaSelf-reportedRecord for Muse Spark 1.3 on OSWorld 2.0 v2026.08.08 — Partial
Riemann-bench October 2026 leaderboardEffort: xHigh28.0%surgehq.ai for Muse Spark 1.3 (xHigh) on Riemann-bench October 2026 leaderboard (opens in a new tab)Archived copy (opens in a new tab)Riemann-bench Pass@1Reported Benchmark leaderboardSurge AIIndependentRecord for Muse Spark 1.3 on Riemann-bench October 2026 leaderboard
SWE Atlas Codebase QnA — Public; mini-swe-agentEffort: MaxConfiguration: mini-swe-agent harness59.4%dev.meta.ai for Muse Spark 1.3 (max) on SWE Atlas Codebase QnA — Public; mini-swe-agent (opens in a new tab)Mean pass@1Reported Model pageMetaSelf-reportedRecord for Muse Spark 1.3 on SWE Atlas Codebase QnA — Public; mini-swe-agent
Terminal-Bench 2.1Effort: Max88.8%dev.meta.ai for Muse Spark 1.3 (max) on Terminal-Bench 2.1 (opens in a new tab)Terminal-Bench AccuracyReported Model pageMetaSelf-reportedRecord for Muse Spark 1.3 on Terminal-Bench 2.1
Terminal-Bench 2.1 — Vals AI Terminus 2Effort: xHighConfiguration: Terminus-2 harness72.3%vals.ai for Muse Spark 1.3 (xhigh) on Terminal-Bench 2.1 — Vals AI Terminus 2 (opens in a new tab)Archived copy (opens in a new tab)Terminal-Bench AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Muse Spark 1.3 on Terminal-Bench 2.1 — Vals AI Terminus 2
Terminal-Bench 4.0 — Muse CodeEffort: xHighConfiguration: Muse Code harness14.6%tbench.ai for Muse Spark 1.3 (xhigh) on Terminal-Bench 4.0 — Muse Code (opens in a new tab)Archived copy (opens in a new tab)Terminal-Bench AccuracyReported Benchmark leaderboardTerminal-BenchIndependentTerminal-Bench TeamRecord for Muse Spark 1.3 on Terminal-Bench 4.0 — Muse Code
Terminal-Bench 4.0 — Vals AI mini-swe-agentEffort: xHighConfiguration: mini-swe-agent harness10.6%vals.ai for Muse Spark 1.3 (xhigh) on Terminal-Bench 4.0 — Vals AI mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Terminal-Bench AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Muse Spark 1.3 on Terminal-Bench 4.0 — Vals AI mini-swe-agent
Vals Index 2.1Effort: xHigh53.2%vals.ai for Muse Spark 1.3 (xhigh) on Vals Index 2.1 (opens in a new tab)Archived copy (opens in a new tab)Vals Index GDP-weighted AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Muse Spark 1.3 on Vals Index 2.1
Vibe Code Bench 1.1 — OpenHands, October 2026 evaluationEffort: xHighConfiguration: OpenHands harness82.9%vals.ai for Muse Spark 1.3 (xhigh) on Vibe Code Bench 1.1 — OpenHands, October 2026 evaluation (opens in a new tab)Archived copy (opens in a new tab)Vibe Code Bench ScoreReported Benchmark leaderboardVals AIIndependentRecord for Muse Spark 1.3 on Vibe Code Bench 1.1 — OpenHands, October 2026 evaluation