Benchmarks

23 results across 17 benchmarks

Benchmark results for Inkling-Small
BenchmarkScoreSource
BrowseComp With context management — Inkling evaluationReported as: effort=0.99Configuration: Context management (Inkling evaluation)77.4%thinkingmachines.ai for Inkling-Small (effort=0.99) on BrowseComp With context management — Inkling evaluation (opens in a new tab)BrowseComp AccuracyReported Launch postThinking Machines LabSelf-reportedRecord for Inkling-Small on BrowseComp With context management — Inkling evaluation
Chartography October 2026 leaderboardEffort: xHigh9.1%surgehq.ai for Inkling-Small (xHigh) on Chartography October 2026 leaderboard (opens in a new tab)Archived copy (opens in a new tab)Chartography Pass@1Reported Benchmark leaderboardSurge AIIndependentRecord for Inkling-Small on Chartography October 2026 leaderboard
CharXiv Reasoning — No toolsReported as: effort=0.99Configuration: No tools77.4%thinkingmachines.ai for Inkling-Small (effort=0.99) on CharXiv Reasoning — No tools (opens in a new tab)CharXiv AccuracyReported Model cardThinking Machines LabSelf-reportedRecord for Inkling-Small on CharXiv Reasoning — No tools
CharXiv Reasoning — Python enabledReported as: effort=0.99Configuration: With Python81.3%thinkingmachines.ai for Inkling-Small (effort=0.99) on CharXiv Reasoning — Python enabled (opens in a new tab)CharXiv AccuracyReported Model cardThinking Machines LabSelf-reportedRecord for Inkling-Small on CharXiv Reasoning — Python enabled
Finance Agent Benchmark 2 — October 2026 evaluationReported as: 0.9941.3%vals.ai for Inkling-Small (0.99) on Finance Agent Benchmark 2 — October 2026 evaluation (opens in a new tab)Archived copy (opens in a new tab)Finance Agent AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Inkling-Small on Finance Agent Benchmark 2 — October 2026 evaluation
GPQA Diamond — Epoch AI harnessEffort: xHighConfiguration: Epoch AI harness88.5%epoch.ai for Inkling-Small (xhigh) on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness (opens in a new tab)Archived copy (opens in a new tab)GPQA Diamond AccuracyReported Evaluated Benchmark leaderboardEpoch AIIndependentRecord for Inkling-Small on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness
GPQA Diamond — Vals AI harnessReported as: 0.99Configuration: Vals AI harness: mean of zero-shot and few-shot CoT83.6%vals.ai for Inkling-Small (0.99) on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)GPQA Diamond AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Inkling-Small on Graduate-Level Google-Proof Q&A Diamond — Vals AI harness
HLAB October 2026 evaluationReported as: 0.991.7%vals.ai for Inkling-Small (0.99) on Harvey's Legal Agent Benchmark October 2026 evaluation (opens in a new tab)Archived copy (opens in a new tab)HLAB Task Pass RateReported Benchmark leaderboardVals AIIndependentRecord for Inkling-Small on Harvey's Legal Agent Benchmark October 2026 evaluation
IFBench OriginalReported as: effort=0.9982.2%thinkingmachines.ai for Inkling-Small (effort=0.99) on IFBench Original (opens in a new tab)AccuracyReported Launch postThinking Machines LabSelf-reportedRecord for Inkling-Small on IFBench Original
MCP Atlas Public, April 2026 update79.2%labs.scale.com for Inkling-Small on MCP Atlas Public, April 2026 update (opens in a new tab)Pass rateReported Benchmark leaderboardScale AIIndependentRecord for Inkling-Small on MCP Atlas Public, April 2026 update
MMLU-Pro Original — Vals AI harnessReported as: 0.99Configuration: Vals AI harness85.6%vals.ai for Inkling-Small (0.99) on MMLU-Pro Original — Vals AI harness (opens in a new tab)Archived copy (opens in a new tab)AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Inkling-Small on MMLU-Pro Original — Vals AI harness
SimpleQA Verified OriginalReported as: effort=0.9920.6%thinkingmachines.ai for Inkling-Small (effort=0.99) on SimpleQA Verified Original (opens in a new tab)AccuracyReported Launch postThinking Machines LabSelf-reportedRecord for Inkling-Small on SimpleQA Verified Original
SimpleQA Verified Original — Epoch AI harnessEffort: xHighConfiguration: Epoch AI harness19.1%epoch.ai for Inkling-Small (xhigh) on SimpleQA Verified Original — Epoch AI harness (opens in a new tab)Archived copy (opens in a new tab)AccuracyReported Evaluated Benchmark leaderboardEpoch AIIndependentRecord for Inkling-Small on SimpleQA Verified Original — Epoch AI harness
StrongREJECT OriginalReported as: effort=0.9998.4%thinkingmachines.ai for Inkling-Small (effort=0.99) on StrongREJECT Original (opens in a new tab)Harmful-request refusal rateReported Launch postThinking Machines LabSelf-reportedRecord for Inkling-Small on StrongREJECT Original
SWE-bench Verified — bash-only harnessReported as: effort=0.99Configuration: bash-only harness80.2%thinkingmachines.ai for Inkling-Small (effort=0.99) on SWE-bench Verified — bash-only harness (opens in a new tab)Resolved TasksReported Launch postThinking Machines LabSelf-reportedRecord for Inkling-Small on SWE-bench Verified — bash-only harness
SWE-bench Verified — Vals AI mini-swe-agentReported as: 0.99Configuration: mini-swe-agent harness82.2%vals.ai for Inkling-Small (0.99) on SWE-bench Verified — Vals AI mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Resolved TasksReported Benchmark leaderboardVals AIIndependentRecord for Inkling-Small on SWE-bench Verified — Vals AI mini-swe-agent
SWE-bench Pro PublicReported as: effort=0.9955.9%thinkingmachines.ai for Inkling-Small (effort=0.99) on SWE-bench Pro Public (opens in a new tab)Public tasks resolvedReported Launch postThinking Machines LabSelf-reportedRecord for Inkling-Small on SWE-bench Pro Public
Terminal-Bench 2.1 — Thinking Machines internal coding harnessReported as: effort=0.99Configuration: Thinking Machines internal coding harness64.7%thinkingmachines.ai for Inkling-Small (effort=0.99) on Terminal-Bench 2.1 — Thinking Machines internal coding harness (opens in a new tab)Terminal-Bench AccuracyReported Launch postThinking Machines LabSelf-reportedRecord for Inkling-Small on Terminal-Bench 2.1 — Thinking Machines internal coding harness
Terminal-Bench 2.1 — Vals AI Terminus 2Reported as: 0.99Configuration: Terminus-2 harness55.1%vals.ai for Inkling-Small (0.99) on Terminal-Bench 2.1 — Vals AI Terminus 2 (opens in a new tab)Archived copy (opens in a new tab)Terminal-Bench AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Inkling-Small on Terminal-Bench 2.1 — Vals AI Terminus 2
Terminal-Bench 4.0 — Vals AI mini-swe-agentReported as: 0.99Configuration: mini-swe-agent harness0.0%vals.ai for Inkling-Small (0.99) on Terminal-Bench 4.0 — Vals AI mini-swe-agent (opens in a new tab)Archived copy (opens in a new tab)Terminal-Bench AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Inkling-Small on Terminal-Bench 4.0 — Vals AI mini-swe-agent
Toolathlon VerifiedReported as: effort=0.9954.4%thinkingmachines.ai for Inkling-Small (effort=0.99) on Toolathlon Verified (opens in a new tab)Tasks completedReported Launch postThinking Machines LabSelf-reportedRecord for Inkling-Small on Toolathlon Verified
Vals Index 2.1Reported as: 0.9925.5%vals.ai for Inkling-Small (0.99) on Vals Index 2.1 (opens in a new tab)Archived copy (opens in a new tab)Vals Index GDP-weighted AccuracyReported Benchmark leaderboardVals AIIndependentRecord for Inkling-Small on Vals Index 2.1
Vibe Code Bench 1.1 — OpenHands, October 2026 evaluationReported as: 0.99Configuration: OpenHands harness19.1%vals.ai for Inkling-Small (0.99) on Vibe Code Bench 1.1 — OpenHands, October 2026 evaluation (opens in a new tab)Archived copy (opens in a new tab)Vibe Code Bench ScoreReported Benchmark leaderboardVals AIIndependentRecord for Inkling-Small on Vibe Code Bench 1.1 — OpenHands, October 2026 evaluation