Benchmarks

61 results across 24 benchmarks

Benchmark results for GPT-6 Astra
BenchmarkScoreSource
APEX-Agents OriginalEffort: Max64.7%mercor.com for GPT-6 Astra (max) on APEX-Agents Original (opens in a new tab)Archived copy (opens in a new tab)Pass rateReported Benchmark leaderboardMercorIndependentRecord for GPT-6 Astra on APEX-Agents Original
AutomationBench 1.0.6Effort: None25.3%zapier.com for GPT-6 Astra (none) on AutomationBench 1.0.6 (opens in a new tab)Archived copy (opens in a new tab)Task pass rateReported Benchmark leaderboardZapierIndependentZapier AutomationBench TeamRecord for GPT-6 Astra on AutomationBench 1.0.6
AutomationBench 1.0.6Effort: Low30.3%openai.com for GPT-6 Astra (low) on AutomationBench 1.0.6 (opens in a new tab)Task pass rateReported Launch postOpenAISelf-reportedRecord for GPT-6 Astra on AutomationBench 1.0.6
AutomationBench 1.0.6Effort: Medium34.1%openai.com for GPT-6 Astra (medium) on AutomationBench 1.0.6 (opens in a new tab)Task pass rateReported Launch postOpenAISelf-reportedRecord for GPT-6 Astra on AutomationBench 1.0.6
AutomationBench 1.0.6Effort: High37.1%openai.com for GPT-6 Astra (high) on AutomationBench 1.0.6 (opens in a new tab)Task pass rateReported Launch postOpenAISelf-reportedRecord for GPT-6 Astra on AutomationBench 1.0.6
AutomationBench 1.0.6Effort: xHigh39.0%openai.com for GPT-6 Astra (xhigh) on AutomationBench 1.0.6 (opens in a new tab)Task pass rateReported Launch postOpenAISelf-reportedRecord for GPT-6 Astra on AutomationBench 1.0.6
AutomationBench 1.0.6Effort: Max41.4%openai.com for GPT-6 Astra (max) on AutomationBench 1.0.6 (opens in a new tab)Task pass rateReported Launch postOpenAISelf-reportedRecord for GPT-6 Astra on AutomationBench 1.0.6
BenchCAD Vision2Code — With toolsReported as: with toolsConfiguration: With tools95.9%openai.com for GPT-6 Astra (with tools) on BenchCAD Vision2Code — With tools (opens in a new tab)Geometric OverlapReported Launch postOpenAISelf-reportedRecord for GPT-6 Astra on BenchCAD Vision2Code — With tools
BrowseComp 130-question subset — Mercor web research agentEffort: xHighConfiguration: Mercor web research agent, 100 steps, 3 hours94.2%mercor.com for GPT-6 Astra (xhigh) on BrowseComp 130-question subset — Mercor web research agent (opens in a new tab)Archived copy (opens in a new tab)BrowseComp AccuracyReported Benchmark leaderboardMercorIndependentRecord for GPT-6 Astra on BrowseComp 130-question subset — Mercor web research agent
BrowseComp91.5%openai.com for GPT-6 Astra on BrowseComp BrowseComp (opens in a new tab)BrowseComp AccuracyReported Launch postOpenAISelf-reportedRecord for GPT-6 Astra on BrowseComp BrowseComp
Chartography October 2026 leaderboardEffort: Max71.0%surgehq.ai for GPT-6 Astra (Max) on Chartography October 2026 leaderboard (opens in a new tab)Archived copy (opens in a new tab)Chartography Pass@1Reported Benchmark leaderboardSurge AIIndependentRecord for GPT-6 Astra on Chartography October 2026 leaderboard
CharXiv Descriptive and reasoning — Mercor single-shotEffort: xHighConfiguration: Mercor single-shot agent95.8%mercor.com for GPT-6 Astra (xhigh) on CharXiv Descriptive and reasoning — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)CharXiv AccuracyReported Benchmark leaderboardMercorIndependentRecord for GPT-6 Astra on CharXiv Descriptive and reasoning — Mercor single-shot
CharXiv Descriptive and reasoning — Mercor single-shotEffort: MaxConfiguration: Mercor single-shot agent96.1%mercor.com for GPT-6 Astra (max) on CharXiv Descriptive and reasoning — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)CharXiv AccuracyReported Benchmark leaderboardMercorIndependentRecord for GPT-6 Astra on CharXiv Descriptive and reasoning — Mercor single-shot
CWE-bench 1 — CodexEffort: HighConfiguration: Codex harness68.0%cwe-bench.com for GPT-6 Astra (high) on CWE-bench 1 — Codex (opens in a new tab)Archived copy (opens in a new tab)CWE-bench Programmatic Pass@1Reported Benchmark leaderboardCollinear AIIndependentRecord for GPT-6 Astra on CWE-bench 1 — Codex
DeepSWE 1.1Effort: Low67.0%openai.com for GPT-6 Astra (low) on DeepSWE 1.1 (opens in a new tab)DeepSWE Pass@1Reported Launch postOpenAISelf-reportedRecord for GPT-6 Astra on DeepSWE 1.1
DeepSWE 1.1Effort: Medium72.8%openai.com for GPT-6 Astra (medium) on DeepSWE 1.1 (opens in a new tab)DeepSWE Pass@1Reported Launch postOpenAISelf-reportedRecord for GPT-6 Astra on DeepSWE 1.1
DeepSWE 1.1Effort: High73.2%openai.com for GPT-6 Astra (high) on DeepSWE 1.1 (opens in a new tab)DeepSWE Pass@1Reported Launch postOpenAISelf-reportedRecord for GPT-6 Astra on DeepSWE 1.1
DeepSWE 1.1Effort: xHigh74.0%deepswe.datacurve.ai for GPT-6 Astra (xhigh) on DeepSWE 1.1 (opens in a new tab)DeepSWE Pass@1Reported Benchmark leaderboardDatacurveIndependentRecord for GPT-6 Astra on DeepSWE 1.1
DeepSWE 1.1Effort: Max73.2%openai.com for GPT-6 Astra (max) on DeepSWE 1.1 (opens in a new tab)DeepSWE Pass@1Reported Launch postOpenAISelf-reportedRecord for GPT-6 Astra on DeepSWE 1.1
DeepSWE 1.1 — Mercor mini-swe-agent, 500 stepsEffort: MaxConfiguration: Mercor mini-swe-agent, 500 steps, 2 hours72.0%mercor.com for GPT-6 Astra (max) on DeepSWE 1.1 — Mercor mini-swe-agent, 500 steps (opens in a new tab)Archived copy (opens in a new tab)DeepSWE Pass@1Reported Benchmark leaderboardMercorIndependentRecord for GPT-6 Astra on DeepSWE 1.1 — Mercor mini-swe-agent, 500 steps
Finance Agent Benchmark 2 — October 2026 evaluationEffort: Max53.5%vals.ai for GPT-6 Astra (max) on Finance Agent Benchmark 2 — October 2026 evaluation (opens in a new tab)Archived copy (opens in a new tab)Finance Agent AccuracyReported Benchmark leaderboardVals AIIndependentRecord for GPT-6 Astra on Finance Agent Benchmark 2 — October 2026 evaluation
FrontierSWE 2 — October 2026 evaluationConfiguration: Proximus harness, 5 trials, 20-hour budget65.5%frontierswe.com for GPT-6 Astra on FrontierSWE 2 — October 2026 evaluation (opens in a new tab)Archived copy (opens in a new tab)FrontierSWE ScoreReported Benchmark leaderboardProximalIndependentRecord for GPT-6 Astra on FrontierSWE 2 — October 2026 evaluation
GPQA Diamond96.0%openai.com for GPT-6 Astra on Graduate-Level Google-Proof Q&A Diamond (opens in a new tab)GPQA Diamond AccuracyReported Launch postOpenAISelf-reportedRecord for GPT-6 Astra on Graduate-Level Google-Proof Q&A Diamond
GPQA Diamond — Epoch AI harnessEffort: MaxConfiguration: Epoch AI harness95.8%epoch.ai for GPT-6 Astra (max) on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness (opens in a new tab)Archived copy (opens in a new tab)GPQA Diamond AccuracyReported Evaluated Benchmark leaderboardEpoch AIIndependentRecord for GPT-6 Astra on Graduate-Level Google-Proof Q&A Diamond — Epoch AI harness
GPQA Diamond — Mercor single-shotEffort: xHighConfiguration: Mercor single-shot agent96.2%mercor.com for GPT-6 Astra (xhigh) on Graduate-Level Google-Proof Q&A Diamond — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)GPQA Diamond AccuracyReported Benchmark leaderboardMercorIndependentRecord for GPT-6 Astra on Graduate-Level Google-Proof Q&A Diamond — Mercor single-shot
HLAB October 2026 evaluationEffort: Max5.4%vals.ai for GPT-6 Astra (max) on Harvey's Legal Agent Benchmark October 2026 evaluation (opens in a new tab)Archived copy (opens in a new tab)HLAB Task Pass RateReported Benchmark leaderboardVals AIIndependentRecord for GPT-6 Astra on Harvey's Legal Agent Benchmark October 2026 evaluation
Humanity's Last Exam Full set — Scale AI evaluation54.8%labs.scale.com for GPT-6 Astra on Humanity's Last Exam Full set — Scale AI evaluation (opens in a new tab)Archived copy (opens in a new tab)Humanity's Last Exam AccuracyReported Benchmark leaderboardScale AIIndependentRecord for GPT-6 Astra on Humanity's Last Exam Full set — Scale AI evaluation
Humanity's Last Exam Text only — Mercor single-shotEffort: xHighConfiguration: Mercor single-shot agent55.6%mercor.com for GPT-6 Astra (xhigh) on Humanity's Last Exam Text only — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)Humanity's Last Exam AccuracyReported Benchmark leaderboardMercorIndependentRecord for GPT-6 Astra on Humanity's Last Exam Text only — Mercor single-shot
Humanity's Last Exam Text only — Mercor single-shotEffort: MaxConfiguration: Mercor single-shot agent52.2%mercor.com for GPT-6 Astra (max) on Humanity's Last Exam Text only — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)Humanity's Last Exam AccuracyReported Benchmark leaderboardMercorIndependentRecord for GPT-6 Astra on Humanity's Last Exam Text only — Mercor single-shot
Humanity's Last Exam Text only — Scale AI evaluation54.2%labs.scale.com for GPT-6 Astra on Humanity's Last Exam Text only — Scale AI evaluation (opens in a new tab)Archived copy (opens in a new tab)Humanity's Last Exam AccuracyReported Benchmark leaderboardScale AIIndependentRecord for GPT-6 Astra on Humanity's Last Exam Text only — Scale AI evaluation
MedXpertQA MM — Mercor single-shotEffort: xHighConfiguration: Mercor single-shot agent86.5%mercor.com for GPT-6 Astra (xhigh) on MedXpertQA MM — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)AccuracyReported Benchmark leaderboardMercorIndependentRecord for GPT-6 Astra on MedXpertQA MM — Mercor single-shot
MedXpertQA MM — Mercor single-shotEffort: MaxConfiguration: Mercor single-shot agent86.8%mercor.com for GPT-6 Astra (max) on MedXpertQA MM — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)AccuracyReported Benchmark leaderboardMercorIndependentRecord for GPT-6 Astra on MedXpertQA MM — Mercor single-shot
MMLU-Pro Original — Mercor single-shotEffort: xHighConfiguration: Mercor single-shot agent91.0%mercor.com for GPT-6 Astra (xhigh) on MMLU-Pro Original — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)AccuracyReported Benchmark leaderboardMercorIndependentRecord for GPT-6 Astra on MMLU-Pro Original — Mercor single-shot
MMLU-Pro Original — Mercor single-shotEffort: MaxConfiguration: Mercor single-shot agent91.5%mercor.com for GPT-6 Astra (max) on MMLU-Pro Original — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)AccuracyReported Benchmark leaderboardMercorIndependentRecord for GPT-6 Astra on MMLU-Pro Original — Mercor single-shot
MMMU-Pro 1,730 questions — Mercor single-shotEffort: xHighConfiguration: Mercor single-shot agent86.8%mercor.com for GPT-6 Astra (xhigh) on MMMU-Pro 1,730 questions — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Not statedReported Benchmark leaderboardMercorIndependentRecord for GPT-6 Astra on MMMU-Pro 1,730 questions — Mercor single-shot
MMMU-Pro 1,730 questions — Mercor single-shotEffort: MaxConfiguration: Mercor single-shot agent87.5%mercor.com for GPT-6 Astra (max) on MMMU-Pro 1,730 questions — Mercor single-shot (opens in a new tab)Archived copy (opens in a new tab)MMMU-Pro accuracySetting: Not statedReported Benchmark leaderboardMercorIndependentRecord for GPT-6 Astra on MMMU-Pro 1,730 questions — Mercor single-shot
OSWorld 2.0 v2026.08.08 Offline — PartialEffort: Low62.2%openai.com for GPT-6 Astra (low) on OSWorld 2.0 v2026.08.08 Offline — Partial (opens in a new tab)OSWorld Partial ScoreReported Launch postOpenAISelf-reportedRecord for GPT-6 Astra on OSWorld 2.0 v2026.08.08 Offline — Partial
OSWorld 2.0 v2026.08.08 Offline — PartialEffort: Medium69.3%openai.com for GPT-6 Astra (medium) on OSWorld 2.0 v2026.08.08 Offline — Partial (opens in a new tab)OSWorld Partial ScoreReported Launch postOpenAISelf-reportedRecord for GPT-6 Astra on OSWorld 2.0 v2026.08.08 Offline — Partial
OSWorld 2.0 v2026.08.08 Offline — PartialEffort: High70.0%openai.com for GPT-6 Astra (high) on OSWorld 2.0 v2026.08.08 Offline — Partial (opens in a new tab)OSWorld Partial ScoreReported Launch postOpenAISelf-reportedRecord for GPT-6 Astra on OSWorld 2.0 v2026.08.08 Offline — Partial
OSWorld 2.0 v2026.08.08 Offline — PartialEffort: xHigh71.3%openai.com for GPT-6 Astra (xhigh) on OSWorld 2.0 v2026.08.08 Offline — Partial (opens in a new tab)OSWorld Partial ScoreReported Launch postOpenAISelf-reportedRecord for GPT-6 Astra on OSWorld 2.0 v2026.08.08 Offline — Partial
OSWorld 2.0 v2026.08.08 Offline — PartialEffort: Max73.5%openai.com for GPT-6 Astra (max) on OSWorld 2.0 v2026.08.08 Offline — Partial (opens in a new tab)OSWorld Partial ScoreReported Launch postOpenAISelf-reportedRecord for GPT-6 Astra on OSWorld 2.0 v2026.08.08 Offline — Partial
OSWorld 2.0 v2026.08.08 Offline — Partial72.6%openai.com for GPT-6 Astra on OSWorld 2.0 v2026.08.08 Offline — Partial (opens in a new tab)OSWorld Partial ScoreReported Launch postOpenAISelf-reportedRecord for GPT-6 Astra on OSWorld 2.0 v2026.08.08 Offline — Partial
PostTrainBench 1.2 — Codex CLIEffort: MaxConfiguration: Codex harness41.9%posttrainbench.com for GPT-6 Astra (Max) on PostTrainBench 1.2 — Codex CLI (opens in a new tab)Archived copy (opens in a new tab)PostTrainBench Weighted AggregateReported Benchmark leaderboardPostTrainBenchIndependentPostTrainBench TeamRecord for GPT-6 Astra on PostTrainBench 1.2 — Codex CLI
Riemann-bench October 2026 leaderboardEffort: Max72.0%surgehq.ai for GPT-6 Astra (Max) on Riemann-bench October 2026 leaderboard (opens in a new tab)Archived copy (opens in a new tab)Riemann-bench Pass@1Reported Benchmark leaderboardSurge AIIndependentRecord for GPT-6 Astra on Riemann-bench October 2026 leaderboard
SimpleQA Verified Original — Epoch AI harnessEffort: MaxConfiguration: Epoch AI harness75.6%epoch.ai for GPT-6 Astra (max) on SimpleQA Verified Original — Epoch AI harness (opens in a new tab)Archived copy (opens in a new tab)AccuracyReported Evaluated Benchmark leaderboardEpoch AIIndependentRecord for GPT-6 Astra on SimpleQA Verified Original — Epoch AI harness
SWE Atlas Codebase QnA — Mercor mini-swe-agent, 250 stepsEffort: xHighConfiguration: Mercor mini-swe-agent, 250 steps, 1 hour48.9%mercor.com for GPT-6 Astra (xhigh) on SWE Atlas Codebase QnA — Mercor mini-swe-agent, 250 steps (opens in a new tab)Archived copy (opens in a new tab)Mean pass@1Reported Benchmark leaderboardMercorIndependentRecord for GPT-6 Astra on SWE Atlas Codebase QnA — Mercor mini-swe-agent, 250 steps
SWE Atlas Codebase QnA — Mercor mini-swe-agent, 250 stepsEffort: MaxConfiguration: Mercor mini-swe-agent, 250 steps, 1 hour47.8%mercor.com for GPT-6 Astra (max) on SWE Atlas Codebase QnA — Mercor mini-swe-agent, 250 steps (opens in a new tab)Archived copy (opens in a new tab)Mean pass@1Reported Benchmark leaderboardMercorIndependentRecord for GPT-6 Astra on SWE Atlas Codebase QnA — Mercor mini-swe-agent, 250 steps
SWE Atlas Codebase QnA — Public; CodexEffort: xHighConfiguration: Codex harness59.1%labs.scale.com for GPT-6 Astra (xHigh) on SWE Atlas Codebase QnA — Public; Codex (opens in a new tab)Archived copy (opens in a new tab)Mean pass@1Reported Benchmark leaderboardScale AIIndependentRecord for GPT-6 Astra on SWE Atlas Codebase QnA — Public; Codex
SWE Atlas Refactoring — CodexEffort: xHighConfiguration: Codex harness59.1%labs.scale.com for GPT-6 Astra (xHigh) on SWE Atlas Refactoring — Codex (opens in a new tab)Archived copy (opens in a new tab)Mean pass@1Reported Benchmark leaderboardScale AIIndependentRecord for GPT-6 Astra on SWE Atlas Refactoring — Codex
Terminal-Bench 2.1 — Vals AI Terminus 2Effort: MaxConfiguration: Terminus-2 harness87.3%vals.ai for GPT-6 Astra (max) on Terminal-Bench 2.1 — Vals AI Terminus 2 (opens in a new tab)Archived copy (opens in a new tab)Terminal-Bench AccuracyReported Benchmark leaderboardVals AIIndependentRecord for GPT-6 Astra on Terminal-Bench 2.1 — Vals AI Terminus 2