Benchmark Registry

τ³-bench

τ³-bench results reported for 22 models across 4 versions. Updated 2026-10-07.

Report an issue

Versions

4 versions, plus 4 variants and configurations · Metric: Task success rate

VersionReleasedModelsResults
τ³-bench τ³ — Banking
4 variants of τ³ — Banking
VariantConfigurationReleasedModelsResults
τ³-bench τ³ — Banking, AllTools retrieval, v1.0.1 gradingAllTools retrievalJuly 22, 20261616
τ³-bench τ³ — Banking, Qwen embeddings retrieval, v1.0.1 gradingQwen embeddings retrievalJuly 22, 202611
τ³-bench τ³ — Banking, Terminal retrieval, v1.0.1 gradingTerminal retrievalJuly 22, 202633
τ³-bench τ³ — Banking, text-embedding-3-large retrieval, v1.0.1 gradingtext-embedding-3-large retrievalJuly 22, 202622
March 18, 202611
τ³-bench τ³ — AirlineLatestMarch 18, 202689
τ³-bench τ³ — RetailMarch 18, 202689
τ³-bench τ³ — TelecomMarch 18, 202689

Recently reported results: τ³-bench τ³ — Airline

Latest version: τ³-bench τ³ — Airline

ModelScoreSource
GPT-5.2 (none)52.5%sierra-tau-bench-public.s3.us-west-2.amazonaws.comArchived copyTask success rateReported Evaluated Benchmark leaderboardSierraIndependentSierra ResearchRecord
Mistral Medium 3.5 (high)72.0%huggingface.coTask success rateReported Model cardMistral AISelf-reportedRecord
Claude Opus 4.5 (high)84.0%sierra-tau-bench-public.s3.us-west-2.amazonaws.comArchived copyTask success rateReported Evaluated Benchmark leaderboardSierraIndependentSierra ResearchRecord
GPT-5.2 (high)83.0%sierra-tau-bench-public.s3.us-west-2.amazonaws.comArchived copyTask success rateReported Evaluated Benchmark leaderboardSierraIndependentSierra ResearchRecord
GLM-5 (enabled)82.5%sierra-tau-bench-public.s3.us-west-2.amazonaws.comArchived copyTask success rateReported Evaluated Benchmark leaderboardSierraIndependentSierra ResearchRecord