Official run

Claude Fable 5.1High

In this Twenty Questions LLM benchmark, Claude Fable 5.1 (high) ranks 2 of 18 with a 12.11 question score and 97% success. Lower is better. The run contains 35 scored episodes across 7 subjects with 5 trials per subject. Choose a subject to inspect its attempts.

Rank 2 of 18anthropicHigh

Question score12.11questions · lower is better95% CI 8.81–15.41

Success: 97%
Contract compliance: >99%
Guesser cost: $17.55
Guesser time: 1h 55m

Drill down

Subjects.

01Albert Einstein9.80 avg 02Albert Schweitzer24.40 avg 03Stephen King11.60 avg 04Garfield17.40 avg 05Achilles7.60 avg 06Genghis Khan9.00 avg 07Mario5.00 avg

Complete benchmark

Run ledger.

All Guesser and benchmark-support activity.

Total tokens: 5,592,470
Wall-clock runtime: 3h 24m
Guesser calls: 458
Completed: Sep 2, 2026, 4:15:30 AM

Reliability

Output contract breached.

1 invalid output affected 1 episode and consumed 1 counted turn.

Provenance

Execution: BX-20260902-official-M0020-001
Model ID: M-0020
Benchmark: B-0001
Base seed: 0
Git commit: 47e9bc269762

Recorded cost

By role.

Cost composition.Share of full-run cost

Guesser74.3%$17.55
Primary Oracle20%$4.71
Reviewer2%$0.48
Judge3.7%$0.86
Validator0%$0.01

Roles in this cost:

Guesser: Asks the questions and submits the scored guess.
Primary Oracle: Searches for evidence and proposes an answer.
Reviewer: Checks each Oracle YES or NO independently.
Judge: Decides when the Oracle and Reviewer disagree.
Validator: Checks a submitted guess against the trusted subject.

Read the role and answer-checking method →

Run configuration

Models.

Model identity, prompt versions, routing, calls, and recorded cost for the full run.

GuesserModel under test

anthropic/claude-fable-5.1

Asks the questions and submits the scored guess.

Calls: 458
Cost: $17.5485
Reasoning: High
Routing: Exact provider · anthropic
Resolved model: anthropic/claude-fable-5.1
Resolved provider: Anthropic
Prompt contract: stateful-category-guesser-v10-unknown-evidence-guidance
Configuration: M-0020

Resolved provider details

Anthropic458 calls$17.54856,903.80 s

Fallback calls: 0
Provider unreported: 0

Game support

Oracle and adjudication.

These models support the game. They are fixed across the run and are not under test.

Primary Oracle

openai/gpt-5.6-luna

Searches for evidence and proposes an answer.

Calls: 392
Cost: $4.7148
Reasoning: Medium
Routing: Exact provider · openai
Resolved model: openai/gpt-5.6-luna
Resolved provider: OpenAI
Prompt contract: live-web-oracle-v8-classified-research

Resolved provider details

OpenAI400 calls$4.71484,345.29 s

Fallback calls: 0
Provider unreported: 0

Reviewer

google/gemini-3.5-flash-lite

Checks each Oracle YES or NO independently.

Calls: 386
Cost: $0.4783
Reasoning: Medium
Routing: Exact provider · google-ai-studio
Resolved provider: Google AI Studio

Resolved provider details

Google AI Studio386 calls$0.4783651.89 s

Fallback calls: 0
Provider unreported: 0

Judge

anthropic/claude-opus-5

Decides when the Oracle and Reviewer disagree.

Calls: 70
Cost: $0.8632
Reasoning: Medium
Routing: OpenRouter automatic routing
Resolved provider: Claude Platform on AWS

Resolved provider details

Claude Platform on AWS70 calls$0.8632248.53 s

Fallback calls: 0
Provider unreported: 0

Guess Validator

openai/gpt-5.6-luna

Checks a submitted guess against the trusted subject.

Calls: 65
Cost: $0.0088
Reasoning: Medium
Routing: Exact provider · openai
Resolved model: openai/gpt-5.6-luna
Resolved provider: OpenAI
Prompt contract: strict-guess-validator-v1
Configuration: gpt-5.6-luna-validator

Resolved provider details

OpenAI65 calls$0.0088100.28 s

Fallback calls: 0
Provider unreported: 0