Official run
Claude Opus 5High
In this Twenty Questions LLM benchmark, Claude Opus 5 (high) ranks 3 of 18 with a 12.34 question score and 100% success. Lower is better. The run contains 35 scored episodes across 7 subjects with 5 trials per subject. Choose a subject to inspect its attempts.
Rank 3 of 18anthropicHigh
- Success
- 100%
- Contract compliance
- 99%
- Guesser cost
- $2.47
- Guesser time
- 42m 27s
Drill down
Subjects.
Complete benchmark
Run ledger.
All Guesser and benchmark-support activity.
- Total tokens
- 3,909,611
- Wall-clock runtime
- 2h 4m
- Guesser calls
- 467
- Completed
Reliability
Output contract breached.
3 invalid outputs affected 2 episodes and consumed 3 counted turns.
Provenance
- Execution
BX-20260728-official-M0006-010- Model ID
M-0006- Benchmark
B-0001- Base seed
0- Git commit
aa131c32c065
Recorded cost
By role.
- Guesser45.1%$2.47
- Primary Oracle38%$2.08
- Reviewer7.6%$0.41
- Judge9%$0.49
- Validator0.2%$0.01
Roles in this cost:
- Guesser
- Asks the questions and submits the scored guess.
- Primary Oracle
- Searches for evidence and proposes an answer.
- Reviewer
- Checks each Oracle YES or NO independently.
- Judge
- Decides when the Oracle and Reviewer disagree.
- Validator
- Checks a submitted guess against the trusted subject.
Run configuration
Models.
Model identity, prompt versions, routing, calls, and recorded cost for the full run.
anthropic/claude-opus-5
Asks the questions and submits the scored guess.
- Calls
- 467
- Cost
- $2.4682
- Reasoning
- High
- Routing
- Exact provider · anthropic
- Resolved model
- anthropic/claude-opus-5
- Resolved provider
- Anthropic
- Prompt contract
stateful-category-guesser-v10-unknown-evidence-guidance- Configuration
M-0006
Resolved provider details
Legacy run. Per-call routing totals were not retained. The configured provider was anthropic.
Game support
Oracle and adjudication.
These models support the game. They are fixed across the run and are not under test.
openai/gpt-5.6-luna
Searches for evidence and proposes an answer.
- Calls
- 426
- Cost
- $2.0802
- Reasoning
- Medium
- Routing
- Exact provider · openai
- Resolved model
- openai/gpt-5.6-luna
- Resolved provider
- OpenAI
- Prompt contract
live-web-oracle-v7-direct-negative-evidence
Resolved provider details
Legacy run. Per-call routing totals were not retained. The configured provider was openai.
google/gemini-3.5-flash-lite
Checks each Oracle YES or NO independently.
- Calls
- 404
- Cost
- $0.4143
- Reasoning
- Medium
- Routing
- Exact provider · google-ai-studio
Resolved provider details
Legacy run. Per-call routing totals were not retained. The configured provider was google-ai-studio.
anthropic/claude-opus-5
Decides when the Oracle and Reviewer disagree.
- Calls
- 41
- Cost
- $0.4932
- Reasoning
- Medium
- Routing
- Exact provider · anthropic
Resolved provider details
Legacy run. Per-call routing totals were not retained. The configured provider was anthropic.
openai/gpt-5.6-luna
Checks a submitted guess against the trusted subject.
- Calls
- 38
- Cost
- $0.0117
- Reasoning
- Medium
- Routing
- Exact provider · openai
- Resolved model
- openai/gpt-5.6-luna
- Resolved provider
- OpenAI
- Prompt contract
strict-guess-validator-v1- Configuration
gpt-5.6-luna-validator
Resolved provider details
Legacy run. Per-call routing totals were not retained. The configured provider was openai.