Official run

GPT-6 AstraHigh

In this Twenty Questions LLM benchmark, GPT-6 Astra (high) ranks 11 of 18 with a 15.09 question score and 94% success. Lower is better. The run contains 35 scored episodes across 7 subjects with 5 trials per subject. Choose a subject to inspect its attempts.

Rank 11 of 18openaiHigh

Question score15.09questions · lower is better95% CI 11.88–18.30

Success: 94%
Contract compliance: 100%
Guesser cost: $11.48
Guesser time: 1h 35m

Drill down

Subjects.

01Albert Einstein8.40 avg 02Albert Schweitzer23.00 avg 03Stephen King38.80 avg 04Garfield14.80 avg 05Achilles6.60 avg 06Genghis Khan9.00 avg 07Mario5.00 avg

Complete benchmark

Run ledger.

All Guesser and benchmark-support activity.

Total tokens: 10,214,955
Wall-clock runtime: 10h 41m
Guesser calls: 561
Completed: Sep 5, 2026, 7:12:29 AM

Reliability

Output contract clean.

All 561 evaluated outputs matched the public structured-action contract.

Provenance

Execution: BX-20260904-official-M0022-001
Model ID: M-0022
Benchmark: B-0001
Base seed: 0
Git commit: 12002aecd849

Recorded cost

By role.

Cost composition.Share of full-run cost

Guesser50.8%$11.48
Primary Oracle44.3%$10.01
Reviewer2.2%$0.49
Judge2.7%$0.62
Validator0%$0.01

Excluded repair overhead$3.154 superseded attempts across 3 affected trials. Not included above.

Roles in this cost:

Guesser: Asks the questions and submits the scored guess.
Primary Oracle: Searches for evidence and proposes an answer.
Reviewer: Checks each Oracle YES or NO independently.
Judge: Decides when the Oracle and Reviewer disagree.
Validator: Checks a submitted guess against the trusted subject.

Read the role and answer-checking method →

Run configuration

Models.

Model identity, prompt versions, routing, calls, and recorded cost for the full run.

GuesserModel under test

openai/gpt-6-astra

Asks the questions and submits the scored guess.

Calls: 561
Cost: $11.4750
Reasoning: High
Routing: Exact provider · openai
Resolved model: openai/gpt-6-astra
Resolved provider: OpenAI
Prompt contract: stateful-category-guesser-v10-unknown-evidence-guidance
Configuration: M-0022

Resolved provider details

OpenAI561 calls$11.47505,686.82 s

Fallback calls: 0
Provider unreported: 0

Game support

Oracle and adjudication.

These models support the game. They are fixed across the run and are not under test.

Primary Oracle

openai/gpt-5.6-luna

Searches for evidence and proposes an answer.

Calls: 510
Cost: $10.0092
Reasoning: Medium
Routing: Exact provider · openai
Resolved model: openai/gpt-5.6-luna
Resolved provider: OpenAI
Prompt contract: live-web-oracle-v8-classified-research

Resolved provider details

OpenAI575 calls$10.00927,970.89 s

Fallback calls: 0
Provider unreported: 0

Reviewer

google/gemini-3.5-flash-lite

Checks each Oracle YES or NO independently.

Calls: 454
Cost: $0.4892
Reasoning: Medium
Routing: Exact provider · google-ai-studio
Resolved provider: Google AI Studio

Resolved provider details

Google AI Studio454 calls$0.4892680.33 s

Fallback calls: 0
Provider unreported: 0

Judge

anthropic/claude-opus-5

Decides when the Oracle and Reviewer disagree.

Calls: 45
Cost: $0.6159
Reasoning: Medium
Routing: OpenRouter automatic routing
Resolved provider: Claude Platform on AWS

Resolved provider details

Claude Platform on AWS45 calls$0.6159198.98 s

Fallback calls: 0
Provider unreported: 0

Guess Validator

openai/gpt-5.6-luna

Checks a submitted guess against the trusted subject.

Calls: 51
Cost: $0.0065
Reasoning: Medium
Routing: Exact provider · openai
Resolved model: openai/gpt-5.6-luna
Resolved provider: OpenAI
Prompt contract: strict-guess-validator-v1
Configuration: gpt-5.6-luna-validator

Resolved provider details

OpenAI51 calls$0.006569.44 s

Fallback calls: 0
Provider unreported: 0