Official run

Grok 4.6High

In this Twenty Questions LLM benchmark, Grok 4.6 (high) ranks 8 of 18 with a 14.29 question score and 100% success. Lower is better. The run contains 35 scored episodes across 7 subjects with 5 trials per subject. Choose a subject to inspect its attempts.

Rank 8 of 18xaiHigh

Question score14.29questions · lower is better95% CI 12.36–16.21

Success: 100%
Contract compliance: 100%
Guesser cost: $2.29
Guesser time: 1h 49m

Drill down

Subjects.

01Albert Einstein9.80 avg 02Albert Schweitzer30.00 avg 03Stephen King14.40 avg 04Garfield14.60 avg 05Achilles8.60 avg 06Genghis Khan16.00 avg 07Mario6.60 avg

Complete benchmark

Run ledger.

All Guesser and benchmark-support activity.

Total tokens: 4,936,122
Wall-clock runtime: 4h 40m
Guesser calls: 535
Completed: Aug 14, 2026, 10:51:12 AM

Reliability

Output contract clean.

All 535 evaluated outputs matched the public structured-action contract.

Provenance

Execution: BX-20260814-official-M0015-013
Model ID: M-0015
Benchmark: B-0001
Base seed: 0
Git commit: 4e197a516062

Recorded cost

By role.

Cost composition.Share of full-run cost

Guesser27.7%$2.29
Primary Oracle53.4%$4.41
Reviewer6.6%$0.54
Judge12.2%$1.01
Validator0%$0.00

Excluded repair overhead$1.256 superseded attempts across 6 affected trials. Not included above.

Roles in this cost:

Guesser: Asks the questions and submits the scored guess.
Primary Oracle: Searches for evidence and proposes an answer.
Reviewer: Checks each Oracle YES or NO independently.
Judge: Decides when the Oracle and Reviewer disagree.
Validator: Checks a submitted guess against the trusted subject.

Read the role and answer-checking method →

Run configuration

Models.

Model identity, prompt versions, routing, calls, and recorded cost for the full run.

GuesserModel under test

x-ai/grok-4.6

Asks the questions and submits the scored guess.

Calls: 535
Cost: $2.2862
Reasoning: High
Routing: Exact provider · xai
Resolved model: x-ai/grok-4.6
Resolved provider: xAI
Prompt contract: stateful-category-guesser-v10-unknown-evidence-guidance
Configuration: M-0015

Resolved provider details

xAI535 calls$2.28626,566.22 s

Fallback calls: 0
Provider unreported: 0

Game support

Oracle and adjudication.

These models support the game. They are fixed across the run and are not under test.

Primary Oracle

openai/gpt-5.6-luna

Searches for evidence and proposes an answer.

Calls: 499
Cost: $4.4071
Reasoning: Medium
Routing: Exact provider · openai
Resolved model: openai/gpt-5.6-luna
Resolved provider: OpenAI
Prompt contract: live-web-oracle-v7-direct-negative-evidence

Resolved provider details

OpenAI499 calls$4.40714,153.62 s

Fallback calls: 0
Provider unreported: 0

Reviewer

google/gemini-3.5-flash-lite

Checks each Oracle YES or NO independently.

Calls: 483
Cost: $0.5438
Reasoning: Medium
Routing: Exact provider · google-ai-studio
Resolved provider: Google AI Studio

Resolved provider details

Google AI Studio483 calls$0.5438755.02 s

Fallback calls: 0
Provider unreported: 0

Judge

anthropic/claude-opus-5

Decides when the Oracle and Reviewer disagree.

Calls: 81
Cost: $1.0093
Reasoning: Medium
Routing: OpenRouter automatic routing
Resolved provider: Amazon Bedrock, Anthropic

Resolved provider details

Amazon Bedrock8 calls$0.090335.47 s

Anthropic73 calls$0.9190293.82 s

Fallback calls: 0
Provider unreported: 0

Guess Validator

openai/gpt-5.6-luna

Checks a submitted guess against the trusted subject.

Calls: 36
Cost: $0.0022
Reasoning: Medium
Routing: Exact provider · openai
Resolved model: openai/gpt-5.6-luna
Resolved provider: OpenAI
Prompt contract: strict-guess-validator-v1
Configuration: gpt-5.6-luna-validator

Resolved provider details

OpenAI36 calls$0.002242.39 s

Fallback calls: 0
Provider unreported: 0