Official run

Gemini 3.8 FlashHigh

In this Twenty Questions LLM benchmark, Gemini 3.8 Flash (high) ranks 5 of 18 with a 13.14 question score and 100% success. Lower is better. The run contains 35 scored episodes across 7 subjects with 5 trials per subject. Choose a subject to inspect its attempts.

Rank 5 of 18google-ai-studioHigh

Question score13.14questions · lower is better95% CI 11.52–14.77

Success: 100%
Contract compliance: 100%
Guesser cost: $3.95
Guesser time: 1h 14m

Drill down

Subjects.

01Albert Einstein12.00 avg 02Albert Schweitzer25.80 avg 03Stephen King13.20 avg 04Garfield9.80 avg 05Achilles11.20 avg 06Genghis Khan12.00 avg 07Mario8.00 avg

Complete benchmark

Run ledger.

All Guesser and benchmark-support activity.

Total tokens: 6,749,550
Wall-clock runtime: 2h 53m
Guesser calls: 495
Completed: Sep 4, 2026, 10:37:42 AM

Reliability

Output contract clean.

All 495 evaluated outputs matched the public structured-action contract.

Provenance

Execution: BX-20260904-official-M0021-001
Model ID: M-0021
Benchmark: B-0001
Base seed: 0
Git commit: fc441117537d

Recorded cost

By role.

Cost composition.Share of full-run cost

Guesser36.5%$3.95
Primary Oracle52.3%$5.66
Reviewer4.2%$0.46
Judge6.9%$0.75
Validator0%$0.01

Roles in this cost:

Guesser: Asks the questions and submits the scored guess.
Primary Oracle: Searches for evidence and proposes an answer.
Reviewer: Checks each Oracle YES or NO independently.
Judge: Decides when the Oracle and Reviewer disagree.
Validator: Checks a submitted guess against the trusted subject.

Read the role and answer-checking method →

Run configuration

Models.

Model identity, prompt versions, routing, calls, and recorded cost for the full run.

GuesserModel under test

google/gemini-3.8-flash

Asks the questions and submits the scored guess.

Calls: 495
Cost: $3.9483
Reasoning: High
Routing: Exact provider · google-ai-studio
Resolved model: google/gemini-3.8-flash
Resolved provider: Google AI Studio
Prompt contract: stateful-category-guesser-v10-unknown-evidence-guidance
Configuration: M-0021

Resolved provider details

Google AI Studio495 calls$3.94834,415.81 s

Fallback calls: 0
Provider unreported: 0

Game support

Oracle and adjudication.

These models support the game. They are fixed across the run and are not under test.

Primary Oracle

openai/gpt-5.6-luna

Searches for evidence and proposes an answer.

Calls: 454
Cost: $5.6612
Reasoning: Medium
Routing: Exact provider · openai
Resolved model: openai/gpt-5.6-luna
Resolved provider: OpenAI
Prompt contract: live-web-oracle-v8-classified-research

Resolved provider details

OpenAI467 calls$5.66124,962.30 s

Fallback calls: 0
Provider unreported: 0

Reviewer

google/gemini-3.5-flash-lite

Checks each Oracle YES or NO independently.

Calls: 439
Cost: $0.4578
Reasoning: Medium
Routing: Exact provider · google-ai-studio
Resolved provider: Google AI Studio

Resolved provider details

Google AI Studio439 calls$0.4578676.13 s

Fallback calls: 0
Provider unreported: 0

Judge

anthropic/claude-opus-5

Decides when the Oracle and Reviewer disagree.

Calls: 58
Cost: $0.7520
Reasoning: Medium
Routing: OpenRouter automatic routing
Resolved provider: Claude Platform on AWS

Resolved provider details

Claude Platform on AWS58 calls$0.7520228.37 s

Fallback calls: 0
Provider unreported: 0

Guess Validator

openai/gpt-5.6-luna

Checks a submitted guess against the trusted subject.

Calls: 41
Cost: $0.0050
Reasoning: Medium
Routing: Exact provider · openai
Resolved model: openai/gpt-5.6-luna
Resolved provider: OpenAI
Prompt contract: strict-guess-validator-v1
Configuration: gpt-5.6-luna-validator

Resolved provider details

OpenAI41 calls$0.005063.02 s

Fallback calls: 0
Provider unreported: 0