Official run

gpt-oss-120BHigh

In this Twenty Questions LLM benchmark, gpt-oss-120B (high) ranks 6 of 18 with a 13.63 question score and 94% success. Lower is better. The run contains 35 scored episodes across 7 subjects with 5 trials per subject. Choose a subject to inspect its attempts.

Rank 6 of 18cerebrasHigh

Question score13.63questions · lower is better95% CI 9.79–17.46

Success: 94%
Contract compliance: 91%
Guesser cost: $1.27
Guesser time: 53m 48s

Drill down

Subjects.

01Albert Einstein5.40 avg 02Albert Schweitzer26.80 avg 03Stephen King12.00 avg 04Garfield19.80 avg 05Achilles11.40 avg 06Genghis Khan17.20 avg 07Mario2.80 avg

Complete benchmark

Run ledger.

All Guesser and benchmark-support activity.

Total tokens: 5,459,066
Wall-clock runtime: 2h 13m
Guesser calls: 490
Completed: Jul 28, 2026, 10:21:35 PM

Reliability

Output contract breached.

46 invalid outputs affected 7 episodes and consumed 46 counted turns.

Provenance

Execution: BX-20260728-official-M0002-011
Model ID: M-0002
Benchmark: B-0001
Base seed: 0
Git commit: 1076abf750e3

Recorded cost

By role.

Cost composition.Share of full-run cost

Guesser27.4%$1.27
Primary Oracle48.8%$2.26
Reviewer10%$0.46
Judge13.6%$0.63
Validator0.3%$0.01

Roles in this cost:

Guesser: Asks the questions and submits the scored guess.
Primary Oracle: Searches for evidence and proposes an answer.
Reviewer: Checks each Oracle YES or NO independently.
Judge: Decides when the Oracle and Reviewer disagree.
Validator: Checks a submitted guess against the trusted subject.

Read the role and answer-checking method →

Run configuration

Models.

Model identity, prompt versions, routing, calls, and recorded cost for the full run.

GuesserModel under test

openai/gpt-oss-120b

Asks the questions and submits the scored guess.

Calls: 490
Cost: $1.2703
Reasoning: High
Routing: Exact provider · cerebras
Resolved model: openai/gpt-oss-120b
Resolved provider: Cerebras
Prompt contract: stateful-category-guesser-v10-unknown-evidence-guidance
Configuration: M-0002

Resolved provider details

Legacy run. Per-call routing totals were not retained. The configured provider was cerebras.

Game support

Oracle and adjudication.

These models support the game. They are fixed across the run and are not under test.

Primary Oracle

openai/gpt-5.6-luna

Searches for evidence and proposes an answer.

Calls: 403
Cost: $2.2636
Reasoning: Medium
Routing: Exact provider · openai
Resolved model: openai/gpt-5.6-luna
Resolved provider: OpenAI
Prompt contract: live-web-oracle-v7-direct-negative-evidence

Resolved provider details

Legacy run. Per-call routing totals were not retained. The configured provider was openai.

Reviewer

google/gemini-3.5-flash-lite

Checks each Oracle YES or NO independently.

Calls: 389
Cost: $0.4624
Reasoning: Medium
Routing: Exact provider · google-ai-studio

Resolved provider details

Legacy run. Per-call routing totals were not retained. The configured provider was google-ai-studio.

Judge

anthropic/claude-opus-5

Decides when the Oracle and Reviewer disagree.

Calls: 54
Cost: $0.6334
Reasoning: Medium
Routing: Exact provider · anthropic

Resolved provider details

Legacy run. Per-call routing totals were not retained. The configured provider was anthropic.

Guess Validator

openai/gpt-5.6-luna

Checks a submitted guess against the trusted subject.

Calls: 40
Cost: $0.0125
Reasoning: Medium
Routing: Exact provider · openai
Resolved model: openai/gpt-5.6-luna
Resolved provider: OpenAI
Prompt contract: strict-guess-validator-v1
Configuration: gpt-5.6-luna-validator

Resolved provider details

Legacy run. Per-call routing totals were not retained. The configured provider was openai.