Official run
Gemini 3.6 FlashHigh
In this Twenty Questions AI benchmark, Gemini 3.6 Flash (high) ranks 14 of 18 with a question score of 16.89: the average number of questions per round, where lower is better. It failed 1 of 35 rounds. The run has 35 scored rounds: 7 subjects, 5 rounds each. Choose a subject to see its rounds.
Rank 14 of 18google-vertexHigh
Round scores are averaged for each subject, then across subjects. A failed round gets the fixed failure score.
The 95% confidence interval shows how certain the average is, based on repeated rounds on these subjects. A wider interval means less consistent results. It does not predict scores on other subjects.
- Solved
- 97%
- Valid format
- 99.8%
- Guesser cost
- $5.97
- Guesser time
- 1h 9m
Drill down
Subjects.
Complete benchmark
Run totals.
Includes the Guesser and all answer-checking models.
- Total tokens
- 5,594,609
- Total runtime
- 2h 52m
- Guesser calls
- 625
- Completed
Reply format
Format errors found.
1 invalid reply in 1 round used up 1 counted turn.
Run details
- Execution
BX-20260728-official-M0004-010- Model ID
M-0004- Benchmark
B-0001- Base seed
0- Git commit
aa131c32c065
Recorded cost
By role.
- Guesser60.6%$5.97
- Oracle29.8%$2.94
- Reviewer4.8%$0.48
- Judge4.6%$0.46
- Validator0.1%$0.01
Roles in this cost:
- Guesser
- The tested model. Asks questions and makes guesses.
- Oracle
- Searches for evidence and proposes an answer.
- Reviewer
- Checks each Oracle answer except Unknown, without seeing it.
- Judge
- Decides when the Oracle and Reviewer disagree.
- Validator
- Checks whether a guess names the hidden subject.
Run configuration
Models.
Models, prompt versions, providers, calls, and cost for the whole run.
google/gemini-3.6-flash
The tested model. Asks questions and makes guesses.
- Calls
- 625
- Cost
- $5.9676
- Reasoning
- High
- Routing
- Fixed provider · google-vertex
- Resolved model
- google/gemini-3.6-flash
- Resolved provider
- Prompt version
stateful-category-guesser-v10-unknown-evidence-guidance- Configuration
M-0004
Provider details
Older run: providers were not recorded per call. The configured provider was google-vertex.
Answer checks
Answering and checking models.
These models answer and check the Guesser's questions and guesses. They are the same for every round and are not being tested.
openai/gpt-5.6-luna
Searches for evidence and proposes an answer.
- Calls
- 582
- Cost
- $2.9414
- Reasoning
- Medium
- Routing
- Fixed provider · openai
- Resolved model
- openai/gpt-5.6-luna
- Resolved provider
- OpenAI
- Prompt version
live-web-oracle-v7-direct-negative-evidence
Provider details
Older run: providers were not recorded per call. The configured provider was openai.
google/gemini-3.5-flash-lite
Checks each Oracle answer except Unknown, without seeing it.
- Calls
- 559
- Cost
- $0.4772
- Reasoning
- Medium
- Routing
- Fixed provider · google-ai-studio
Provider details
Older run: providers were not recorded per call. The configured provider was google-ai-studio.
anthropic/claude-opus-5
Decides when the Oracle and Reviewer disagree.
- Calls
- 38
- Cost
- $0.4565
- Reasoning
- Medium
- Routing
- Fixed provider · anthropic
Provider details
Older run: providers were not recorded per call. The configured provider was anthropic.
openai/gpt-5.6-luna
Checks whether a guess names the hidden subject.
- Calls
- 41
- Cost
- $0.0129
- Reasoning
- Medium
- Routing
- Fixed provider · openai
- Resolved model
- openai/gpt-5.6-luna
- Resolved provider
- OpenAI
- Prompt version
strict-guess-validator-v1- Configuration
gpt-5.6-luna-validator
Provider details
Older run: providers were not recorded per call. The configured provider was openai.