Deep20Bench
v1.0Previous

Benchmark edition

Edition 1.0 settings

7 subjects5 rounds per subject50 question limit
Question answersYesNoUnknown
What changed
Source
Menu
Round

Round 4

Round 4

Did not identify Stephen King after 50 counted questions.

Back to all Stephen King rounds
Questions
50 Scored as 51
Outcome
Too many format errors
Format errors
No errors
Duration
15m 19s
Round cost
$0.6237 All 51 turns

Transcript

Questions and answers.

Turns are shown in the order they happened. Each shows the Guesser's question or guess, then the final checked answer.

Exact Guesser setupSystem message and BEGIN event

The Guesser received these messages at the start of the round. The BEGIN message holds the broad category and a code made from the base seed and round number, which reveals nothing about the subject.

System message
You are the Guesser being evaluated in a Twenty Questions benchmark.

One subject is hidden. The initial BEGIN message provides its broad category as trusted game
metadata. Identify the subject using as few counted questions as possible. ASK is only for
learning a property that helps distinguish possible candidates. Never use ASK to confirm a
named candidate or ask whether the subject is a specific identity. When you are ready to name
a candidate, use GUESS immediately. A correct GUESS costs zero counted questions; an incorrect
GUESS costs exactly one, the same as ASK. A GUESS must provide both a name and a short
identifying description. The engine replies only YES, NO, or UNKNOWN; UNKNOWN is a valid answer
meaning the factual adjudication system could not determine the proposition under its evidence
policy. Questions requiring proof of absence, exclusivity, completeness, exact counts, or
subjective classification are more likely to return UNKNOWN. Prefer direct, positively
verifiable properties when possible.

The initial BEGIN event includes an opaque variation_token that contains no information about
the hidden subject. Do not interpret, repeat, mention, or derive meaning from it. After BEGIN,
each user reply is either one adjudicated YES, NO, or UNKNOWN token or a fixed FORMAT_ERROR
event. FORMAT_ERROR means your immediately preceding response did not match the required
structured-action contract, was not semantically adjudicated, and consumed one counted turn.
Use the formats in that event and try again. Never infer that the attempted question or
guess was right or wrong.

At most 50 questions or rejected guesses are counted. After that limit you receive
one final guess-only opportunity. Format errors also consume counted turns. The provider wire
response is an object with exactly one `result` action. On the final opportunity ASK is
forbidden. Return only the required structured response. Do not include analysis, hidden
reasoning, or additional fields.
BEGIN event
{"category":"person","event":"BEGIN","variation_token":"BYOMPCJW"}

Valid replies are shown in the required format. Rejected replies are shown separately, without provider or call IDs.

Q1
1 · Guesser asks

Is the person male?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person male?"}}
Oracle evidence1 model-reported source

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns Yes
Q2
1 · Guesser asks

Is the person a historical figure?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person a historical figure?"}}
2 · Answer check returns Unknown
Q3
1 · Guesser asks

Is the person alive?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person alive?"}}
Oracle evidence2 model-reported sources

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns Yes
Q4
1 · Guesser asks

Is the person a politician?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person a politician?"}}
Oracle evidence1 model-reported source

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns No
Q5
1 · Guesser asks

Is the person an actor?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person an actor?"}}
Oracle evidence2 model-reported sources

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns Yes
Q6
1 · Guesser asks

Is the person primarily known for work in film rather than television?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person primarily known for work in film rather than television?"}}
Oracle evidence2 model-reported sources

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns No
Q7
1 · Guesser asks

Is the person primarily known for comedy?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person primarily known for comedy?"}}
Oracle evidence2 model-reported sources

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns No
Q8
1 · Guesser asks

Is the person American?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person American?"}}
Oracle evidence1 model-reported source

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns Yes
Q9
1 · Guesser asks

Is the person known for starring in a major network television drama series?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person known for starring in a major network television drama series?"}}
Oracle evidence2 model-reported sources

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns No
Q10
1 · Guesser asks

Is the person primarily known for work in television rather than other media?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person primarily known for work in television rather than other media?"}}
Oracle evidence2 model-reported sources

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns No
Q11
1 · Guesser asks

Is the person primarily known for theater or stage work?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person primarily known for theater or stage work?"}}
Oracle evidence2 model-reported sources

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns No
Q12
1 · Guesser asks

Is the person primarily known as a voice actor?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person primarily known as a voice actor?"}}
Oracle evidence1 model-reported source

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns No
Q13
1 · Guesser asks

Is the person primarily known for roles in movies?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person primarily known for roles in movies?"}}
Oracle evidence2 model-reported sources

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns No
Q14
1 · Guesser asks

Is the person primarily known for music or singing?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person primarily known for music or singing?"}}
Oracle evidence1 model-reported source

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns No
Q15
1 · Guesser asks

Has the person won an Academy Award (Oscar)?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Has the person won an Academy Award (Oscar)?"}}
Oracle evidence1 model-reported source

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns No
Q16
1 · Guesser asks

Is the person under 40 years old?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person under 40 years old?"}}
Oracle evidence2 model-reported sources

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns No
Q17
1 · Guesser asks

Is the person over 60 years old?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person over 60 years old?"}}
Oracle evidence1 model-reported source

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns Yes
Q18
1 · Guesser asks

Is the person primarily known as a television host or presenter?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person primarily known as a television host or presenter?"}}
Oracle evidence2 model-reported sources

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns No
Q19
1 · Guesser asks

Is the person African American?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person African American?"}}
Oracle evidence1 model-reported source

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns No
Q20
1 · Guesser asks

Is the person known for appearing in commercials or as a product spokesperson?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person known for appearing in commercials or as a product spokesperson?"}}
Oracle evidence2 model-reported sources

The Oracle reported these excerpts and URLs. They are not independently checked.

Source 02
American Express had a series of commercials in the 80s featuring celebrities asking viewers, “Do you know me?” ... This was Stephen King's commercial.
Open wokq.com
2 · Answer check returns Yes
Q21
1 · Guesser asks

Is the person known especially for a long-running beer commercial campaign?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person known especially for a long-running beer commercial campaign?"}}
Oracle evidence2 model-reported sources

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns No
Q22
1 · Guesser asks

Is the person known especially for insurance commercials?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person known especially for insurance commercials?"}}
Oracle evidence2 model-reported sources

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns No
Q23
1 · Guesser asks

Is the person known for Geico commercials?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person known for Geico commercials?"}}
Oracle evidence2 model-reported sources

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns No
Q24
1 · Guesser asks

Is the person known for Progressive insurance commercials?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person known for Progressive insurance commercials?"}}
2 · Answer check returns Unknown
Q25
1 · Guesser asks

Is the person known for Priceline commercials?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person known for Priceline commercials?"}}
Oracle evidence2 model-reported sources

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns No
Q26
1 · Guesser asks

Is the person known especially for automobile or car commercials?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person known especially for automobile or car commercials?"}}
Oracle evidence2 model-reported sources

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns No
Q27
1 · Guesser asks

Is the person known for a long-running campaign as a pitchman for a consumer product?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person known for a long-running campaign as a pitchman for a consumer product?"}}
2 · Answer check returns Unknown
Q28
1 · Guesser asks

Is the person known for Dos Equis commercials?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person known for Dos Equis commercials?"}}
Oracle evidence2 model-reported sources

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns No
Q29
1 · Guesser asks

Is the person known for lawn care or pest control commercials?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person known for lawn care or pest control commercials?"}}
Oracle evidence1 model-reported source

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns No
Q30
1 · Guesser asks

Is the person known for credit card commercials?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person known for credit card commercials?"}}
Oracle evidence2 model-reported sources

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns Yes
Q31
1 · Guesser asks

Is the person known for Capital One commercials?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person known for Capital One commercials?"}}
Oracle evidence2 model-reported sources

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns No
Q32
1 · Guesser asks

Is the person known for American Express commercials?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person known for American Express commercials?"}}
Oracle evidence1 model-reported source

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns Yes
Q33
1 · Guesser asks

Has the person appeared in major Hollywood films?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Has the person appeared in major Hollywood films?"}}
Oracle evidence2 model-reported sources

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns Yes
Q34
1 · Guesser asks

Is the person also known as a film director?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person also known as a film director?"}}
Oracle evidence1 model-reported source

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns Yes
Q35
1 · Guesser asks

Is the person known for the American Express 'My life. My card.' advertising campaign?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person known for the American Express 'My life. My card.' advertising campaign?"}}
Oracle evidence2 model-reported sources

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns No
Q36
1 · Guesser asks

Is the person Jerry Seinfeld?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person Jerry Seinfeld?"}}
Oracle evidence2 model-reported sources

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns No
Q37
1 · Guesser asks

Is the person also known for directing Academy Award-winning films?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person also known for directing Academy Award-winning films?"}}
Oracle evidence2 model-reported sources

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns No
Q38
1 · Guesser asks

Is the person primarily known for acting in both film and American Express commercials?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person primarily known for acting in both film and American Express commercials?"}}
Oracle evidence2 model-reported sources

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns No
Q39
1 · Guesser asks

Is the person Robert De Niro?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person Robert De Niro?"}}
Oracle evidence2 model-reported sources

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns No
Q40
1 · Guesser asks

Is the person known especially for Mastercard commercials?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person known especially for Mastercard commercials?"}}
Oracle evidence2 model-reported sources

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns No
Q41
1 · Guesser asks

Is the person known for Visa commercials?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person known for Visa commercials?"}}
Oracle evidence1 model-reported source

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns No
Q42
1 · Guesser asks

Is the person known as an actor who has done American Express commercials?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person known as an actor who has done American Express commercials?"}}
Oracle evidence2 model-reported sources

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns No
Q43
1 · Guesser asks

Is the person known for appearing as a celebrity endorser of American Express?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person known for appearing as a celebrity endorser of American Express?"}}
Oracle evidence1 model-reported source

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns Yes
Q44
1 · Guesser asks

Is the person white or Caucasian?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person white or Caucasian?"}}
Oracle evidence1 model-reported source

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns Yes
Q45
1 · Guesser asks

Is the person over 70 years old?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person over 70 years old?"}}
Oracle evidence1 model-reported source

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns Yes
Q46
1 · Guesser asks

Is the person Martin Scorsese?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person Martin Scorsese?"}}
Oracle evidence2 model-reported sources

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns No
Q47
1 · Guesser asks

Is the person primarily known as a film director rather than as an actor?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person primarily known as a film director rather than as an actor?"}}
Oracle evidence2 model-reported sources

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns No
Q48
1 · Guesser asks

Is the person known for appearing in American Express ads starring as himself?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person known for appearing in American Express ads starring as himself?"}}
Oracle evidence2 model-reported sources

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns Yes
Q49
1 · Guesser guesses

Clint Eastwood

American actor and director known for Westerns, Dirty Harry films, and American Express commercials

Validator rationale The proposal identifies Clint Eastwood, a different person from the trusted subject Stephen King.

Recorded Guesser outputRequired JSON format
{"result":{"action":"GUESS","description":"American actor and director known for Westerns, Dirty Harry films, and American Express commercials","name":"Clint Eastwood","question":null}}
2 · Validator returns No
Q50
1 · Guesser asks

Is the person known for directing films in addition to acting?

Recorded Guesser outputRequired JSON format
{"result":{"action":"ASK","description":null,"name":null,"question":"Is the person known for directing films in addition to acting?"}}
Oracle evidence2 model-reported sources

The Oracle reported these excerpts and URLs. They are not independently checked.

2 · Answer check returns Yes