Official run

Muse Spark 1.3 ContributorHigh

In this Twenty Questions LLM benchmark, Muse Spark 1.3 Contributor (high) ranks 8 of 12 with a 18.37 question score and 97% success. Lower is better. The run contains 30 scored episodes across 10 subjects with 3 trials per subject. Choose a subject to inspect its attempts.

Rank 8 of 12metaHigh

Question score18.37questions · lower is better95% CI 16.61–20.13

Success: 97%
Contract compliance: 100%
Guesser cost: $0.15
Guesser time: 2h 33m

Drill down

Subjects.

01Albert Einstein13.33 avg 02Albert Schweitzer20.00 avg 03Garfield14.33 avg 04Achilles12.33 avg 05Genghis Khan14.33 avg 06Bike pump34.00 avg 07Spider web19.67 avg 08Eyebrow17.00 avg 09Moon10.00 avg 10Door handle28.67 avg

Complete benchmark

Run ledger.

All Guesser and benchmark-support activity.

Total tokens: 5,837,825
Wall-clock runtime: 36h 7m
Guesser calls: 580
Completed: Sep 10, 2026, 7:35:39 PM

Reliability

Output contract clean.

All 580 evaluated outputs matched the public structured-action contract.

Provenance

Execution: BX-20260909-B-0003-experimental-M0026-003
Model ID: M-0026
Benchmark: B-0003
Base seed: 0
Git commit: 2d5409df2ba6

Recorded cost

By role.

Cost composition.Share of full-run cost

Guesser5.7%$0.15
Primary Oracle50.9%$1.32
Reviewer24.7%$0.64
Judge18.5%$0.48
Validator0.2%$0.01

Excluded repair overhead$0.022 superseded attempts across 1 affected trial. Not included above.

Roles in this cost:

Guesser: Asks the questions and submits the scored guess.
Primary Oracle: Searches for evidence and proposes an answer.
Reviewer: Checks each Oracle YES or NO independently.
Judge: Decides when the Oracle and Reviewer disagree.
Validator: Checks a submitted guess against the trusted subject.

Read the role and answer-checking method →

Run configuration

Models.

Model identity, prompt versions, routing, calls, and recorded cost for the full run.

GuesserModel under test

meta/muse-spark-1.3-contributor

Asks the questions and submits the scored guess.

Calls: 580
Cost: $0.1473
Reasoning: High
Routing: Exact provider · meta
Resolved model: meta/muse-spark-1.3-contributor
Resolved provider: Meta
Prompt contract: stateful-category-guesser-v16-five-answer-category-guide
Configuration: M-0026

Resolved provider details

Meta580 calls$0.14739,190.97 s

Fallback calls: 0
Provider unreported: 0

Game support

Oracle and adjudication.

These models support the game. They are fixed across the run and are not under test.

Primary Oracle

openai/gpt-5.6-luna

Searches for evidence and proposes an answer.

Calls: 494
Cost: $1.3211
Reasoning: Medium
Routing: Exact provider · openai
Resolved model: openai/gpt-5.6-luna
Resolved provider: OpenAI
Prompt contract: live-web-oracle-v17-labelled-source-context

Resolved provider details

OpenAI494 calls$1.32114,591.15 s

Fallback calls: 0
Provider unreported: 0

Reviewer

google/gemini-3.5-flash-lite

Checks each Oracle YES or NO independently.

Calls: 478
Cost: $0.6418
Reasoning: Medium
Routing: Exact provider · google-ai-studio
Resolved provider: Google AI Studio

Resolved provider details

Google AI Studio478 calls$0.6418815.95 s

Fallback calls: 0
Provider unreported: 0

Judge

anthropic/claude-opus-5

Decides when the Oracle and Reviewer disagree.

Calls: 22
Cost: $0.4798
Reasoning: Medium
Routing: OpenRouter automatic routing
Resolved provider: Claude Platform on AWS

Resolved provider details

Claude Platform on AWS22 calls$0.4798162.41 s

Fallback calls: 0
Provider unreported: 0

Guess Validator

openai/gpt-5.6-luna

Checks a submitted guess against the trusted subject.

Calls: 30
Cost: $0.0057
Reasoning: Medium
Routing: Exact provider · openai
Resolved model: openai/gpt-5.6-luna
Resolved provider: OpenAI
Prompt contract: strict-guess-validator-v2-generic-kinds
Configuration: gpt-5.6-luna-validator

Resolved provider details

OpenAI30 calls$0.005747.76 s

Fallback calls: 0
Provider unreported: 0