Cost and quality

Efficiency 1.1

Compare question quality with the recorded cost of the model under test.

Models: 11
Guesser cost range: 176×
Pareto-efficient: 5 of 11
Direction: Lower is better

Official efficiency ranking

Distance from the lower-left ideal.

Question score and Guesser cost are each normalized from 0 to 1 across this cohort. The ranking measures equal-weight distance from their combined minimum. Lower is better.

Normalized distance · lower is betterSelect a model row to view its full run

Trade-off map

Normalized cost and question score.

Both axes use the same 0-to-1 scale. Dashed curves mark equal distance from the lower-left ideal.

Diamond Pareto-efficient - no other model is both cheaper and better Circle Other ranked model

Lower-left is betterCurves show equal ideal distance

Gemini 3.7 Flash (high)0.263 distance · Pareto-efficient
Gemini 3.8 Flash (high)0.321 distance · Pareto-efficient
GPT-5.6 Sol (high)0.345 distance
Claude Opus 5 (high)0.438 distance
GPT-5.6 Luna (high)0.476 distance · Pareto-efficient
Grok 4.6 (high)0.539 distance
gpt-oss-120B (high)0.603 distance
GLM-5.3-Flash (high)0.619 distance · Pareto-efficient
GPT-6 Astra (high)0.631 distance · Pareto-efficient
MiniMax M3 (high)1.000 distance
Claude Fable 5.1 (high)1.004 distance

Efficiency ranking
Ideal-distance rank	Model	Idealdistance	Paretoefficient	Questionrank	Questionscore	Guesser costper episode	Success
1	Gemini 3.7 Flash (high)google-ai-studio	0.263	Yes	4	16.57	$0.0525	93%
2	Gemini 3.8 Flash (high)google-ai-studio	0.321	Yes	2	14.87	$0.0875	100%
3	GPT-5.6 Sol (high)openai	0.345		6	18.07	$0.0557	97%
4	Claude Opus 5 (high)anthropic	0.438		6	18.07	$0.0934	93%
5	GPT-5.6 Luna (high)openai	0.476	Yes	8	21.03	$0.0063	90%
6	Grok 4.6 (high)xai	0.539		5	17.13	$0.1367	100%
7	gpt-oss-120B (high)cerebras	0.603		9	22.23	$0.0682	67%
8	GLM-5.3-Flash (high)z-ai	0.619	Yes	10	23.27	$0.0016	73%
9	GPT-6 Astra (high)openai	0.631	Yes	1	13.67	$0.1756	100%
10	MiniMax M3 (high)coreweave	1.000		11	29.17	$0.0100	50%
11	Claude Fable 5.1 (high)anthropic	1.004		3	15.10	$0.2772	93%

#1Gemini 3.7 Flash (high)google-ai-studio

Ideal distance: 0.263
Pareto-efficient: Yes
Question score: 16.57
Guesser cost / episode: $0.0525

Explore full run · questions, answers & evidence

#2Gemini 3.8 Flash (high)google-ai-studio

Ideal distance: 0.321
Pareto-efficient: Yes
Question score: 14.87
Guesser cost / episode: $0.0875

Explore full run · questions, answers & evidence

#3GPT-5.6 Sol (high)openai

Ideal distance: 0.345
Pareto-efficient: No
Question score: 18.07
Guesser cost / episode: $0.0557

Explore full run · questions, answers & evidence

#4Claude Opus 5 (high)anthropic

Ideal distance: 0.438
Pareto-efficient: No
Question score: 18.07
Guesser cost / episode: $0.0934

Explore full run · questions, answers & evidence

#5GPT-5.6 Luna (high)openai

Ideal distance: 0.476
Pareto-efficient: Yes
Question score: 21.03
Guesser cost / episode: $0.0063

Explore full run · questions, answers & evidence

#6Grok 4.6 (high)xai

Ideal distance: 0.539
Pareto-efficient: No
Question score: 17.13
Guesser cost / episode: $0.1367

Explore full run · questions, answers & evidence

#7gpt-oss-120B (high)cerebras

Ideal distance: 0.603
Pareto-efficient: No
Question score: 22.23
Guesser cost / episode: $0.0682

Explore full run · questions, answers & evidence

#8GLM-5.3-Flash (high)z-ai

Ideal distance: 0.619
Pareto-efficient: Yes
Question score: 23.27
Guesser cost / episode: $0.0016

Explore full run · questions, answers & evidence

#9GPT-6 Astra (high)openai

Ideal distance: 0.631
Pareto-efficient: Yes
Question score: 13.67
Guesser cost / episode: $0.1756

Explore full run · questions, answers & evidence

#10MiniMax M3 (high)coreweave

Ideal distance: 1.000
Pareto-efficient: No
Question score: 29.17
Guesser cost / episode: $0.0100

Explore full run · questions, answers & evidence

#11Claude Fable 5.1 (high)anthropic

Ideal distance: 1.004
Pareto-efficient: No
Question score: 15.10
Guesser cost / episode: $0.2772

Explore full run · questions, answers & evidence

How it is calculated

Normalized ideal distance.

Both measures have equal weight after cohort min/max normalization. Lower is better.

Formula√(normalized question score² + normalized Guesser cost²)

Calculation detailsSteps and limits

Normalize question score as (value − cohort minimum) ÷ cohort range.
Normalize Guesser cost per episode with the same calculation.
Measure Euclidean distance from (0, 0). Lower is better.

A model with normalized question score 0.06 and normalized cost 0.08 has distance √(0.06² + 0.08²) = 0.10.

Question score still uses the average penalized trial values. Failed trials therefore remain part of the quality dimension.

Adding or removing a model can change every normalized value and rank.