Repeated-trial stability

Stability 1.1

Whether a model produces similar scores across repeated trials on the same fixed subjects.

Models: 11
Smallest CI width: 2.36
Largest CI width: 8.90
More stable: Smaller CI width

Two-dimensional view

Stability.

Each dot compares two results. Lower means a better average question score. Further left means a smaller CI width, so the model produced more consistent results across repeated trials on the fixed subjects.

Lower-left is betterSelect a model point to view its full run

GPT-6 Astra (high)13.67 q · CI width 2.36
Claude Fable 5.1 (high)15.10 q · CI width 2.69
Claude Opus 5 (high)18.07 q · CI width 3.13
Grok 4.6 (high)17.13 q · CI width 3.19
Gemini 3.8 Flash (high)14.87 q · CI width 3.35
Gemini 3.7 Flash (high)16.57 q · CI width 4.44
GPT-5.6 Luna (high)21.03 q · CI width 5.05
gpt-oss-120B (high)22.23 q · CI width 5.31
GPT-5.6 Sol (high)18.07 q · CI width 7.37
GLM-5.3-Flash (high)23.27 q · CI width 7.38
MiniMax M3 (high)29.17 q · CI width 8.90

Stability ranking
Stability rank	Model	CIwidth	Question rank	Question score	95% CI	Success
1	GPT-6 Astra (high)openai	2.36	1	13.67	12.49–14.85	100%
2	Claude Fable 5.1 (high)anthropic	2.69	3	15.10	13.76–16.44	93%
3	Claude Opus 5 (high)anthropic	3.13	6	18.07	16.50–19.63	93%
4	Grok 4.6 (high)xai	3.19	5	17.13	15.54–18.73	100%
5	Gemini 3.8 Flash (high)google-ai-studio	3.35	2	14.87	13.19–16.54	100%
6	Gemini 3.7 Flash (high)google-ai-studio	4.44	4	16.57	14.35–18.79	93%
7	GPT-5.6 Luna (high)openai	5.05	8	21.03	18.51–23.56	90%
8	gpt-oss-120B (high)cerebras	5.31	9	22.23	19.58–24.89	67%
9	GPT-5.6 Sol (high)openai	7.37	6	18.07	14.38–21.75	97%
10	GLM-5.3-Flash (high)z-ai	7.38	10	23.27	19.58–26.96	73%
11	MiniMax M3 (high)coreweave	8.90	11	29.17	24.71–33.62	50%

#1GPT-6 Astra (high)openai

CI width: 2.36
Question score: 13.67
95% CI: 12.49–14.85
Success: 100%

Explore full run · questions, answers & evidence

#2Claude Fable 5.1 (high)anthropic

CI width: 2.69
Question score: 15.10
95% CI: 13.76–16.44
Success: 93%

Explore full run · questions, answers & evidence

#3Claude Opus 5 (high)anthropic

CI width: 3.13
Question score: 18.07
95% CI: 16.50–19.63
Success: 93%

Explore full run · questions, answers & evidence

#4Grok 4.6 (high)xai

CI width: 3.19
Question score: 17.13
95% CI: 15.54–18.73
Success: 100%

Explore full run · questions, answers & evidence

#5Gemini 3.8 Flash (high)google-ai-studio

CI width: 3.35
Question score: 14.87
95% CI: 13.19–16.54
Success: 100%

Explore full run · questions, answers & evidence

#6Gemini 3.7 Flash (high)google-ai-studio

CI width: 4.44
Question score: 16.57
95% CI: 14.35–18.79
Success: 93%

Explore full run · questions, answers & evidence

#7GPT-5.6 Luna (high)openai

CI width: 5.05
Question score: 21.03
95% CI: 18.51–23.56
Success: 90%

Explore full run · questions, answers & evidence

#8gpt-oss-120B (high)cerebras

CI width: 5.31
Question score: 22.23
95% CI: 19.58–24.89
Success: 67%

Explore full run · questions, answers & evidence

#9GPT-5.6 Sol (high)openai

CI width: 7.37
Question score: 18.07
95% CI: 14.38–21.75
Success: 97%

Explore full run · questions, answers & evidence

#10GLM-5.3-Flash (high)z-ai

CI width: 7.38
Question score: 23.27
95% CI: 19.58–26.96
Success: 73%

Explore full run · questions, answers & evidence

#11MiniMax M3 (high)coreweave

CI width: 8.90
Question score: 29.17
95% CI: 24.71–33.62
Success: 50%

Explore full run · questions, answers & evidence

How it is calculated

CI width.

A smaller CI width means the model produced more consistent aggregate results across the current repeated trials.

FormulaCI width = upper 95% CI bound − lower 95% CI bound

Calculation detailsSteps, example, interpretation, and limits

Calculate the fixed-subject repeated-trial 95% CI.
Subtract its lower bound from its upper bound.
Sort exact widths from smallest to largest.

Example: 17.46 − 9.79 = 7.67 questions.

Every model uses the same 95% confidence level, so the level itself cannot define the order. CI width is the comparison measure.

Stable does not mean good. A model can produce a poor score consistently and rank well here. A strong average with a large CI width ranks lower because its repeated results vary more.

This is an approximate stability measure for the current fixed subjects. It is not a prediction interval for individual trials or a pairwise significance test.