AI leaderboard 1.0
- Models
- 18
- Rounds per model
- 35
- Total benchmark cost
- $175.27
- Total model time
- 33h 50m
Ranked models
Lower question scores are better.
Models rank by exact average score, then by narrower 95% confidence interval (CI). Equal scores and CI widths share a rank.
The score includes failed rounds. Every subject has equal weight.
| Rank | ||||||
|---|---|---|---|---|---|---|
| Rank 1 | Claude Fable 5 (high) | Question score12.06questions · lower is better10.13–13.98 | 100% | 100% | $0.1776 | 1m 59s |
| Rank 2 | Claude Fable 5.1 (high) | Question score12.11questions · lower is better8.81–15.41 | 97% | 99.8% 1 violation | $0.5014 | 3m 17s |
| Rank 3 | Claude Opus 5 (high) | Question score12.34questions · lower is better11.41–13.27 | 100% | 99.4% 3 violations | $0.0705 | 1m 13s |
| Rank 4 | Kimi K3 (high) | Question score12.74questions · lower is better9.16–16.33 | 94% | 97.7% 11 violations | $0.2032 | 6m 54s |
| Rank 5 | Gemini 3.8 Flash (high) | Question score13.14questions · lower is better11.52–14.77 | 100% | 100% | $0.1128 | 2m 6s |
| Rank 6 | gpt-oss-120B (high) | Question score13.63questions · lower is better9.79–17.46 | 94% | 90.6% 46 violations | $0.0363 | 1m 32s |
| Rank 7 | Gemini 3.7 Flash (high) | Question score14.03questions · lower is better10.78–17.28 | 97% | 100% | $0.0653 | 1m 17s |
| Rank 8 | Grok 4.6 (high) | Question score14.29questions · lower is better12.36–16.21 | 100% | 100% | $0.0653 | 3m 8s |
| Rank 9 | GPT-5.6 Sol (high) | Question score14.40questions · lower is better12.84–15.96 | 100% | 100% | $0.1711 | 2m 16s |
| Rank 10 | Claude Sonnet 5 (high) | Question score14.74questions · lower is better12.78–16.71 | 100% | 97.1% 16 violations | $0.0493 | 1m 24s |
| Rank 11 | GPT-6 Astra (high) | Question score15.09questions · lower is better11.88–18.30 | 94% | 100% | $0.3279 | 2m 42s |
| Rank 12 | Grok 4.5 (high) | Question score15.17questions · lower is better11.70–18.64 | 94% | 98.2% 10 violations | $0.0286 | 1m 10s |
| Rank 13 | GPT-5 Nano (medium) | Question score15.49questions · lower is better13.84–17.13 | 94% | 100% | $0.0138 | 5m 32s |
| Rank 14 | Gemini 3.6 Flash (high) | Question score16.89questions · lower is better14.08–19.69 | 97% | 99.8% 1 violation | $0.1705 | 1m 59s |
| Rank 15 | Ox Alpha (high) | Question score17.60questions · lower is better14.60–20.60 | 91% | 92.7% 47 violations | $0.0000 | 5m 14s |
| Rank 16 | GPT-5.6 Luna (high) | Question score17.74questions · lower is better14.74–20.75 | 91% | 99.8% 1 violation | $0.0184 | 1m 17s |
| Rank 17 | Mistral Medium 3.5 (high) | Question score20.06questions · lower is better16.11–24.01 | 89% | 99.7% 2 violations | $0.4113 | 14m 32s |
| Rank 18 | Llama 4 Maverick (non-thinking) | Question score32.23questions · lower is better26.82–37.63 | 49% | 100% | $0.0086 | 28s |
Rank 1Claude Fable 5 (high)anthropic12.06Question score
- 95% CI
- 10.13–13.98
- Solved
- 100%
- Guesser cost / round
- $0.1776
Rank 2Claude Fable 5.1 (high)anthropic12.11Question score
- 95% CI
- 8.81–15.41
- Solved
- 97%
- Guesser cost / round
- $0.5014
Rank 3Claude Opus 5 (high)anthropic12.34Question score
- 95% CI
- 11.41–13.27
- Solved
- 100%
- Guesser cost / round
- $0.0705
Rank 4Kimi K3 (high)moonshotai12.74Question score
- 95% CI
- 9.16–16.33
- Solved
- 94%
- Guesser cost / round
- $0.2032
Rank 5Gemini 3.8 Flash (high)google-ai-studio13.14Question score
- 95% CI
- 11.52–14.77
- Solved
- 100%
- Guesser cost / round
- $0.1128
Rank 6gpt-oss-120B (high)cerebras13.63Question score
- 95% CI
- 9.79–17.46
- Solved
- 94%
- Guesser cost / round
- $0.0363
Rank 7Gemini 3.7 Flash (high)google-ai-studio14.03Question score
- 95% CI
- 10.78–17.28
- Solved
- 97%
- Guesser cost / round
- $0.0653
Rank 8Grok 4.6 (high)xai14.29Question score
- 95% CI
- 12.36–16.21
- Solved
- 100%
- Guesser cost / round
- $0.0653
Rank 9GPT-5.6 Sol (high)openai14.40Question score
- 95% CI
- 12.84–15.96
- Solved
- 100%
- Guesser cost / round
- $0.1711
Rank 10Claude Sonnet 5 (high)anthropic14.74Question score
- 95% CI
- 12.78–16.71
- Solved
- 100%
- Guesser cost / round
- $0.0493
Rank 11GPT-6 Astra (high)openai15.09Question score
- 95% CI
- 11.88–18.30
- Solved
- 94%
- Guesser cost / round
- $0.3279
Rank 12Grok 4.5 (high)xai15.17Question score
- 95% CI
- 11.70–18.64
- Solved
- 94%
- Guesser cost / round
- $0.0286
Rank 13GPT-5 Nano (medium)openai15.49Question score
- 95% CI
- 13.84–17.13
- Solved
- 94%
- Guesser cost / round
- $0.0138
Rank 14Gemini 3.6 Flash (high)google-vertex16.89Question score
- 95% CI
- 14.08–19.69
- Solved
- 97%
- Guesser cost / round
- $0.1705
Rank 15Ox Alpha (high)stealth17.60Question score
- 95% CI
- 14.60–20.60
- Solved
- 91%
- Guesser cost / round
- $0.0000
Rank 16GPT-5.6 Luna (high)openai17.74Question score
- 95% CI
- 14.74–20.75
- Solved
- 91%
- Guesser cost / round
- $0.0184
Rank 17Mistral Medium 3.5 (high)mistral20.06Question score
- 95% CI
- 16.11–24.01
- Solved
- 89%
- Guesser cost / round
- $0.4113
Rank 18Llama 4 Maverick (non-thinking)parasail32.23Question score
- 95% CI
- 26.82–37.63
- Solved
- 49%
- Guesser cost / round
- $0.0086
Score distribution
Question score.
Lower is better. The blue marker is the average question score. The line is its 95% confidence interval (CI); darker lines are wider. The side plot shows each CI width in three equal bands.
Question scorelower is better95% CI · color = CI widthNarrowMediumWide
CI width stabilityRows follow question-score order · lower is betterColors describe CI width. Compare question scores separately.
- 1. Claude Fable 5 (high): 12.06 questions. Best score. 95% confidence interval of the average: 10.13–13.98 questions. CI width: 3.85 questions; narrow band on the displayed scale. View results for Claude Fable 5 (high)
- 2. Claude Fable 5.1 (high): 12.11 questions. 95% confidence interval of the average: 8.81–15.41 questions. CI width: 6.60 questions; medium band on the displayed scale. View results for Claude Fable 5.1 (high)
- 3. Claude Opus 5 (high): 12.34 questions. 95% confidence interval of the average: 11.41–13.27 questions. CI width: 1.86 questions; narrow band on the displayed scale. Smallest CI width. View results for Claude Opus 5 (high)
- 4. Kimi K3 (high): 12.74 questions. 95% confidence interval of the average: 9.16–16.33 questions. CI width: 7.18 questions; medium band on the displayed scale. View results for Kimi K3 (high)
- 5. Gemini 3.8 Flash (high): 13.14 questions. 95% confidence interval of the average: 11.52–14.77 questions. CI width: 3.25 questions; narrow band on the displayed scale. View results for Gemini 3.8 Flash (high)
- 6. gpt-oss-120B (high): 13.63 questions. 95% confidence interval of the average: 9.79–17.46 questions. CI width: 7.67 questions; medium band on the displayed scale. View results for gpt-oss-120B (high)
- 7. Gemini 3.7 Flash (high): 14.03 questions. 95% confidence interval of the average: 10.78–17.28 questions. CI width: 6.49 questions; medium band on the displayed scale. View results for Gemini 3.7 Flash (high)
- 8. Grok 4.6 (high): 14.29 questions. 95% confidence interval of the average: 12.36–16.21 questions. CI width: 3.85 questions; narrow band on the displayed scale. View results for Grok 4.6 (high)
- 9. GPT-5.6 Sol (high): 14.40 questions. 95% confidence interval of the average: 12.84–15.96 questions. CI width: 3.11 questions; narrow band on the displayed scale. View results for GPT-5.6 Sol (high)
- 10. Claude Sonnet 5 (high): 14.74 questions. 95% confidence interval of the average: 12.78–16.71 questions. CI width: 3.93 questions; narrow band on the displayed scale. View results for Claude Sonnet 5 (high)
- 11. GPT-6 Astra (high): 15.09 questions. 95% confidence interval of the average: 11.88–18.30 questions. CI width: 6.42 questions; medium band on the displayed scale. View results for GPT-6 Astra (high)
- 12. Grok 4.5 (high): 15.17 questions. 95% confidence interval of the average: 11.70–18.64 questions. CI width: 6.94 questions; medium band on the displayed scale. View results for Grok 4.5 (high)
- 13. GPT-5 Nano (medium): 15.49 questions. 95% confidence interval of the average: 13.84–17.13 questions. CI width: 3.29 questions; narrow band on the displayed scale. View results for GPT-5 Nano (medium)
- 14. Gemini 3.6 Flash (high): 16.89 questions. 95% confidence interval of the average: 14.08–19.69 questions. CI width: 5.61 questions; medium band on the displayed scale. View results for Gemini 3.6 Flash (high)
- 15. Ox Alpha (high): 17.60 questions. 95% confidence interval of the average: 14.60–20.60 questions. CI width: 5.99 questions; medium band on the displayed scale. View results for Ox Alpha (high)
- 16. GPT-5.6 Luna (high): 17.74 questions. 95% confidence interval of the average: 14.74–20.75 questions. CI width: 6.01 questions; medium band on the displayed scale. View results for GPT-5.6 Luna (high)
- 17. Mistral Medium 3.5 (high): 20.06 questions. 95% confidence interval of the average: 16.11–24.01 questions. CI width: 7.90 questions; medium band on the displayed scale. View results for Mistral Medium 3.5 (high)
- 18. Llama 4 Maverick (non-thinking): 32.23 questions. 95% confidence interval of the average: 26.82–37.63 questions. CI width: 10.81 questions; wide band on the displayed scale. View results for Llama 4 Maverick (non-thinking)
The 95% CI comes from repeated rounds on this edition's subjects. It does not cover other subjects, model versions, or providers. The three CI width bands split the shown range into equal parts; they are not quality grades. All models, A to Z →