Model cost 1.0
- Models
- 18
- Total benchmark cost
- $175.27
- Total Guesser cost
- $85.11
- Answer-checking share
- 51%
Guesser cost
Guesser cost across the run.
Each bar is the provider-reported cost of all Guesser calls in the run. It excludes answer-checking costs and earlier attempts of rounds rerun after a test system failure.
- 1. Ox Alpha (high): $0.00. View results for Ox Alpha (high)
- 2. Llama 4 Maverick (non-thinking): $0.30. View results for Llama 4 Maverick (non-thinking)
- 3. GPT-5 Nano (medium): $0.48. View results for GPT-5 Nano (medium)
- 4. GPT-5.6 Luna (high): $0.64. View results for GPT-5.6 Luna (high)
- 5. Grok 4.5 (high): $1.00. View results for Grok 4.5 (high)
- 6. gpt-oss-120B (high): $1.27. View results for gpt-oss-120B (high)
- 7. Claude Sonnet 5 (high): $1.72. View results for Claude Sonnet 5 (high)
- 8. Gemini 3.7 Flash (high): $2.28. View results for Gemini 3.7 Flash (high)
- 9. Grok 4.6 (high): $2.29. View results for Grok 4.6 (high)
- 10. Claude Opus 5 (high): $2.47. View results for Claude Opus 5 (high)
- 11. Gemini 3.8 Flash (high): $3.95. View results for Gemini 3.8 Flash (high)
- 12. Gemini 3.6 Flash (high): $5.97. View results for Gemini 3.6 Flash (high)
- 13. GPT-5.6 Sol (high): $5.99. View results for GPT-5.6 Sol (high)
- 14. Claude Fable 5 (high): $6.22. View results for Claude Fable 5 (high)
- 15. Kimi K3 (high): $7.11. View results for Kimi K3 (high)
- 16. GPT-6 Astra (high): $11.48. View results for GPT-6 Astra (high)
- 17. Mistral Medium 3.5 (high): $14.39. View results for Mistral Medium 3.5 (high)
- 18. Claude Fable 5.1 (high): $17.55. View results for Claude Fable 5.1 (high)
Total benchmark cost
Where the total cost came from.
Bar length shows the total benchmark cost. Colors separate the Guesser, the Oracle, and the other answer checks (Reviewer, Judge, and Validator). Open the exact breakdown to compare those three.
- Guesser
- Oracle
- Other answer checks
Select a model row to view its results.
Exact answer-checking costs
| Model | Reviewer | Judge | Validator |
|---|---|---|---|
| Llama 4 Maverick (non-thinking) | $0.31 | $0.60 | $0.25 |
| GPT-5 Nano (medium) | $0.55 | $0.75 | $0.02 |
| gpt-oss-120B (high) | $0.46 | $0.63 | $0.01 |
| Grok 4.5 (high) | $0.53 | $0.65 | $0.01 |
| Claude Sonnet 5 (high) | $0.47 | $0.49 | $0.01 |
| Claude Opus 5 (high) | $0.41 | $0.49 | $0.01 |
| GPT-5.6 Luna (high) | $0.67 | $0.91 | $0.01 |
| Claude Fable 5 (high) | $0.35 | $0.38 | $0.00 |
| Grok 4.6 (high) | $0.54 | $1.01 | $0.00 |
| Gemini 3.7 Flash (high) | $0.48 | $0.85 | $0.00 |
| Ox Alpha (high) | $0.65 | $1.16 | $0.01 |
| Gemini 3.6 Flash (high) | $0.48 | $0.46 | $0.01 |
| GPT-5.6 Sol (high) | $0.41 | $0.35 | $0.01 |
| Kimi K3 (high) | $0.37 | $0.37 | $0.02 |
| Gemini 3.8 Flash (high) | $0.46 | $0.75 | $0.01 |
| Mistral Medium 3.5 (high) | $0.50 | $0.47 | $0.06 |
| GPT-6 Astra (high) | $0.49 | $0.62 | $0.01 |
| Claude Fable 5.1 (high) | $0.48 | $0.86 | $0.01 |
- Llama 4 Maverick (non-thinking): $3.73. Guesser $0.30. Oracle $2.27. Other answer checks $1.16. Reviewer $0.31. Judge $0.60. Validator $0.25. View results for Llama 4 Maverick (non-thinking)
- GPT-5 Nano (medium): $4.59. Guesser $0.48. Oracle $2.78. Other answer checks $1.32. Reviewer $0.55. Judge $0.75. Validator $0.02. View results for GPT-5 Nano (medium)
- gpt-oss-120B (high): $4.64. Guesser $1.27. Oracle $2.26. Other answer checks $1.11. Reviewer $0.46. Judge $0.63. Validator $0.01. View results for gpt-oss-120B (high)
- Grok 4.5 (high): $4.99. Guesser $1.00. Oracle $2.80. Other answer checks $1.19. Reviewer $0.53. Judge $0.65. Validator $0.01. View results for Grok 4.5 (high)
- Claude Sonnet 5 (high): $5.01. Guesser $1.72. Oracle $2.31. Other answer checks $0.97. Reviewer $0.47. Judge $0.49. Validator $0.01. View results for Claude Sonnet 5 (high)
- Claude Opus 5 (high): $5.47. Guesser $2.47. Oracle $2.08. Other answer checks $0.92. Reviewer $0.41. Judge $0.49. Validator $0.01. View results for Claude Opus 5 (high)
- GPT-5.6 Luna (high): $5.88. Guesser $0.64. Oracle $3.65. Other answer checks $1.59. Reviewer $0.67. Judge $0.91. Validator $0.01. View results for GPT-5.6 Luna (high)
- Claude Fable 5 (high): $7.87. Guesser $6.22. Oracle $0.91. Other answer checks $0.74. Reviewer $0.35. Judge $0.38. Validator $0.00. View results for Claude Fable 5 (high)
- Grok 4.6 (high): $8.25. Guesser $2.29. Oracle $4.41. Other answer checks $1.56. Reviewer $0.54. Judge $1.01. Validator $0.00. View results for Grok 4.6 (high)
- Gemini 3.7 Flash (high): $9.58. Guesser $2.28. Oracle $5.97. Other answer checks $1.33. Reviewer $0.48. Judge $0.85. Validator $0.00. View results for Gemini 3.7 Flash (high)
- Ox Alpha (high): $9.78. Guesser $0.00. Oracle $7.96. Other answer checks $1.82. Reviewer $0.65. Judge $1.16. Validator $0.01. View results for Ox Alpha (high)
- Gemini 3.6 Flash (high): $9.86. Guesser $5.97. Oracle $2.94. Other answer checks $0.95. Reviewer $0.48. Judge $0.46. Validator $0.01. View results for Gemini 3.6 Flash (high)
- GPT-5.6 Sol (high): $9.97. Guesser $5.99. Oracle $3.21. Other answer checks $0.77. Reviewer $0.41. Judge $0.35. Validator $0.01. View results for GPT-5.6 Sol (high)
- Kimi K3 (high): $10.09. Guesser $7.11. Oracle $2.22. Other answer checks $0.75. Reviewer $0.37. Judge $0.37. Validator $0.02. View results for Kimi K3 (high)
- Gemini 3.8 Flash (high): $10.82. Guesser $3.95. Oracle $5.66. Other answer checks $1.21. Reviewer $0.46. Judge $0.75. Validator $0.01. View results for Gemini 3.8 Flash (high)
- Mistral Medium 3.5 (high): $18.55. Guesser $14.39. Oracle $3.12. Other answer checks $1.03. Reviewer $0.50. Judge $0.47. Validator $0.06. View results for Mistral Medium 3.5 (high)
- GPT-6 Astra (high): $22.60. Guesser $11.48. Oracle $10.01. Other answer checks $1.11. Reviewer $0.49. Judge $0.62. Validator $0.01. View results for GPT-6 Astra (high)
- Claude Fable 5.1 (high): $23.61. Guesser $17.55. Oracle $4.71. Other answer checks $1.35. Reviewer $0.48. Judge $0.86. Validator $0.01. View results for Claude Fable 5.1 (high)
| Benchmark cost rank | ||||||
|---|---|---|---|---|---|---|
| Rank 1 | Llama 4 Maverick (non-thinking) | $0.0086 | $0.1064 | $0.0979 | 92% | $3.73 |
| Rank 2 | GPT-5 Nano (medium) | $0.0138 | $0.1311 | $0.1173 | 89% | $4.59 |
| Rank 3 | gpt-oss-120B (high) | $0.0363 | $0.1326 | $0.0963 | 73% | $4.64 |
| Rank 4 | Grok 4.5 (high) | $0.0286 | $0.1426 | $0.1140 | 80% | $4.99 |
| Rank 5 | Claude Sonnet 5 (high) | $0.0493 | $0.1431 | $0.0938 | 66% | $5.01 |
| Rank 6 | Claude Opus 5 (high) | $0.0705 | $0.1562 | $0.0857 | 55% | $5.47 |
| Rank 7 | GPT-5.6 Luna (high) | $0.0184 | $0.1681 | $0.1497 | 89% | $5.88 |
| Rank 8 | Claude Fable 5 (high) | $0.1776 | $0.2248 | $0.0472 | 21% | $7.87 |
| Rank 9 | Grok 4.6 (high) | $0.0653 | $0.2357 | $0.1704 | 72% | $8.25 Rerun cost (excluded): $1.25 |
| Rank 10 | Gemini 3.7 Flash (high) | $0.0653 | $0.2737 | $0.2084 | 76% | $9.58 |
| Rank 11 | Ox Alpha (high) | $0.0000 | $0.2793 | $0.2793 | 100% | $9.78 Rerun cost (excluded): $2.53 |
| Rank 12 | Gemini 3.6 Flash (high) | $0.1705 | $0.2816 | $0.1111 | 39% | $9.86 |
| Rank 13 | GPT-5.6 Sol (high) | $0.1711 | $0.2849 | $0.1139 | 40% | $9.97 |
| Rank 14 | Kimi K3 (high) | $0.2032 | $0.2882 | $0.0850 | 29% | $10.09 |
| Rank 15 | Gemini 3.8 Flash (high) | $0.1128 | $0.3093 | $0.1965 | 64% | $10.82 |
| Rank 16 | Mistral Medium 3.5 (high) | $0.4113 | $0.5300 | $0.1187 | 22% | $18.55 Rerun cost (excluded): $1.72 |
| Rank 17 | GPT-6 Astra (high) | $0.3279 | $0.6456 | $0.3177 | 49% | $22.60 Rerun cost (excluded): $3.15 |
| Rank 18 | Claude Fable 5.1 (high) | $0.5014 | $0.6747 | $0.1733 | 26% | $23.61 |
Rank 1Llama 4 Maverick (non-thinking)parasail$0.0086Guesser cost / round
- Benchmark cost / round
- $0.1064
- Benchmark run cost
- $3.73
Rank 2GPT-5 Nano (medium)openai$0.0138Guesser cost / round
- Benchmark cost / round
- $0.1311
- Benchmark run cost
- $4.59
Rank 3gpt-oss-120B (high)cerebras$0.0363Guesser cost / round
- Benchmark cost / round
- $0.1326
- Benchmark run cost
- $4.64
Rank 4Grok 4.5 (high)xai$0.0286Guesser cost / round
- Benchmark cost / round
- $0.1426
- Benchmark run cost
- $4.99
Rank 5Claude Sonnet 5 (high)anthropic$0.0493Guesser cost / round
- Benchmark cost / round
- $0.1431
- Benchmark run cost
- $5.01
Rank 6Claude Opus 5 (high)anthropic$0.0705Guesser cost / round
- Benchmark cost / round
- $0.1562
- Benchmark run cost
- $5.47
Rank 7GPT-5.6 Luna (high)openai$0.0184Guesser cost / round
- Benchmark cost / round
- $0.1681
- Benchmark run cost
- $5.88
Rank 8Claude Fable 5 (high)anthropic$0.1776Guesser cost / round
- Benchmark cost / round
- $0.2248
- Benchmark run cost
- $7.87
Rank 9Grok 4.6 (high)xai$0.0653Guesser cost / round
- Benchmark cost / round
- $0.2357
- Benchmark run cost
- $8.25
- Rerun cost (excluded)
- $1.25
Rank 10Gemini 3.7 Flash (high)google-ai-studio$0.0653Guesser cost / round
- Benchmark cost / round
- $0.2737
- Benchmark run cost
- $9.58
Rank 11Ox Alpha (high)stealth$0.0000Guesser cost / round
- Benchmark cost / round
- $0.2793
- Benchmark run cost
- $9.78
- Rerun cost (excluded)
- $2.53
Rank 12Gemini 3.6 Flash (high)google-vertex$0.1705Guesser cost / round
- Benchmark cost / round
- $0.2816
- Benchmark run cost
- $9.86
Rank 13GPT-5.6 Sol (high)openai$0.1711Guesser cost / round
- Benchmark cost / round
- $0.2849
- Benchmark run cost
- $9.97
Rank 14Kimi K3 (high)moonshotai$0.2032Guesser cost / round
- Benchmark cost / round
- $0.2882
- Benchmark run cost
- $10.09
Rank 15Gemini 3.8 Flash (high)google-ai-studio$0.1128Guesser cost / round
- Benchmark cost / round
- $0.3093
- Benchmark run cost
- $10.82
Rank 16Mistral Medium 3.5 (high)mistral$0.4113Guesser cost / round
- Benchmark cost / round
- $0.5300
- Benchmark run cost
- $18.55
- Rerun cost (excluded)
- $1.72
Rank 17GPT-6 Astra (high)openai$0.3279Guesser cost / round
- Benchmark cost / round
- $0.6456
- Benchmark run cost
- $22.60
- Rerun cost (excluded)
- $3.15
Rank 18Claude Fable 5.1 (high)anthropic$0.5014Guesser cost / round
- Benchmark cost / round
- $0.6747
- Benchmark run cost
- $23.61
Costs are as reported by the providers for the selected official runs. Earlier attempts of rounds rerun after a test system failure are excluded. A missing provider price can affect the comparison.