OpenAI
The 2024-08-06 version of GPT-4o offers improved performance in structured outputs, with the ability to supply a JSON schema in the respone_format. Read more [here](https://openai.com/index/introducing-structured-outputs-in-the-api/). GPT-4o ("o" for "omni") is...
OpenAI model releasesRank #367 across 566
No rank
Rank #303 across 371
Percentile score by analysis domain.
* Cost is inverted: lower input, output, and blended prices rank higher.
Higher bars mean stronger relative placement.
| Metric | Domain | Value | Rank |
|---|---|---|---|
| Artificial Analysis Intelligence Index | overall | 9.6 | #367 |
| MMLU-Pro | reasoning | 74.7% | #189 |
| GPQA | reasoning | 52.1% | #382 |
| Humanity's Last Exam | reasoning | 2.9% | #530 |
| LiveCodeBench |
| coding |
| 31.7% |
| #214 |
| SciCode | coding, reasoning | 33.1% | #270 |
| MATH-500 | math | 79.5% | #110 |
| AIME | math | 11.7% | #122 |
| Output Speed | speed | 99.6 tok/s | #142 |
| Time to First Token | speed | 0.55s | #43 |
| Blended Price | cost | $4.38/M | #303 |
| Input Price | cost | $2.50/M | #316 |
| Output Price | cost | $10.00/M | #293 |
| Value Index | cost, overall | 2.2 | #346 |