Understanding AI Benchmarks: What MMLU, GPQA, and Arena Elo Actually Mean
A plain-English guide to AI benchmarks like MMLU, GPQA, SWE-Bench, and Chatbot Arena Elo, explaining what they measure and why no single score tells the whole story.
Articles Tagged "Benchmarks"
Coding Benchmarks Leaderboard: SWE-Bench, Terminal-Bench, and LiveCodeBench
Rankings of the best AI models for coding tasks across SWE-Bench, Terminal-Bench, and LiveCodeBench benchmarks, measuring real-world software engineering and algorithmic problem-solving ability.
Reasoning Benchmarks: GPQA, AIME, and Humanity's Last Exam
Rankings of AI models on the hardest reasoning benchmarks available: GPQA Diamond, AIME competition math, and the notoriously difficult Humanity's Last Exam.
Multimodal AI Leaderboard: Vision, Video, and Beyond
Rankings of the best multimodal AI models for image understanding, video analysis, and visual reasoning, covering MMMU-Pro, Video-MMMU, and more.
MMLU-Pro Leaderboard: Graduate-Level Knowledge Rankings
Complete MMLU-Pro benchmark rankings measuring graduate-level knowledge across 14 subjects with 12,000 questions and 10 answer options per question.
DeepSeek V3.2 Goes Open Source Under MIT License, Matches GPT-5 Performance
DeepSeek releases V3.2 under MIT license with 671B MoE architecture, matching GPT-5 at one-tenth the cost and achieving gold-medal performance on IMO and IOI competitions.
The Gap Between Open-Source and Proprietary AI Has Effectively Vanished
Analysis of how the MMLU benchmark gap between open-source and proprietary AI narrowed from 17.5 to 0.3 percentage points in a single year, reshaping the industry landscape.
Claude Opus 4.5
Anthropic's November 2025 flagship model delivers top SWE-bench scores, a new effort parameter for reasoning control, and a 66% price cut from its predecessor.
Claude Haiku 4.5
Anthropic's fastest and most cost-efficient model, delivering 73.3% on SWE-bench Verified and first-in-family extended thinking and computer use at $1/$5 per million tokens.
GPT-OSS 20B
OpenAI's open-weight 21B MoE reasoning model with 131K context, Apache 2.0 license, and o3-mini-level benchmark performance running in 16 GB of memory.
Gemini 2.5 Pro
Google DeepMind's flagship thinking model with 1M-token context, 84% GPQA Diamond, and native multimodal understanding of text, images, audio, and video.
OpenAI o3-pro
OpenAI's maximum-compute reasoning model targets the hardest problems where o3 falls short, at $20/$80 per million tokens.
