What is an LLM leaderboard?

An LLM leaderboard is a public ranking of AI language models, ordered either by their scores on standardized tests or by how often real people prefer their answers in blind head-to-head matchups.

If a benchmark is the exam, the leaderboard is the results board posted on the wall. It turns dozens of scattered scores into one sortable list.

There are two main flavors:

• Benchmark leaderboards collect test scores — coding, math, general knowledge — and rank models by the numbers.
• Preference leaderboards like LMArena work differently: you ask a question, two anonymous models answer, and you vote for the better response. Millions of votes get crunched into chess-style Elo ratings.

Leaderboards are genuinely useful for spotting trends — which lab is shipping, which open model is catching up. But read them with squinted eyes.

Rankings churn every few weeks as new models drop, and the top spots are usually separated by margins too small to feel in everyday use. The model ranked first overall might also rank third at the one thing you actually need, like coding or long-form writing.

So use a leaderboard to build a shortlist, not to crown a winner. Run a quick evaluation of two or three candidates on your own work — and check the model tracker for current scores and pricing side by side.