What is an LLM's parameter count, and does size matter?

A model's parameter count is the number of internal "knobs" — the numerical weights — it adjusted while training on data, and no, a bigger number doesn't automatically make a better model anymore.

What actually is a parameter?

Parameters are what the network actually learned: each one is a small number that gets nudged during training until the whole system gets better at predicting text. For years, parameter count was the best rough proxy for capability — going from millions to hundreds of billions of parameters is what took language models from autocomplete toys to something that could hold a real conversation. More knobs, more room to store patterns.

If size isn't everything, what changed?

That's still true up to a point, but it's not the whole story anymore. Three shifts changed the math:

• Training data quality matters as much as size — a smaller model trained on carefully curated data can out-punch a larger model trained on messier data.
• Distillation lets engineers compress a big "teacher" model's knowledge into a much smaller "student" model that keeps most of the performance at a fraction of the size.
• Mixture-of-experts designs only activate part of the model for any given question, so a model can be huge on paper while actually running lean in practice.

Put those together and a well-engineered small model can now beat an older, much larger one on plenty of everyday tasks — coding, summarizing, answering questions — even though it would lose on raw scale alone. That's not a one-off: distillation and quantization are now standard parts of how most current model families get built, not an exotic workaround.

So does size still matter?

So parameter count still tells you something, just not everything. "Biggest" and "best for what you're actually trying to do" stopped being the same question. It's a bit like judging an engine purely by its size — displacement is one signal, but tuning, fuel, and design decide the actual performance.