What is LLM training?

LLM training is the process of teaching a large language model to understand and generate language by having it learn patterns from enormous amounts of text. It happens in stages. The first and biggest is pretraining: the model reads a huge slice of the internet, books, and code, and repeatedly tries to predict the next word.

Each time it guesses, it checks against the real text and nudges its billions of internal settings (parameters) to do better. Repeated across trillions of words, this is how it absorbs grammar, facts, reasoning patterns, and style.

What are the main stages of training?

Training usually moves through a few distinct phases, each building on the last:

1. Pretraining — the model reads a massive body of text and learns to predict the next word, absorbing grammar, facts, and reasoning patterns. This is the longest and most expensive stage.
2. Fine-tuning — the raw model is trained on cleaner, task-focused examples so it learns to follow instructions instead of just continuing text.
3. Alignment — human feedback is used to reward helpful, honest, and safe answers, shaping how the model actually behaves in a conversation.

Pretraining alone produces a model that's knowledgeable but raw — good at completing text, not necessarily at being helpful or safe. The later stages are what turn it into something you'd want to talk to.

Why is training so expensive?

Training the largest models is enormously slow and costly. It can take weeks running on thousands of specialized chips and cost millions of dollars in computing power alone, before you count the people and data involved. That price tag is why only well-funded labs train frontier models from scratch, while most companies take an existing model and adapt it instead.

The cost scales with size: more parameters and more training text mean more computation, which is the main reason building a top-tier model has stayed out of reach for all but a handful of organizations.

How is training different from inference?

Training happens once, up front — it's how the model is built. After that, the finished model is frozen: its parameters stop changing. Actually using that frozen model to answer a question is a completely separate step called inference.

The distinction matters because the two have very different costs. Training is a huge one-time investment, while inference is the ongoing cost of running the model every time someone uses it. A model doesn't learn from your conversations while it answers; it only changed during training, which already ended.