What is a context window in an LLM?

A context window is the maximum amount of text an LLM can consider at one time, measured in tokens. Think of it as the model's short-term memory: everything it can see right now — your prompt, the earlier conversation, any documents you pasted, and the answer it's writing — has to fit inside that window. Anything beyond it is simply invisible to the model.

What is a token, and how big can a context window get?

A token is a chunk of text — usually a word or a piece of one, roughly three-quarters of a word in English on average. Windows are measured in tokens because that's the unit a model actually processes. They vary widely between models, from a few thousand tokens to well over a million in the largest ones.

A bigger window lets you feed in longer documents, more conversation history, or entire codebases without the model losing the thread. It's why you can drop a long PDF into some chat tools and ask questions about it directly.

What happens when you exceed the window?

When a conversation or document grows past the window, something has to give. Usually the oldest content gets dropped or summarized to make room — which is why a long chat can seem to forget what you said at the start. The model isn't choosing to ignore that text; it has simply fallen out of view, and nothing outside the current window can influence the answer.

Does a bigger context window always mean better answers?

Not necessarily. More room helps, but it isn't magic, and a few things work against you as inputs get longer:

• Lost in the middle — models can overlook details buried in the middle of a very long input, even when the window technically fits them.
• Cost and speed — every token you add costs more and makes the response slower.
• Noise — padding the window with marginally relevant text can crowd out what actually matters.

Managing what goes into the context window — feeding the model what it needs and leaving out what it doesn't — is one of the core skills of working well with LLMs.