Why tokens are not words
Common words may fit in one token, while an unusual word, code string, URL, or language with different writing rules may split into several. The exact count depends on the tokenizer and model, so token estimates are useful planning numbers rather than a universal character count.
Where tokens show up
- Your instructions and system message.
- The user’s text and attached context.
- Retrieved documents and tool results.
- The model’s generated response.
- The total context window and API bill.
A practical cost lesson
A long prompt sent on every request can cost more than the visible answer. Remove repeated boilerplate, retrieve only the passages needed, and use caching or smaller models when the task allows it. Do not cut context blindly: removing the one fact the model needs can increase errors and review time.