What is LLM temperature?

Temperature is a setting that controls how random or predictable an LLM's output is. At a low temperature (close to 0), the model almost always picks the most likely next word, so answers are focused, consistent, and repeatable — ideal for factual questions, code, or anything where you want the same reliable result. Turn the temperature up, and the model starts taking more chances, choosing less obvious words.

That makes writing feel more creative and varied — but also more prone to wandering or making things up.

What is actually happening under the hood?

At each step the model doesn't have one answer — it has a ranked list of possible next tokens, each with a probability. Temperature reshapes that list before the model picks one. A low temperature sharpens the distribution toward the single top choice, so the most likely word wins almost every time.

A high temperature flattens it, giving unlikely options a real shot at being selected. At a temperature of 0 the model is effectively deterministic, taking the top choice every time; crank it high and even oddball words can surface. Nothing about the model's knowledge changes — you're only changing how boldly it gambles on each pick.

When should you use low versus high temperature?

It comes down to whether you want reliability or variety. A rough guide:

Neither end is better — it depends entirely on the task. For a contract summary you want low; for a batch of tagline ideas you might want high.

What temperature should you actually set?

Most tools default to a middle value that balances reliability and creativity, so you often don't need to touch it. When you use a model through its API, though, temperature is usually an adjustable setting, and it's one of the easiest knobs for getting better results. If answers feel bland or repetitive, nudge it up; if they feel sloppy or inconsistent, pull it down.

Change it in small steps and test — the sweet spot varies by model and by what you're asking for.