What is tree-of-thought prompting?

Tree-of-thought prompting is a technique where a model explores several different lines of reasoning at once, evaluates how promising each one looks, and can back up and try another path when a branch turns out to be a dead end. Instead of committing to a single chain of steps, it branches out like a decision tree — which makes it much stronger on problems that need search, planning, or trial and error.

How is it different from chain-of-thought?

Chain-of-thought walks a single straight line: one thought after another, all the way to an answer. If an early step goes wrong, the whole chain inherits the mistake and there's no way to recover — the model just keeps going.

Tree-of-thought treats reasoning as a tree instead of a line. At each stage it generates multiple candidate next steps, scores them, keeps the good ones, and abandons the weak ones. Because it can backtrack, a bad guess early on doesn't sink the whole attempt.

How does the tree actually get explored?

The rough recipe repeats a few moves until an answer emerges:

1. Branch — generate several possible next thoughts from the current point.
2. Evaluate — have the model judge which branches look most likely to lead somewhere.
3. Select — keep the strongest branches and expand those further.
4. Backtrack — if a path stalls or fails, return to an earlier node and try a different branch.

This is the same idea as classic search algorithms, with the language model acting as both the thing that proposes moves and the thing that scores them.

When is it worth the extra cost?

Exploring many branches and scoring each one means many more model calls than a single chain — so it's slower and more expensive. That trade-off only pays off on the right kind of problem.

• Good fit: puzzles, planning, math, or games where there are many possible paths and a clear way to tell a promising one from a bad one.
• Poor fit: simple lookups, quick answers, or tasks a single chain already handles well — the branching just burns compute.

Use tree-of-thought when a problem genuinely needs search and you can judge partial progress. For everything else, a plain chain of thought is cheaper and usually enough.