Good reasons to add routing
- Simple tasks represent a large share of traffic.
- A smaller model meets the approval bar for those tasks.
- Expensive model calls create a measurable budget problem.
- Different privacy or latency requirements need different providers.
- A fallback improves availability during outages or limits.
Reasons to wait
If you do not have a task taxonomy, evaluation set, or route-level metrics, a router will hide the decision rather than improve it. First measure your workload and establish a single-model baseline.
A practical threshold
Add routing after you can show that a repeatable class of requests can use a cheaper or faster model without increasing correction time, failures, or support burden. The proof is in cost per approved task, not the number of models in the diagram.