Add it before the first serious rollout
A prototype can be debugged by watching the screen. A live system needs to explain why one request was slow, expensive, wrong, or routed to a particular tool. Adding traces after an incident means the evidence may already be gone.
Minimum signals
- Trace ID and user/workspace context.
- Model and prompt version.
- Retrieved sources and tool calls.
- Latency, retries, errors, and token cost.
- Outcome, correction, escalation, or approval result.
When basic logs are enough
For a small, low-risk pilot, structured server logs and a request ID may be enough. Upgrade to richer tracing when multiple steps make failures hard to localize, or when users depend on the output and you need evidence for support and improvement.