Start with a cost trace
Record input tokens, output tokens, model, latency, retry count, and human correction time for a representative week. Without that baseline, token reduction can become a false economy: the API bill falls while review work rises.
The repair sequence
- Remove repeated instructions that are not needed on every call.
- Trim retrieved passages by relevance and metadata filters.
- Cache stable system prompts or repeated context where the provider supports it.
- Route classification and simple transforms to smaller models.
- Set output limits that match the actual deliverable.
- Stop retries that repeat the same failed request.
Do not shorten blindly
Deleting context without checking the answer can increase hallucinations. The goal is the lowest cost per approved result, not the lowest token count in isolation.