Local is not automatically private
A local model can keep prompts away from a hosted provider, but the application may still leak data through logs, backups, monitoring, plugins, or an exposed server. Privacy depends on the whole system, not just where model inference runs.
Local models make sense when
- Data cannot be sent to an external provider.
- The task is narrow enough for the available model quality.
- You can maintain hardware, updates, access controls, and monitoring.
- Offline or low-connectivity operation matters.
- The volume makes hosted inference costly over time.
Hosted models may be better when
You need frontier reasoning, multimodal capability, managed scaling, or a small team without infrastructure capacity. A contractual no-training and retention-controlled hosted service may be safer in practice than an under-maintained local server.