How do you choose a vector database for RAG?

Choose a vector database for RAG by matching it to three things: your scale, how much infrastructure you want to run, and the search features your retrieval actually needs. Work through a short checklist — data size, managed versus self-hosted, hybrid search, metadata filtering, and cost — and check whether a database you already run, like Postgres with pgvector, is enough before adding a whole new system.

What factors actually matter?

Weigh these in roughly this order:

1. Scale — thousands of chunks behave very differently from hundreds of millions. Small corpora run fine almost anywhere; large ones need a store built for it.
2. Managed vs self-hosted — a hosted service removes ops work but costs more; self-hosting is cheaper but you own uptime, backups, and scaling.
3. Hybrid search — if exact keywords matter (names, product codes, jargon), you want keyword and vector search combined, not vectors alone.
4. Metadata filtering — RAG almost always filters by source, date, tenant, or permissions, so first-class filtering is essential, not optional.
5. Cost — compare total cost at your real data size, including your own time to operate it, not just the sticker price.

Do you even need a dedicated vector database?

Often, no. If you already run Postgres, the pgvector extension adds vector search to the database you already back up and query — one fewer system to operate, and your vectors sit next to your relational data for easy filtering. It's a strong default for small-to-medium RAG. Reach for a dedicated vector database when your corpus grows large, when query latency under load becomes a problem, or when you need features like advanced hybrid search that a general-purpose database handles less well.

How do you make the final call?

Turn the checklist into a decision:

• Already on Postgres and under a few million chunks? Try pgvector first.
• Want zero operations? Pick a fully-managed service.
• Need hybrid search or heavy metadata filtering? Prioritize a store that treats those as first-class.
• Optimizing for cost at scale and willing to self-host? An open-source engine on your own hardware usually wins.

Don't over-engineer the first version. Start with the simplest option that covers your scale and filtering needs, measure retrieval quality and latency on your own data, and migrate only when a real limit forces it.