Best Vector Database for RAG: Filters, Cost and Scale

The best vector database for RAG depends on filtered retrieval quality, the deployment you can operate, and total application cost. A useful comparison includes the queries your application actually permits: tenant restrictions, document permissions, metadata filters, and updates. An unfiltered nearest-neighbor chart does not answer that whole question.

This is a documentation-based comparison of Pinecone, Weaviate, Qdrant, pgvector, Chroma, and Milvus. The selection criteria below are editorial recommendations derived from documented capabilities. No latency, throughput, price, or corpus-size cutoff is presented as a universal product limit.

Best vector database for RAG: comparison table

The capabilities come from each project’s documentation in the source list. Read the comparison as a shortlist, then evaluate finalists with the same corpus and request mix. A product having hybrid search or metadata filtering does not establish that its defaults satisfy your application.

The LLMOps tools on GitHub guide places these projects within the wider stack. Pair database selection with vLLM vs TGI if generation runs on your own infrastructure, or, for a wider engine shortlist, with the LLM serving frameworks comparison.

Define recall against the permitted corpus

For approximate nearest-neighbor evaluation, define the reference neighbors using the same distance function and the same filters as the candidate query. A reference set drawn from the whole database can count unauthorized or out-of-scope documents as relevant neighbors.

One suggested calculation is:

filtered_recall_at_k =
  overlap(candidate_results, exact_filtered_neighbors)
  / number_of_exact_filtered_neighbors

Use up to k exact neighbors, and specify tie handling. If the eligible corpus has fewer than k records, the denominator should reflect the eligible reference set. If it has none, report a no-result case separately instead of dividing by zero.

This is an index-fidelity metric. It measures agreement with exact vector search, not whether retrieved passages answer the user. Add a separate set of relevance judgments and inspect downstream answers. The distinction matters when an embedding model ranks a semantically similar but unhelpful passage highly.

Keep permission enforcement separate from recall targets. A result from an unauthorized tenant is an access-control failure even if it improves a relevance score. Build allow/deny cases into validation and ensure the server derives access scope from authenticated context.

How the candidates handle the selection problem

Pinecone: specify the metadata boundary

Pinecone’s filtering guide documents metadata expressions applied to search results. That supports evaluating category, tenant, or document-state restrictions as part of retrieval, rather than filtering every result after it reaches application code.

Record how your index and namespace layout represent tenants. Then check both successful retrieval and empty-result behavior under the actual restrictions. A managed service still requires application decisions about identity, permissions, indexing, and deletion. Obtain a quote for the planned read/write pattern rather than importing another workload’s monthly bill.

Weaviate: test hybrid relevance and resource use together

Weaviate’s hybrid-search documentation describes combining vector and BM25 results through fusion. This is relevant when queries include identifiers, error strings, or exact phrases alongside natural-language descriptions.

Use examples that isolate those needs. A purely semantic query and an exact identifier lookup should not be collapsed into one average relevance score. Record the fusion configuration with the dataset so later changes can be compared.

The compression documentation explains that compressing vectors changes resource use and can require rescoring to recover recall. It also distinguishes vector storage from the graph’s overhead. Plan for the actual index type and compression settings; a raw-vector byte count is only one part of capacity.

Qdrant: treat payload indexes as part of the design

Qdrant’s filtering guide recommends payload indexes for fields used in filters. Its indexing documentation describes their role in cardinality estimation and search planning, as well as their resource cost.

Choose indexed fields from real query patterns. Include both broad and highly selective filters in evaluation. Create the intended indexes before assessing ingestion and search behavior, and keep their definitions in the release manifest.

Avoid indexing every field by habit. Indexes consume resources, while unindexed predicates can affect query behavior. Compare the full candidate configuration, including filter indexes, against the alternatives.

pgvector: inspect the filtered query plan

The pgvector README documents exact nearest-neighbor search, approximate indexes, and iterative index scans. It specifically explains that filtering after an approximate scan can produce fewer results, and that iterative scans can search further subject to configured limits.

That makes PostgreSQL query plans part of the selection process. Test joins and filters as the application issues them. An isolated vector query does not represent retrieval that also consults relational permissions or competes with transactional work.

Existing PostgreSQL operations can make pgvector attractive, but they do not remove capacity work. Include index builds, backup/restore, and database contention in the comparison. There is no universal vector-count threshold at which all applications must leave PostgreSQL.

Chroma: choose the deployment before judging limits

Chroma’s introduction documents embeddings, metadata, filtering, and retrieval capabilities. Those are relevant to both application development and deployment evaluation.

Use the deployment mode you intend to operate when assessing persistence, concurrency, and recovery. Results from a local development setup do not define every deployment’s production limits. Avoid a blanket migration threshold based only on the number of vectors; dimensionality, payloads, queries, and operational requirements belong in the decision.

Milvus: include operations in the architecture comparison

The Milvus overview describes deployment choices and vector-search infrastructure. The appropriate question is which form matches your workload and operating capacity.

Evaluate ingestion, query serving, storage dependencies, and recovery together. If the candidate uses multiple services, document which ones must be restored to recover a usable index. Scaling options are useful only when the team can operate the chosen configuration and maintain the desired freshness and availability.

Capacity planning starts with bytes, then adds overhead

For an illustrative uncompressed float32 calculation:

raw_vector_bytes = vector_count × dimensions × 4

One million vectors with 1,536 dimensions contain 6,144,000,000 bytes of raw vector values, about 6.14 GB in decimal units. This is arithmetic, not a memory requirement or a provider quote.

Add the structures your deployment actually stores: index graph or partitions, metadata, documents, replicas, write buffers, and backup copies. Account for compression and whether each structure resides in memory or on disk. A rebuild may require temporary capacity alongside the existing index.

Carry this into self hosting LLM vs API cost. Retrieval storage, embedding work, and operations remain part of a private LLM application’s cost even when generation is purchased through an API.

A repeatable RAG comparison protocol

Use a fixed, versioned corpus and embedding revision. Record the distance function, dimensions, chunking method, metadata schema, and permission model.

Prepare query cohorts for broad filters, highly selective filters, exact identifiers, natural-language questions, and empty-result cases. Measure index fidelity and judged relevance separately for each cohort. Report sample sizes so a small difficult cohort cannot disappear inside a large easy one.

Then examine latency under the intended arrival pattern, with ingestion and updates active. Include cold and warm states, rather than describing a warm-cache run as the whole service. Define the timing boundary: database-only latency and end-to-end retrieval latency answer different questions.

Exercise document updates and removals using benign fixtures. Verify that changed permissions and deleted content stop appearing within the freshness window the application requires. Record observed behavior as part of your own acceptance process, not as a guarantee implied by a vendor feature list.

Finally, compare total costs for the same workload. Include hosting, storage, embedding refreshes, backups, observability, and operating time. Avoid a ranking built from one candidate’s paid managed deployment and another’s development laptop.

Retrieval can move generation cost

Returning additional chunks can improve context coverage, but also changes input length at the generation step. Reranking introduces another processing step. Evaluate the full application outcome rather than minimizing vector-query latency in isolation.

Track retrieved context size and generation input tokens together. Token cost observability explains how to attribute those costs, and RAG observability in production covers the retrieval-quality signals.

Repeated queries may justify a cache, provided its permissions and freshness policy are explicit. The semantic caching guide discusses threshold and invalidation decisions. A cache hit should not bypass a permission change.

Frequently asked questions

Is pgvector enough for production RAG?

It is a candidate when its documented search behavior and your PostgreSQL operations fit the workload. Establish that with filtered queries, relevance checks, concurrent load, and recovery requirements. Corpus size alone cannot decide it.

Which vector database has the best recall?

Recall depends on the reference set, index settings, embedding model, and filters. Compare candidates at the same target recall and workload before comparing speed or cost. Exact-vector recall also needs a separate application-relevance check.

Should RAG use hybrid search?

Consider it when exact terms carry meaning that semantic similarity alone misses. Validate it with representative relevance judgments. Weaviate documents one implementation, but a documented feature does not establish that its default ranking fits every corpus.