What is a vector database?
Knowledge, data and integration
A vector database is a system for storing and searching embeddings, the numerical representations of content. Instead of matching exact words, it retrieves items by similarity, finding the vectors closest to a query vector, usually with metadata filters applied. It uses specialised nearest-neighbour indexes to do this quickly across millions of items. Vector databases power semantic search, recommendations and the retrieval step in retrieval-augmented generation. They are not always a separate product: vector search now also appears inside existing search engines, relational databases and cloud data platforms.
Reviewed by Jackie, Head of Learning & Development, Levellers · Last reviewed 8 June 2026
What this means
Once content has been turned into embeddings, you need somewhere to keep those vectors and a fast way to search them. That is what a vector database does. You give it a query, it converts the query to a vector with the same model used for the stored content, and it returns the items whose vectors are nearest. Nearest means closest in meaning.
A search for a "late payment reminder" can surface a document titled "overdue invoice follow-up", because the two are close in meaning even though they share no keywords. Alongside each vector you can store metadata such as the source, date, author and access rules, then filter on those fields so results are both relevant and allowed.
Why it matters
Search by meaning is the practical payoff of embeddings, and a vector database is what makes that search fast enough to use in a live product. Without one, comparing a query against millions of vectors one by one would be too slow.
It also matters because retrieval quality sets the ceiling on any tool built on top. In retrieval-augmented generation, the language model can only work with what retrieval hands it. If the vector database returns weak or stale passages, the final answer suffers, no matter how capable the model.
How it works
The five-step setup
First, collect the content you want searchable. Second, split large documents into chunks. Third, embed each chunk with one model. Fourth, store the vectors with metadata. Fifth, at query time, embed the query with the same model and retrieve the nearest vectors, applying filters and, often, reranking the top results for relevance.
Approximate nearest neighbour
Finding the exact closest vectors in a huge collection is expensive. Vector databases use approximate nearest-neighbour indexes that trade a little accuracy for a large gain in speed. A widely used method, Hierarchical Navigable Small World graphs, builds a layered graph so search can hop quickly toward the closest items with logarithmic-scaling cost. Library work such as FAISS made billion-scale similarity search practical on commodity and GPU hardware. The trade-off is recall: an approximate index may occasionally miss a true nearest neighbour, and you tune how aggressively to favour speed over completeness.
Hybrid retrieval
Vector search and keyword search have different strengths. Keyword search is precise for names, codes and exact phrases; vector search is strong on meaning and paraphrase. Hybrid retrieval combines both, and it often beats either alone.
Not always a separate platform
You do not always need a dedicated product. Vector capability now sits inside many existing search engines, relational databases and cloud data platforms, so the right choice depends on what you already run. Gartner predicts that organisations will develop 80 percent of generative AI business applications on their existing data management platforms by 2028, which points to building on what you have rather than adding a separate store by default.
Examples
Semantic search across a knowledge base, returning passages by meaning rather than keyword.
Recommendations, matching items to a profile or a recently viewed item by closeness in vector space.
The retrieval step in retrieval-augmented generation, where relevant passages are fetched and passed to a language model to ground its answer.
Deduplication and clustering, grouping near-identical or related records together.
Common misunderstandings
"A vector database is the same as RAG." No. The database is one component. RAG is the wider pattern of retrieving content and feeding it to a model.
"It replaces keyword search." Often it complements it. Hybrid retrieval usually performs best.
"Embeddings alone solve search quality." Quality depends on chunking, the model, filtering and reranking, not vectors in isolation.
"You must buy a specialist platform." Many existing systems now offer vector search, so a separate product is a choice, not a requirement.
Risks and boundaries
Retrieval quality is the main risk. Poor chunks or a poorly matched model give weak results that are hard to spot.
Security and permissions must be enforced. A vector database that ignores access rules can surface content a user should not see. Permission filtering belongs in the retrieval path.
The approximate nearest-neighbour trade-off means recall is not guaranteed. Tune it deliberately and measure it.
Freshness needs managing. Vectors must be updated when content changes or is deleted, or results drift out of date.
What to do next
Start with one document set and one task. Build a small set of real queries with known good answers. Then compare vector-only, keyword-only and hybrid retrieval on those queries and judge the results with someone who knows the material. This tells you whether you need a specialist store at all, and which retrieval approach earns its place. Decide upfront how you will keep vectors fresh as content changes, and confirm that access rules are applied during retrieval, not bolted on afterwards.
Have a question or a suggestion, or want to understand how we research and review these guides? Read about our editorial standards and how to reach us.
FAQs
Do I need a separate vector database?
Not always. Vector search is now built into many search engines, relational databases and cloud data platforms. Use what fits your stack and scale.
How is it different from a normal database?
A normal database retrieves by exact values and structured queries. A vector database retrieves by similarity between embeddings, returning the nearest items by meaning.
What should I store alongside the vectors?
Metadata such as source, date, author, document type and access rules, so you can filter results and enforce permissions.
Is it only for chatbots?
No. It powers semantic search, recommendations, clustering and deduplication, as well as the retrieval step in retrieval-augmented generation.
How do I know if retrieval is good enough?
Build a set of real queries with known good answers, then measure how often the right passages appear near the top. Judge with someone who knows the content.
What is approximate nearest neighbour?
A search method that trades a little accuracy for a large gain in speed, so similarity search stays fast across millions of items.
Does it guarantee it finds the closest match?
Approximate indexes may occasionally miss a true nearest neighbour. You tune the balance between speed and completeness.
What is hybrid retrieval?
Combining vector search with keyword search, which often gives better results than either method alone.
Sources
Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs (arXiv (Malkov and Yashunin), also IEEE TPAMI). Foundational approximate nearest-neighbour method used in vector search.
Billion-scale similarity search with GPUs (arXiv (Johnson, Douze, Jegou), also IEEE Transactions on Big Data). Foundational work making large-scale similarity search practical.
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (NeurIPS (Lewis et al.)). Establishes the retrieval-and-generate pattern that vector databases support.
Gartner Predicts 80% of GenAI Business Apps Will Be Developed on Existing Data Management Platforms by 2028 (Gartner). Analyst evidence that vector and RAG capability is increasingly built on existing platforms.