The Marsen glossary

Vector Database

A system that stores numerical representations of information so semantically related content can be retrieved.

By Aranyak Sengupta3 min read

The short answer

A vector database stores numerical representations of content, called embeddings, and finds items whose meaning is similar to a query even when the words differ. It is the retrieval layer behind most semantic search and retrieval-augmented generation systems, but similarity is not the same as relevance or truth.

How a vector database works

An embedding model converts each passage of text into a list of numbers that places similar meanings close together. The vector database stores these vectors with the original text and metadata such as source, date and access permissions. A query is converted into a vector the same way, and the database returns the nearest stored vectors using an approximate nearest-neighbour index, which keeps search fast across millions of items. Many systems combine this with keyword search and metadata filters.

A worked example

A software company has 800 help articles. A customer searches for 'I can't get into my account'. Keyword search finds little because the articles say 'login' and 'password reset'. Vector search returns the password-reset and two-factor-authentication articles because their meaning is close to the query. A metadata filter restricts results to the customer's product plan, and the top three passages are passed to a language model that drafts the answer.

  • Store: article passages, embeddings, product and plan metadata
  • Query: 'I can't get into my account'
  • Return: password reset and two-factor articles, filtered by plan

What to check before using one

Evaluate retrieval with real questions and check whether the right passages come back, before judging the generated answers. Enforce document permissions in the query, not after the answer is written. Re-embed content when you change embedding models, and remove outdated passages so they are not retrieved. For small collections, a database you already run with a vector extension, such as PostgreSQL with pgvector, may be enough.

Retrieval-augmented generation explained
Business automation guide

Limits

Nearest neighbours are similar, not necessarily correct: an outdated policy can be very similar to the current one. Exact lookups such as order numbers, prices or dates are better served by a normal database query or keyword search. Retrieval quality also depends on how documents are split: passages that are too long dilute the match, and passages that are too short lose the context needed to answer.

Take the next step

Explore the related guide, or look up another term.

Continue reading
Browse the glossary

Built around your business

Put what you’ve learned to work.

Book a demo

Bring us your questions. We’ll help you find a useful next step for your business.