The Marsen glossary
Retrieval-Augmented Generation
A pattern that retrieves relevant information from approved sources before an AI model produces an answer.
The short answer
Retrieval-augmented generation (RAG) is a pattern where a system first retrieves relevant passages from approved sources, such as your policies, product documents or help articles, and then gives them to a language model to answer from. It keeps answers current and grounded in your own information, but retrieval and generation can each still go wrong.
How RAG works
Documents are split into passages and indexed, usually as vector embeddings in a vector database and often alongside keyword search. When a question arrives, the system finds the most relevant passages, adds them to the model's prompt with an instruction to answer only from them, and returns the answer, ideally with references to the passages used. Updating a document updates the answers without retraining the model.
A worked example
A clinic's front desk uses an assistant to answer staff questions about procedures and policies. A receptionist asks how much notice is required to cancel an appointment. The system retrieves the cancellation section of the current clinic policy and the relevant FAQ, and the model answers '24 hours, otherwise a cancellation fee applies', citing the policy section. When the clinic changes the notice period, staff update the policy document and the next answer reflects it. If no relevant passage is found, the assistant says so instead of guessing.
- Index: clinic policies, FAQs, procedure notes
- Retrieve: the passages most relevant to the question
- Answer: only from those passages, with the source shown
- Fallback: say 'not found' rather than invent an answer
What to check before using it
Check that retrieval respects permissions, so a user cannot receive passages from documents they should not see. Keep sources current and remove outdated versions. Test with real questions and check whether the cited passage actually supports the answer. Measure answer quality on a fixed question set before and after changes, because a better-sounding answer is not always a more accurate one.
What a vector database does
How large language models work
Business automation guide
Where RAG falls short
RAG answers from what it retrieves. If the right passage is missing, poorly written or ranked too low, the answer will be incomplete or wrong. It is also not a substitute for structured data: questions like 'how many open orders does this customer have' are better answered by querying the system of record directly.
Take the next step
Explore the related guide, or look up another term.
Built around your business
Put what you’ve learned to work.
Book a demoBring us your questions. We’ll help you find a useful next step for your business.