RAG (Retrieval Augmented Generation)
AI first searches for relevant documents before answering.
Explanation
RAG combines information search with text generation. Instead of just answering from the training, the system first searches a knowledge base and uses the documents found as a basis.
How it works
1) User asks a question → 2) System searches relevant documents in a vector database → 3) Texts found are passed to the LLM as context → 4) LLM generates an answer based on the documents.
Example
An internal AI assistant that answers company policy questions by finding and citing relevant policy documents.
Why it matters
RAG is the standard approach to equip LLMs with company-specific knowledge - without expensive follow-up training.
What is RAG? Retrieval-augmented generation explained
The term comes from a 2020 research paper by Facebook AI Research (now Meta). The idea: before answering, a language model is given relevant information retrieved from an external knowledge source.
A RAG pipeline has two phases. During indexing, documents from sources such as SharePoint, a wiki or a ticket system are loaded, split via chunking, converted into embeddings and stored. At query time, the user question is embedded as well, the most similar passages are retrieved, often reordered with reranking, and then passed to the LLM together with the question.
RAG vs. fine-tuning: what is the difference?
Both approaches bring company knowledge into an AI system, but in different ways. RAG leaves the model unchanged and supplies knowledge at runtime. New documents are available immediately, answers can cite sources, and access rights can be enforced per user. Fine-tuning retrains a model on your own examples and changes its behavior, such as tone, format or handling of domain language.
For knowledge questions, RAG is almost always the better choice: cheaper, more current and verifiable. Fine-tuning is worth considering when a model needs to master a very specific style or a narrowly defined task. The two can be combined.
Why RAG projects fail, and what helps
Most weak RAG systems do not have a model problem but a retrieval problem: the right passage is never found, so the LLM cannot use it. Common causes are poorly extracted PDFs and tables, unsuitable chunk sizes, missing keyword search for part numbers and outdated documents in the index. Hybrid search, reranking, well-maintained metadata and above all a test set of real questions with known correct answers make every change measurable.
Frequently asked questions
What does RAG stand for?
RAG stands for retrieval-augmented generation. An AI model first retrieves relevant information from a knowledge base and then generates its answer based on it.
Does RAG prevent hallucinations?
RAG significantly reduces hallucinations but does not eliminate them. Source citations, explicit permission to say “I don't know” and regular evals are still needed.
Do I need a vector database for RAG?
Not necessarily a dedicated one. For smaller knowledge bases, PostgreSQL with pgvector or the vector search of an existing search engine is often enough.