RAG (Retrieval-Augmented Generation) is a pattern where an AI system retrieves relevant information from an external knowledge base before generating a response. Instead of relying solely on what the model learned during training, the agent grounds its answer in your documents, database, or vector store.
How RAG works
- Index — Documents are chunked and embedded into a vector store
- Retrieve — When a query arrives, similar chunks are fetched by semantic search
- Augment — Retrieved content is injected into the model's prompt as context
- Generate — The model produces a response grounded in the retrieved material
RAG vs. fine-tuning
| RAG | Fine-tuning | |
|---|---|---|
| Updates | Add documents anytime | Retrain the model |
| Cost | Lower, pay per query | Higher upfront |
| Best for | Dynamic knowledge bases | Style, format, domain behavior |
| Citations | Can trace to source docs | Opaque |
Most production agents use RAG (or similar retrieval) for knowledge that changes frequently — product docs, policies, customer records.
Common RAG pitfalls
- Chunking too large or too small — hurts retrieval precision
- No reranking — top-k vector matches aren't always the best context
- Stale indexes — documents update but the vector store doesn't
- Over-retrieval — stuffing too much context into the prompt wastes tokens and confuses the model
Related reading
See Agent Memory for how RAG fits into the broader memory architecture.