Large language models know a lot, but they do not know your documents. Retrieval-Augmented Generation (RAG) fixes that by fetching the right information at question time and handing it to the model.
How RAG works
- Chunk your documents into small passages.
- Embed each chunk into a vector and store it in a vector database such as pgvector or Pinecone.
- Retrieve the chunks closest to the user's question.
- Generate an answer from the question plus those chunks.
When to use it
Use RAG when answers must come from private or frequently changing content: product docs, contracts, invoices, support tickets. It avoids retraining a model every time the data changes.
Common mistakes
- Chunks that are too large or split mid-sentence.
- No evaluation set, so quality is never measured.
- Not showing sources, which makes wrong answers hard to catch.
Takeaway
Good RAG is mostly good retrieval. Spend your time on chunking, search quality and evaluation before changing the model.