What is retrieval-augmented generation (RAG)?
Retrieval-augmented generation, or RAG, combines information retrieval with a large language model. Instead of relying only on its training data, the model receives relevant passages from a document alongside a question. Those passages provide context for a document-grounded answer.
How does document retrieval work?
A RAG pipeline extracts text, splits it into smaller passages called chunks, and converts those chunks into numerical embeddings. A vector search compares the question with the stored embeddings to find related passages. The language model then generates an answer using the retrieved context. Retrieval quality depends on the document, chunk boundaries, embedding model, and search settings.
How is RAG different from fine-tuning?
RAG supplies reference material at query time without changing the model's weights. Fine-tuning changes model weights through additional training, often to adapt behavior, style, or task performance. The two approaches can complement each other. For questions about a changing collection of documents, retrieval makes it possible to update the reference material separately from the model.
Can document chat give incorrect answers?
Yes. Retrieval can miss important context, and a language model can misinterpret a passage or produce unsupported claims. Scanned PDFs also need optical character recognition before their text can be retrieved. Compare important answers with the original document; a fluent response is not proof of accuracy.
Further reading: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, the research paper introducing the RAG approach.