What is Retrieval-Augmented Generation (RAG)?

Core Concept: Retrieval Augmented Generation (RAG) framework
Problem: Large language models (LLMs) have 2 challenges: no source for answer + out-of-date information
Solution: Add retrieval step before generation. LLM instruction: first retrieve relevant content from content store, combine with user query, then generate response
Content Store: Can be open (internet) or closed (collection of documents, policies)
Benefit 1: Out-of-date problem solved - update data store with new info instead of retraining model
Benefit 2: Source problem solved - LLM instructed to pay attention to primary source data before generating, reduces hallucination and data leak
Additional Benefit: Model can say "I don't know" when user question not reliably answerable from data store
RAG Prompt Structure: 3 parts: instruction to pay attention to retrieved content + user question + response
Risk: Poor retriever quality means good user query may not get answer
Research Direction: IBM working both sides: improve retriever for best quality grounding data + improve generative part for richest response
Specific Technologies: Large language models (LLMs), retrieval augmented generation (RAG)
Organization: IBM Research
Person: Marina Danoletski, senior research scientist

More in this category