TL;DR

Lewis et al. combine a pretrained sequence-to-sequence model with a neural retriever. For a question, the retriever fetches passages, and the generator conditions on those passages when it writes the answer. The index can be updated when facts change, without retraining the whole language model.

Why it matters

A language model that stores facts only in its weights is hard to inspect and hard to refresh. RAG treats documents as an external memory. That pattern, retrieve then generate, became the template for systems that need a source for an answer rather than a guess from parameters alone.

Key takeaways

• The retriever and generator are trained together so the model learns which passages to use.
• Answers are conditioned on retrieved text, which gives a trail back to a document.
• Swapping or updating the document index changes what the system can cite without a full pretraining run.
• The paper targets knowledge-intensive tasks, where parametric models lagged specialized architectures.