What RAG Is and Why It Exists | Beyond the Prompt

Lesson 8: What RAG Is and Why It Exists

Blackboard

Give the model a library card

10 min

Concept 1

The five problems RAG solves

RAG addresses context window limits, knowledge gaps, hallucinations, freshness (documents can be updated without retraining), and lack of sources (you can trace answers back to specific documents).

Concept 2

How RAG works at a high level

You take your documents, embed them into numerical representations, store those in a vector database, and when a user asks a question, you embed their query, find the most relevant documents by distance, and inject those documents into the prompt alongside the question.

Concept 3

Developer control without retraining

RAG sits on top of the model — you never modify its weights. You control what documents it can see, what prompt template it uses, and what instructions it follows. If a new report comes in, you add it to the database and the system is immediately up to date.

Concept 4

The medical example

A user asks about drug side effects. RAG retrieves the latest safety documents from a medical database, injects them into the prompt with instructions to cite specific pages, and the model produces a sourced, current answer instead of a potentially hallucinated one.

Pause and Think

You are building a legal Q&A tool for a law firm. A lawyer reports that the system confidently cited a case that does not exist. What would you check first, and what would RAG change?

Flashcards

Key Terms

Click or press Enter to flip a card. Review these before the practice question.

Additional Resources

Article — What Is RAG? (AWS)
AWS explainer on what RAG is and how the retrieve-augment-generate loop works. Stays vendor-neutral despite the source. Good starting point if this is new territory.
Google Research — RAG and Sufficient Context
Google Research asking how much retrieved context RAG actually needs. Turns out more is not always better — which is exactly the point this lesson makes.
Paper — RAG Survey
Large RAG survey covering architectures, retrieval strategies, and how to evaluate them. Keep it open — you'll want to come back to it during lessons 8 through 10.
Glossary — Retrieval-augmented generation (RAG)
A short definition with related terms.

Practice Question

What is the main advantage of RAG over fine-tuning for keeping an AI system up to date?

Does not count for certificate

Choose one answer, use the hint if needed, then check your answer.

Use the hint before checking if you need a nudge.

Choose one answer, then click Check Answer.