Home>AI Courses>Beyond the Prompt>Lesson 9

Beyond the Prompt

Module 4 — RAG

Lesson 9: How RAG Actually Works

Last updated: 2026-10-09

Understanding the mechanics — embeddings, vector databases, retrieval, and prompt templates — turns RAG from a buzzword into something you can reason about.

Lesson 9: How RAG Actually Works

Blackboard

The pipeline from question to answer

10 min
Concept 1

Embedding documents

Each document (or chunk of a document) is converted into a list of numbers — an embedding — using a specialised model. If the representation is too small, you lose information. If it is too large, retrieval slows down. Finding the right size is a key engineering decision.

Concept 2

Storing and searching

Embeddings go into a vector database designed for fast similarity search. When a user asks a question, their query is also embedded. The system finds the documents whose embeddings are closest to the query embedding using a distance metric like cosine similarity.

Concept 3

Augmenting the prompt

The retrieved documents are injected into a prompt template: 'Using the following documents, answer the user's question. If the answer is not in the documents, say I don't know.' This grounds the model and reduces hallucination.

Concept 4

Sourcing precision

You can extend the prompt template to ask for exact citations: 'Quote the specific passage you used and provide the page or chapter reference.' This makes the system traceable and builds user trust.

Pause and Think

If the embedding of your question is far from the embedding of the right document, what goes wrong?

Flashcards

Key Terms

Click or press Enter to flip a card. Review these before the practice question.

Additional Resources
Practice Question

A RAG system retrieves irrelevant documents for a query. Which part of the pipeline is most likely failing?

Does not count for certificate
Choose one answer, use the hint if needed, then check your answer.
Use the hint before checking if you need a nudge.