Pruning My Pothos
← StoryboardsStoryboard · 8 frames · Stage 02, Context & retrieval

Retrieval-Augmented Generation in Plain Terms

Finding the right document and handing it to the model are two different steps. Eight frames on where the evidence gets lost.

Download PDFRead the article →

01 / 08

  1. A man feeds pages into a letterbox with teeth, on a door marked the model. Found it, fed it, it ate it. The model only sees what it is handed. Finding it and handing it over are separate steps, and either can drop the evidence.

  2. Six stages: source material, whatever holds the answer. Preparation and indexing; if it never got in, nothing finds it. Retrieval, candidates for this question. Ranking and filtering, where most get dropped. Context assembly, what the model actually receives. Generation, an answer from what it was handed. When the answer is wrong, usually exactly one stage lost the evidence.

  3. A search returns five results, and the right one, the current refund policy, is fourth. Context assembly keeps the top three. The Model says: this is all I got. Retrieval succeeded; the model never saw it. Nothing in the search logs looks wrong. Log what was retrieved and what was assembled, because they are different lists.

  4. Asked about a Turkish visa. Got a gravy recipe. Two passages can be close in an embedding space and still answer different questions.

  5. Matching words finds an exact identifier like part 4417-B, but misses a passage that says puppy when you asked about a small dog. Matching meaning finds the paraphrase but can miss the exact identifier. Hybrid systems run both, because they fail in different places.

  6. Wrong, now with a citation. Retrieval does not guarantee the evidence is relevant, current or correct, or that the model uses it well. Given a wrong document, it is wrong with a source attached, which is harder to catch.

  7. In an embedding space, a refund policy that was replaced last month sits right next to the current one. They are neighbours: same subject, different answer. Nearest is crossed out in favour of still true. Two passages can be close and still answer different questions.

  8. The Model answers according to the document, holding a source stamped cited, but the source is an old, replaced policy. Retrieval moved the problem rather than fixing it. It can provide evidence, but not guarantee it is relevant, current or correct, or used well. First ask whether the evidence reached the model at all.