← StoryboardsStoryboard · 6 frames · Stage 02, Context & retrieval

Retrieval-Augmented Generation in Plain Terms

Finding the right document and handing it to the model are two different steps. Six frames on where the evidence gets lost.

01 / 06

  1. PRUNING MY POTHOSSYSTEMS · STAGE 02A STORYBOARD IN 6 FRAMESRetrieval inplain termsINPUTfound it. dropped it.it only seeswhat it's handed.
    Retrieval fetches material when a question comes in and puts it into the model's input. Finding it and handing it over are separate steps, and either can drop the evidence.
    PRUNINGMYPOTHOS.COM01 / 06

    A door marked INPUT with a letterbox. One page is going through the slot; another lies on the doormat, found and dropped. The model only sees what it is handed. Finding it and handing it over are separate steps, and either can drop the evidence.

  2. PRUNING MY POTHOSTHE STAGESSix stages.Any one can lose it.1Source materialwhatever holds the answer2Preparation and indexingif it never got in, nothing finds it3Retrievalcandidates for this question4Ranking and filteringmost of them get dropped5Context assemblywhat the model actually receives6Generationan answer from what it was handed
    Small systems fold several of these into one search call. When the answer is wrong, usually exactly one of them lost the evidence.
    PRUNINGMYPOTHOS.COM02 / 06

    Six stages: source material, whatever holds the answer. Preparation and indexing; if it never got in, nothing finds it. Retrieval, candidates for this question. Ranking and filtering, where most get dropped. Context assembly, what the model actually receives. Generation, an answer from what it was handed. When the answer is wrong, usually exactly one stage lost the evidence.

  3. PRUNING MY POTHOSRETRIEVAL IS NOT ASSEMBLYFound it.Dropped it.#1 shipping FAQ#2 returns overview#3 refund timeline#4 refund policy 2026#5 gift cardskept: top 3← the right onetop 3this is allI got.retrieval succeeded.the model never saw it.
    Nothing in the search logs looks wrong, because nothing about the search was wrong. Log what was retrieved and what was assembled: they are different lists.
    PRUNINGMYPOTHOS.COM03 / 06

    A search returns five results, and the right one, the current refund policy, is fourth. Context assembly keeps the top three. The Model says: this is all I got. Retrieval succeeded; the model never saw it. Nothing in the search logs looks wrong. Log what was retrieved and what was assembled, because they are different lists.

  4. PRUNING MY POTHOSWHAT SEARCH MEANS HEREWords,or meaning?part no.4417-Bin stockMatching wordsfinds exact IDs,misses “puppy”for “small dog”small dogpuppy4417-B?Matching meaningfinds paraphrases,misses exact IDs
    Lexical search misses a paraphrase; vector search misses an exact identifier. Hybrid systems run both, because the two fail in different places.
    PRUNINGMYPOTHOS.COM04 / 06

    Matching words finds an exact identifier like part 4417-B, but misses a passage that says puppy when you asked about a small dog. Matching meaning finds the paraphrase but can miss the exact identifier. Hybrid systems run both, because they fail in different places.

  5. PRUNING MY POTHOSSIMILAR IS NOT CORRECTCloseisn't correct.EMBEDDING SPACEPOLICYreplacedlast monthPOLICYthe currentoneneighbourssame subject. different answer.neareststill true?
    Two passages can sit close together in an embedding space and still answer different questions. That is retrieval working as designed.
    PRUNINGMYPOTHOS.COM05 / 06

    In an embedding space, a refund policy that was replaced last month sits right next to the current one. They are neighbours: same subject, different answer. Nearest is crossed out in favour of still true. Two passages can be close and still answer different questions.

  6. PRUNING MY POTHOSTHE HONEST BOUNDARYWrong, now witha citation.accordingto the doc…SOURCEold policy(replaced)CITEDretrieval moved the problem. it didn't fix it.
    Retrieval can give the model evidence. It doesn't guarantee the evidence is relevant, current or correct, or that the model uses it well. So first ask: did the evidence reach it at all?
    Read the whole explanationpruningmypothos.com/systems/retrieval-augmented-generation-in-plain-terms/PRUNINGMYPOTHOS.COM06 / 06

    The Model answers according to the document, holding a source stamped cited, but the source is an old, replaced policy. Retrieval moved the problem rather than fixing it. It can provide evidence, but not guarantee it is relevant, current or correct, or used well. First ask whether the evidence reached the model at all.