The Third Umpire · Lesson 7 — The Footage Bank ← Course

The footage bank — retrieval (RAG).

Don't ask the umpire to remember everything. Fetch the relevant footage, then let it rule from what's on the screen.

Ask a model to answer from memory and it can invent things — confident, fluent, and wrong. The fix is to stop trusting its memory and hand it the evidence to read from instead. Doing that at scale has a name — retrieval-augmented generation, or RAG.

RAG is two steps, and — this is the key — they are done by two different things. First a search step takes your question and pulls the few most relevant pieces out of an archive of your documents. Then those pieces are dropped into the model's context window (Lesson 2), and the model answers from them. The model is no longer recalling from memory; it is reading what it was just handed.

Here is the part people expect to work the other way: the model does not do the searching. A separate search tool finds the documents and places them in front of the model. The model never reaches into the archive itself — it only ever reads the text already sitting in its window.

Picture the third umpire. A producer in the truck finds the right clips — this batter, this over — and puts them up on the screen; the umpire does not go digging through the tape library, it just watches what is shown and rules on it. RAG splits exactly the same way: the search step is the producer, the model is the umpire.

The clever part is the fetch. You cannot paste the whole archive into the window (Lesson 2's budget). So retrieval finds the few most relevant records by meaning, not just matching words — a search for “edged behind” also surfaces a “leg-side strangle”, because they mean the same kind of thing.

This is what lets a model answer from information it was never trained on — your private documents, today's data, this exact match — and ground every claim in something you can point to and check.

Pick a question and watch the search pull the clips that fit — then the umpire reads only those, and rules, citing each one.

Pick a question. A search step (not the umpire) finds the clips; the umpire then reads only those.
① the search · finds the clips — the umpire is not involved yet
↓ only these three reach the umpire ↓
② the umpire · reads only these, then rules

Search first, then answer.

A RAG call is two steps: fetch the most relevant records, then answer from those — never from memory alone. 


      

    


    

      
Read from the record, don't recall from memory.

      

        RAG turns a question into a search: fetch the few most relevant records, load them
        into the context, and answer from those. The model stops recalling and starts
        reading — so it can speak to your data, stay current, and ground every claim in a
        source you can check.
      

      

        The whole trick is the fetch — find the right few records by meaning, fit them in
        the window, and let the model rule from the footage rather than from memory.
      

      

        Next: by meaning →
        
      

    


  


  

    

      ▸ Go deeper — how the fetch works optional
      

        
Two jobs, two systems

        

          Worth saying once more, plainly: retrieval and generation are separate. A search
          system does the fetching; the model does the reading. The model has no special
          access to your files — it only ever sees the passages the search step chose to
          paste into its window. Swap in a better search and the same model gives better
          answers, because you handed it better footage.
        

        
Retrieval by meaning (embeddings)

        

          Each record — and the question — is turned into a long list of numbers, an
          embedding, that captures its meaning. The records whose numbers sit closest
          to the question's are the most relevant. That is why “edged behind” can find a
          “leg-side strangle”: close in meaning, even with no words in common. The search
          terms are embeddings, vector search, and semantic search.
        

        
Chunking

        

          You do not store whole documents as single lumps; you split them into small
          passages — chunks. Retrieval then returns just the relevant paragraph rather than
          a whole book, which keeps the answer focused and fits it inside the window
          (Lesson 2).
        

        
It is only as good as the archive

        

          RAG cannot retrieve what is not there, and it can retrieve the wrong thing. A
          missing record, a stale one, or a bad match all lead to a confidently wrong
          answer — grounded in the wrong footage. Retrieval quality is most of the battle;
          the model is only reading what you handed it.
        

        
RAG versus training it in

        

          RAG keeps knowledge outside the model and fetches it on demand, which makes
          it easy to update and easy to cite. Baking knowledge into the model's own weights
          is fine-tuning (Lesson 11) — a different tool for a different job. Most systems
          reach for RAG first.