The Improviser · Lesson 7 — The Sample Library ← Course

The sample library — retrieval (RAG).

Don't make the soloist remember every tune. Pull the right recordings — short snippets from your library — onto the stand first, then let it play from what's in front of it.

Ask a model to answer from memory and it can invent things — fluent, confident, and wrong (Lesson 6). The fix is almost too simple: stop asking it to remember, and hand it the real material to read from instead. Done at scale, that move has a name — retrieval-augmented generation, or RAG.

RAG is two steps, run by two different systems. A search step takes your question and pulls the few most relevant records from a library of your material. Those records drop into the model's context window (Lesson 2), and the model answers from them — reading what it was just handed rather than recalling from memory.

Here is the part people get backwards: the model never does the digging. Picture a record-digger and a soloist. The record-digger flips through the shelves, finds the takes that fit — this feel, this sound — and sets them on the stand. The soloist just plays what's in front of it; it never reaches into the archive itself. RAG splits the same way: the search step is the digger, the model is the soloist.

The clever part is that fetch. You can't tip the whole library into the window (Lesson 2's budget), so retrieval finds records by meaning, not matching words — a search for “modal opener” also surfaces a “dorian vamp”, because they mean the same kind of thing.

This is what lets a model answer from material it was never trained on — your own library, today's session, this exact set — and cite a record you can point to. It also means staying current is a matter of updating the library, not retraining the model.

Pick what you're after and watch the digger pull the records that fit — then the soloist reads only those, and answers, citing each one. Tap a record to hear it.

Ask the soloist which record fits. A search step (not the soloist) pulls a few candidates; the soloist then picks from only those.
① the search · pulls the records — the soloist isn't involved yet
↓ only these three reach the soloist ↓
② the soloist · recommends one, choosing only from the records above — not from memory

Search first, then answer.

A RAG call is two steps: fetch the most relevant records, then answer from those — never from memory alone. 


      

    


    

      
Play from the record, don't recall from memory.

      

        RAG turns a question into a search: fetch the few most relevant records, load them
        into the context, and answer from those. The model stops recalling and starts
        reading — so it can speak to your material, stay current, and ground every choice in
        a source you can cue up.
      

      

        The whole trick is the fetch — find the right few records by meaning, fit them on
        the stand, and let the model play from the library rather than from memory.
      

      

        Next: by meaning →
        
      

    


  


  

    

      ▸ Go deeper — how the fetch works optional
      

        
Two jobs, two systems

        

          Worth saying once more, plainly: retrieval and generation are separate. A search
          system does the digging; the model does the playing. The model has no special
          access to your library — it only ever sees the records the search step chose to set
          on the stand. Swap in a better search and the same model gives better answers,
          because you handed it better material.
        

        
Retrieval by meaning (embeddings)

        

          Each record — and the question — is turned into a long list of numbers, an
          embedding, that captures its meaning. The records whose numbers sit closest
          to the question's are the most relevant. That is why “modal opener” can find a
          “dorian vamp”: close in meaning, even with no words in common. The search terms are
          embeddings, vector search, and semantic search — the next page.
        

        
Chunking

        

          You don't store whole sessions as single lumps; you split them into short takes —
          chunks. Retrieval then returns just the relevant passage rather than a whole reel,
          which keeps the answer focused and fits it inside the window (Lesson 2).
        

        
It is only as good as the library

        

          RAG can't retrieve what isn't there, and it can retrieve the wrong thing. A missing
          record, a stale one, or a bad match all lead to a confidently wrong answer —
          grounded in the wrong take. Retrieval quality is most of the battle; the model is
          only playing what you handed it.
        

        
RAG versus training it in

        

          RAG keeps knowledge outside the model and fetches it on demand, which makes
          it easy to update and easy to cite. Baking knowledge into the model's own weights
          is fine-tuning (Lesson 12) — a different tool for a different job. Most systems
          reach for RAG first.