The Improviser · Lesson 8 — The Inner Ear ← Course

The inner ear — attention.

A soloist hears every note played so far, but never weighs them all the same. That selective listening — turning some notes up, most notes down — is attention.

Lesson 2 handed the model a window of everything so far. A good soloist has that whole window too, yet leans on only a slice of it at a time. To land a phrase home, they listen for where home is; to keep a run going, they lock onto the last few notes; to answer a phrase, they think back to how it opened. Same notes on the page — what moves is where the ear leans.

That leaning is attention. For each next note it chooses, the model looks back over everything so far and decides how much to lean on each one. A few notes get turned up loud in its inner ear; most get turned down low — so the next note is shaped mostly by the handful it's really listening to.

This is the idea modern AI is built on — the “T” in GPT stands for Transformer, which is this mechanism at scale. It's what lets a model follow your instructions instead of drifting, keep straight who “they” pointed to three sentences back, and stay coherent across a long answer. Over the next three pages it turns practical: it explains why models lose the thread in long prompts and why those prompts cost more, each with a simple habit that follows.

It's also the machinery under Lesson 1's “weighs everything so far” — not a vague blur, but a specific set of weights, recomputed for every single note, that says which parts of the past matter right now. That's how the model holds a thread across a long passage: to resolve a line it began sixteen bars ago, it can reach back and turn that opening note up loud, however many notes sit in between.

Tell the soloist what to do with the phrase, then hear its inner ear: the notes it's leaning on ring out loud, the rest stay quiet — and that blend shapes the next note.

The phrase so far is C · D · E · G. Pick an aim and watch which notes it listens back to.

You're pressing these to try it out — but a real model isn't told which aim to take. The context does it: your prompt and the notes so far imply the aim, and attention leans accordingly.

how loud each earlier note rings in the inner ear
↓ that blend points to ↓
the next note: ?

A weight on every earlier note.

For each next note, attention scores how relevant every earlier note is, turns those scores into weights that add to 1, and blends the notes by those weights. 


      

    


    

      
It listens on purpose.

      

        Attention is the model deciding, note by note, which parts of everything-so-far to
        lean on. A handful of earlier notes get turned up loud; the rest fade back. Change
        what you're trying to do and the weights shift — the same phrase gets heard a
        different way.
      

      

        This is the engine behind “weighs everything so far”: not an even blur, but a precise,
        re-computed set of weights that lets the model stay on a thread across a long line —
        the reason it can resolve a phrase it began bars ago.
      

      

        Next: The Whole Section →
        
      

    


  


  

    

      ▸ Go deeper — how attention actually works optional
      

        
Query, key, value

        

          Each note offers a key (what it's about) and a value (what it contributes).
          The spot choosing the next note sends out a query (what it's looking for). The
          query is compared against every key to get a relevance score; the closer the match, the
          higher the score — the same “nearness in meaning” idea from Lesson 7, used here between
          positions in one line.
        

        
Softmax turns scores into weights

        

          Those raw scores are run through softmax — the very same function from Lessons 1 and 6 —
          so they come out positive and add up to 1. That's the set of “how loud” weights you see
          here. The next note is then the values blended by those weights: mostly the notes it
          scored high, barely the ones it scored low.
        

        
Many ears at once (multi-head)

        

          Real models don't listen just one way. They run many attention “heads” in parallel — one
          might track the harmony, another the rhythm, another how the phrase opened — and combine
          them. It's less a single inner ear than a whole section listening for different things at
          the same time.
        

        
Why this was the breakthrough

        

          Attention is the “T” in GPT — the Transformer. Letting every position look directly at
          every other, and learn what to weigh, is what let models hold long-range threads instead
          of forgetting the start of a line by the time they reached the end. It's the core idea
          modern LLMs are built on.