The Improviser · Lesson 8 — The Whole Section ← Course

The whole section — multi-head attention.

One inner ear can only hear a phrase one way. Real attention runs many ears side by side — each listening to the same phrase for something different, then pooling what they catch. That's multi-head attention.

A single set of attention weights can only follow one kind of relationship at a time. So a model runs many in parallel — each one a head. Picture a horn section reading the same line together: one player follows the harmony, one locks onto the most recent note, one holds how the phrase opened. Same notes, different things to listen for — then their reads merge into one.

Why it matters: heads running at once let a model hold several threads in a single step — a pronoun and who it points to, the tense, the topic — so its writing coheres instead of only making sense word to word. A real transformer layer runs dozens of heads like this, with dozens of layers stacked above. Nothing here changes how you prompt; it's simply the reason coherence is possible at all.

Same phrase — C · D · E · G. Three heads listen, each for a different thing. Toggle them on and off to watch the combined read shift, then hear how each one is tuned.

Three heads are listening. The bottom card blends whichever are switched on.
tap a head to switch it off · each card has its own “hear”

Many heads, then joined.

Each head runs the same attention step with its own learned way of scoring relevance, producing its own blend. The heads are then joined back into one vector the model carries forward. 

# one head = one full round of "which notes to lean on"
harmony = attention(phrase, tuned_for="key & chord")
recent  = attention(phrase, tuned_for="the last move")
shape   = attention(phrase, tuned_for="how it opened")

# run them in parallel, then join the results into one read
heard = combine([harmony, recent, shape])   # "multi-head"

# a real model does this with dozens of heads per layer,
# and dozens of layers — many ears, stacked.

Not one ear — a section.

Multi-head attention is the model listening to the same phrase several ways at once. Each head is tuned, through training, to a different kind of relationship — and their separate reads are pooled into one. Switch a head off and the combined sense shifts, because you've taken away one way of hearing.

The last page showed one inner ear. This is the truth behind it: a whole section of them, each catching something the others miss, stacked across every layer of the model.

Next: The Long Thread →
▸ Go deeper — how heads divide the work optional

Nobody assigns the jobs

The labels here — harmony, recent move, opening shape — are made up to make it concrete. In a real model no one tells a head what to specialise in. Each head gets its own query, key and value projections, and training alone settles what each one ends up tracking. Researchers later probe trained models and find heads that seem to follow grammar, position, or specific relationships — but that structure emerged, it wasn't designed.

Why it matters in practice

Multi-head is a big reason models handle language so flexibly: one head can keep a pronoun tied to who it refers to while another tracks the tense and another the topic — all in the same step. Take away multi-head and the model is forced to see every phrase through a single lens.