The arrangement — a band that's never all playing.
You've heard this your whole life. A singer out front, and behind them two or three parts at a time — never the whole band at once. That is the everyday version of the last page.
Listen to any record with a band on it. The singer carries the tune from the first bar to the last. Behind the singer, the band is rarely all in: a piano and a bass under the verse, the horns arriving for the chorus, strings for one bridge and then gone. At any moment most of the chairs are quiet — and you never notice, because what you hear is one band with one sound. The arrangement is the plan for who plays when, and it is how a large band stays light on its feet.
Map that onto the model and the pieces land neatly. The singer is the part that is always on: the inner ear (Lesson 8), the listening-back that every note passes through. It is shared by the whole band and never split. The sections behind the singer are the experts — the block after the inner ear, split into groups, two of which play each note. The listener hears one voice; the bill is paid by a duo.
Now the two places where the model is not a band, and both matter. First, an arranger writes the handoffs in advance, in ink, on the chart — the written arrangement every player reads from — and for musical reasons: horns for the chorus because horns lift a chorus. The router writes nothing in advance. It decides live, note by note, from what it is hearing in context — and the cues it uses were learned in training, not chosen. Second, the arranger's sections have names and jobs. The router's don't. Look inside a trained model and the groups it wakes together seldom line up with any topic, style, or instrument a person would name. They are simply the groups that training found useful to keep apart.
So keep the band picture, because it is right about the thing that matters: a large ensemble can sound whole while only a few parts play, and that is the entire trick. Just don't let it tell you that the model knows which section is the saxophones. It doesn't need to.
A singer and four sections. Play the tune with the parts handed out by an arranger's chart — written in advance, by the phrase. Then hand them out by a router, live — decided note by note. Watch the singer stay lit throughout, count the chairs playing, and read who was chosen and why.
A chart is a lookup. A router is a decision.
Same band, same sections, same "two at a time". The only difference is where the choice comes from: a table written in advance, or a small learned function of the note in its context.
# the arranger: who plays is written down, by phrase, before the gig chart = {"verse": ["piano", "bass"], "chorus": ["horns", "bass"], ...} playing = chart[phrase_of(note)] # a lookup — same answer every night # the router: who plays is decided from the note itself, right now scores = router(x) # x: this note in its context playing = top_k(scores, k=2) # a decision — can differ for the same note elsewhere # in both: the singer (attention) is always on; only the sections are chosen y = singer(x) + sum(w * sections[s](x) for s, w in playing)
One voice, a few chairs, and a decision made live.
A real band already works the way a mixture of experts does: a singer who is always on, and an arrangement that has only a few sections playing under them at any moment, so a big ensemble stays cheap to run and still sounds whole. In the model the always-on part is attention, shared by everyone; the sections are the experts behind it, two per note. The differences are where the picture stops: the router chooses live, from context, by cues it learned rather than was given — and its sections have no names, no instruments, and no jobs a person would recognise.
The band tells you why it's cheap. The router tells you why it isn't a band: nothing is written in advance, and nobody in it knows which chairs are the horns.