The Improviser · Lesson 1 — Note by Note ← Course

Note by note — tokens.

A model doesn't take in a whole melody at once — it reads a stream of small pieces called tokens.

A musician reading sheet music doesn't swallow the tune whole. They take it in one event at a time — this note, that duration, the next note — and it's the sequence of those small events that makes a line. Nothing arrives as "the melody"; it arrives as a run of little symbols on the page.

An LLM reads much the same way. Before it can do anything, it first cuts the input into tokens — small pieces. A single plain note is one token; something richer, like a chord, splits into several. The model only ever sees this stream of tokens — never "the tune" itself. And each token reaches it as a plain ID number, which is all the numbers on the right are.

Take a short phrase and break it into tokens yourself — hear each one land — then see the raw numbers the model actually reads.

A phrase is just a run of events. Break it into tokens.
C · D · E · G · [ C-major chord ]

The same phrase, as the model takes it in.

You hand over the notes; the model first splits them into tokens. A single note is one token — but a chord stacks into several, one per pitch. 

tokens = model.tokenize(["C", "D", "E", "G", "Cmaj"])
# → ['C', 'D', 'E', 'G', 'C', 'E', 'G']

len(tokens)     # 7 tokens
tokens[4:7]    # ['C', 'E', 'G'] — one chord, three tokens

Everything is tokens.

The phrase, the answer, every note the model reads or plays — all of it is a sequence of tokens. The model never sees a staff or a feeling directly; it sees these small pieces, and the numbers behind them.

Tokens are the unit an LLM works in. Where the odds over the next token come from — how the model learned to guess it at all — is the next page.

Quick check · your ear
The phrase looked like 5 events, but tokenized to 7. Why the extra two?
Next: the practice room →
▸ Go deeper — the theory of tokens optional

What a token is

A token is the unit a language model actually operates on — not a single letter, and not a whole word or a whole bar, but a chunk in between. Every model carries a fixed vocabulary of these chunks (tens of thousands of them, on the order of 100,000–200,000 for recent text models), and any input is cut into a sequence of them before the model does anything else. It never sees your sentence — or your melody — as meaning; it sees a list of token IDs, like the MIDI-style numbers on the right.

Music is tokenized too

The same idea powers models that generate music. A piece is serialised into a stream of small events — a pitch, a duration, a bar line, a rest — and each becomes a token from a fixed vocabulary. A chord isn't one symbol; it's several note-tokens sounded together, which is why our C-major chord came apart into C + E + G. Give the model the tokens so far and it predicts the next one, exactly as a text model predicts the next word-piece.

Why sub-word (and sub-event) pieces

It's a trade-off. Work at the finest level — every letter, every raw MIDI tick — and sequences get very long and the model wastes capacity relearning how the pieces assemble. Work at the coarsest level — whole words, whole bars — and the vocabulary explodes while anything unseen has no entry at all. Mid-sized pieces thread the needle: common things get one token, rare things are rebuilt from familiar fragments.

Why it matters when you build

Two things are measured in tokens, not words or notes: the context limit — how much the model can hold at once (Lesson 2) — and the price you pay per call. Tokenization also explains some odd blind spots: because a text model sees sub-word chunks rather than letters, asking it to count the letters in a word or reverse a string is genuinely hard — it isn't looking at characters at all.