The Improviser · Lesson 5 — The Maths ← Course
Optional · go further

The maths — softmax and temperature.

How a single dial sharpens or flattens a set of scores into odds — the formula under all of Lesson 5.

You don't need any of this to use temperature. But the maths is unusually clean — one formula, softmax, with a single extra symbol dropped in (÷ T) — so it's worth seeing once, set down in one place.

Softmax turns the model's raw scores for each candidate note into probabilities. Temperature divides those scores first, which stretches or squeezes the gaps between them. That's the whole mechanism — the daring dial from Page 1 was this, and the other dials from Page 2 were trims applied afterwards.

Nudge the three raw scores for notes C, E and G, then drag the temperature and watch the odds spike or flatten — then read the formula that's doing it.

Set the raw scores, then let temperature reshape the odds. Low T sharpens; high T flattens.
raw scores (logits) — z
note C2.4
note E1.0
note G0.4
temperature — T
T1.0
plain softmax
→ probabilities · softmax(z / T)
C0%
E0%
G0%

The formulas, in one place.

Each is written the way you'd meet it in a textbook, with a plain-language line above and where it turned up in this lesson below.

Softmax
Each candidate note gets a raw score — a logit, z — any number at all, plus or minus, and not a probability. Softmax fixes that: exponentiate each score so they're all positive, then divide by the total so they sum to 1. The unlocking identity is that only the gaps between scores matter — pi/pj = ezi−zj — so adding the same amount to every score changes nothing.
pi = ezi / Σj ezj — scores → probabilities
pi / pj = ezi − zj — only the gap matters
ExampleScores 2.4, 1.0, 0.4. e2.4≈11.0, e1.0≈2.72, e0.4≈1.49; total ≈15.2 → 0.72, 0.18, 0.10. The C–E gap is 1.4, and 0.72/0.18 ≈ 4.0 = e1.4.
In Lesson 1: the engine behind every bar. Here: what temperature reaches into.
Temperature
Temperature divides every score by T before softmax runs. The unlocking point is what that does to the gaps: dividing the scores by T scales every gap by 1/T — the gap zi−zj becomes (zi−zj)/T. Below 1, T blows the gaps up, so the favourite note runs away and dominates; above 1, T shrinks the gaps, so the notes even out. T = 1 is plain softmax, untouched.
pi(T) = ezi/T / Σj ezj/T
gap zi−zj → (zi−zj) / T — every gap scaled by 1/T
ExampleThe C–E gap is 1.4. At T = 0.5 it acts like 2.8, so the ratio is e2.8 ≈ 16 — C pulls away. At T = 2 it acts like 0.7, so the ratio is e0.7 ≈ 2.0 — far closer. Same scores; T sets how daring the next note is.
In Lesson 5: Page 1 — the dial that made the improviser safe or wild.
The two extremes
Push T toward 0 and the favourite note's share races to 1 while the rest vanish — softmax becomes argmax, always the single top pick (greedy, deterministic: the same note every time). Push T toward infinity and every gap shrinks to 0, so the scores stop mattering and the odds flatten to uniform, 1/n each — a blind pick among all notes, maximum daring. Everyday settings sit in between.
T → 0 : p → 1 for the top note, 0 for the rest (argmax)
T → ∞ : pi → 1/n for every note (uniform)
ExampleScores 2.4 / 1.0 / 0.4. At T = 0.1: C ≈ 99.99% — all but certain. At T = 10: about 37 / 32 / 30% — nearly a three-way tie, the scores barely felt.
In Lesson 5: Page 1's “safe at zero, wild at the top”, made exact.
Trimming — top-k & top-p
The other dials don't touch the scores; they cut the distribution after softmax and renormalise. Pick a surviving set S — the k highest for top-k, or the smallest set whose probabilities reach p for top-p — zero out the rest, then divide each survivor by the survivors' own total so they add to 1 again. The unlocking idea is simply that renormalising means dividing by the mass you kept.
pi′ = pi / Σj∈S pj for i ∈ S, else 0 — keep S, re-scale to 1
ExampleOdds 0.45 / 0.25 / 0.15 / 0.08 / 0.05 / 0.02 across six notes. Top-p 0.9 keeps the first four (they sum to 0.93). Renormalise: 0.45/0.93 = 0.48, 0.25/0.93 = 0.27, and so on — the survivors soak up the 7% that was cut.
In Lesson 5: Page 2 — the quietest notes dropping to zero as the survivors grew.

Where to take it next.

Each idea here is a doorway. If you want to keep going, these are the next words to search — each is the natural step up from what you just saw:

Temperature scalingthe same ÷ T trick used to recalibrate a model's confidence after training.
Greedy vs sampling decodingalways taking the top note, versus rolling the dice on the odds.
Entropy of the distributiona single number for how flat versus peaked the odds are — raising temperature raises entropy.
Top-k / top-p samplingthe trimming dials from Page 2, and the papers that introduced them.

None of it is needed to finish the course — but each deepens the picture of what the daring dial is really doing. This is the exact softmax a real LLM runs to turn its logits into next-token odds, and the same maths, borrowed straight from physics, turns up far beyond music-making models.

Quick check
You want the improviser to play the exact same phrase every time. Set the temperature…
Next: the phantom note →
▸ Go deeper — why the formula looks like this optional

Why exponentiate at all?

The exp does two jobs: it makes every score positive (so the results can be probabilities), and it turns adding scores into multiplying odds. That's exactly what makes “only the gaps matter” true, and it's the mirror image of the log-probabilities from Lesson 1, where multiplying probabilities became adding logs.

The numerical-stability trick

In practice you subtract the largest score from all of them before exponentiating: softmax(z) = softmax(z − c) for any constant c, because a shared constant cancels top and bottom. The answer is identical, but it keeps the numbers from blowing up. That's the m = max(...) you'd see in real code — and in the demo above.

Temperature is borrowed from physics

The name isn't a metaphor invented for AI — it's lifted from the Boltzmann distribution in statistical physics, where a high temperature makes all states nearly equally likely and a low one pins the system into its lowest-energy state. Same formula, same intuition, a century older.

If you do want to go there

The search terms are: softmax, the Boltzmann distribution, entropy, and nucleus sampling. The first three are one shared idea wearing different hats across ML, physics, and information theory.