The Improviser · Lesson 4 — The Maths ← Course
Optional · go further

The maths — conditioning and diminishing returns.

The two ideas from probability that sit under a few example phrases — gathered in one place.

None of this is required to hand the player a few phrases — it is just the probability sitting underneath. If you want that machinery, it rests on two things: one idea that explains why example phrases work at all, and one simple curve that explains why a few are plenty.

The first is conditioning — the player always plays the next note "given" everything it has already heard, and example phrases are extra things to be given. The second is diminishing returns — reliability rises toward a ceiling while the token cost rises in a straight line.

Start with the curve: drag the number of example phrases and watch reliability climb toward its ceiling while the cost keeps rising straight past it.

Drag the example phrases. Green rises to a ceiling; red keeps climbing in a straight line.
n — number of example phrases
0
→ reliability vs token cost
reliability25%
cost · tokens120

The formulas, in one place.

Each is written the way you'd meet it in a textbook, with a plain-language line above and where it turned up in this lesson below.

Conditional probability
A conditional probability is the chance of one thing given that another is already true — the vertical bar reads "given". The unlocking identity is P(A | B) = P(A and B) / P(B): you throw away every case where B is false and ask how often A happens in what's left. That's why conditioning on the right thing can swing a probability hard — it changes the pool of cases you count over. In general P(A | B) is not equal to P(A).
P(A | B) = P(A ∩ B) / P(B) — "A, given B"
P(A | B) ≠ P(A) in general — the condition moves the odds
ExampleAcross all bars of music, P(next note = a flat blue third) might be just 0.05. But given a walking bassline already sitting on a dominant chord, P(flat third | that context) could be 0.6. Same note; the condition changed the odds.
In Lesson 1: every next-note guess is conditional on the music heard so far.
Example phrases = extra conditioning
Zero-shot asks the player for P(answer | ask). A few example phrases hand it P(answer | phrases, ask) — the phrases join the right-hand side of the bar as more "given" information. The ask itself is unchanged; what changes is everything the player is told to assume. The illuminating part: relevant conditioning concentrates the distribution — it piles probability onto continuations that fit the phrases (the right phrasing, the right shape) and starves everything else. Few-shot isn't teaching the player new notes; it's continuing a pattern — and pattern-continuation is conditioning.
zero-shot: P(y | x)
few-shot: P(y | e1, e2, …, ek, x) — phrases join the condition
ExampleAsk "give me a two-bar phrase over this chord" on its own and you can't guess the style — it might ramble, it might land a clean bebop line, it might go outside. Only about 1 in 4 replies land in one tidy shape. Now do it few-shot: paste two example phrases in that same shape first, then ask. About 8 in 10 replies now copy the phrasing. Same ask; the worked phrases made that style the likely continuation.
In Lesson 4: Page 1 — the phrase snapping into the demonstrated shape.
Diminishing returns & linear cost
Reliability doesn't climb in a straight line; it approaches a ceiling. Each example phrase closes the same fraction of the gap that's left, so the absolute gain shrinks every time — a geometric decay of the gap, gap(n) = (1 − f)·gap(n−1), which in closed form is an exponential approach. Meanwhile every phrase is more text, so the token cost is a straight line: tokens(n) = base + per·n. Put them together and the sweet spot is the n where the next phrase's shrinking gain stops being worth its flat cost.
reliability(n) = R∞ − (R∞ − R0)·e−k·n — saturates
tokens(n) = base + per·n — linear
gain(n) = (R∞−R0)·e−k(n−1)·(1−e−k) shrinks · cost step = per flat
ExampleWith R0 = 0.25, R∞ = 0.94: n=1 → 0.60, n=2 → 0.80, n=3 → 0.88. The jumps are +35, +20, +8 points — each closing about half the gap that was left. Phrase 1 buys +35 points for 60 tokens; phrase 6 buys under +1 for the same 60 — which is why two or three is usually where you stop.
In Lesson 4: Page 2 — the reliability meter that shot up, then flattened while the cost bar kept climbing.

Where to take it next.

Each idea here is a doorway. If you want to keep going, these are the next words to search — each is the natural step up from what you just saw:

In-context learningthe research name for a model learning from examples placed in the prompt, without any training.
Conditional probabilitythe "given" bar itself — the chance of one thing once another is assumed true. A full idea in its own right.
Information gain & diminishing returnshow much each new example actually tells you — and why that shrinks toward a ceiling, the same maths as cooling coffee or fading echoes.
Prompt-token budgetingweighing the gain of one more example against the tokens it spends — the "where to stop" calculation in practice.

None of it is needed to finish the course — but each deepens the picture of what a few example phrases are really doing.

Quick check · your ear
You add a 6th, 7th and 8th example phrase to the prompt and the phrasing barely improves. This is…
On to Lesson 5 →
▸ Go deeper — what this page does and doesn't claim optional

Conditioning is exact; the curve is a picture

P(y | phrases, ask) is genuinely what the player computes — a few example phrases really are extra conditioning, not an analogy. The smooth reliability curve, by contrast, is a useful picture of the behaviour, not a literal calculation happening inside the network.

The curve is a model, not a law

Real few-shot reliability is bumpy and depends on the task, the phrases, and the model. The exponential-approach curve captures the shape everyone observes — steep, then flat — and the numbers here (R0, R∞, k) are illustrative, chosen to match Page 2. Don't read exact percentages off it.

Why "fixed fraction of the gap" shows up so often

Any process where each step removes a constant proportion of what remains gives an exponential — cooling, a fading echo, a loan paid down. A few example phrases fit the same mould: each one removes a constant slice of the reliability still missing, so the gains fall away on the same curve while the tokens keep adding up in a straight line.

If you want to go there

The search terms are: conditional probability, in-context learning, information gain, and exponential decay. The first is a proper idea in its own right; the others are short, rewarding detours.