The maths — conditioning and diminishing returns.
The few ideas from probability that sit under few-shot — gathered in one place.
None of this is required to use few-shot — it is the probability that sits under it, set down in one place. If you want the machinery, it rests on just two things: one idea from probability that explains why examples work at all, and one simple curve that explains why a few are plenty.
The first is conditioning — the model always answers “given” everything it has been shown, and examples are extra things to be given. The second is diminishing returns — reliability rises toward a ceiling while cost rises in a straight line.
Start with the curve: drag the number of examples and watch reliability climb toward its ceiling while the cost keeps rising straight past it.
Drag the examples. Green curves up to a ceiling; red rises in a straight line.
reliabilitycost (tokens)
n0
The formulas, in one place.
Each is written the way you would meet it in a textbook, with a plain-language line above and where it turned up in this lesson below.
Conditional probability
A conditional probability is the chance of one thing given that another is already true — the vertical bar reads “given”. The unlocking identity is P(A | B) = P(A and B) / P(B): you throw away every case where B is false and ask how often A happens in what is left. That is why conditioning on the right thing can swing a probability hard — it changes the pool of cases you count over. In general P(A | B) is not equal to P(A).
P(A | B) = P(A ∩ B) / P(B) — “A, given B”
P(A | B) ≠ P(A) in general — the condition moves the odds
P(A | B) ≠ P(A) in general — the condition moves the odds
ExampleAcross all incidents, P(penalty) might be just 0.05. But given the ball striking an arm raised away from the body, P(penalty | ball to a raised arm) could be 0.7. Same incident; the condition changed the odds.
In Lesson 1: every next-token guess is conditional on the text so far.
Few-shot = extra conditioning
Zero-shot asks the model for P(answer | question). Few-shot hands it P(answer | examples, question) — the examples join the right-hand side of the bar as more “given” information. The question itself is unchanged; what changes is everything the model is told to assume. The illuminating part: relevant conditioning concentrates the distribution — it piles probability onto answers that fit the examples (the right format, the right kind of verdict) and starves everything else. That concentration is the steering you felt on Page 1.
zero-shot: P(y | x)
few-shot: P(y | e1, e2, …, ek, x) — examples join the condition
few-shot: P(y | e1, e2, …, ek, x) — examples join the condition
ExampleAsk the VAR “Handball — penalty?” on its own and you cannot guess the style of the reply — it might ramble, it might just say “Penalty.”, it might give a neat one-liner. Only about 1 in 5 replies land in one tidy format: a verdict, a short reason, then a confidence level — like “PENALTY · arm away from body · high”. Now do it few-shot: first paste in two example incidents with their answers already filled in, in that same format, then ask your real one. About 9 in 10 replies now copy the format. Same question; the two worked examples made that style the likely answer.
In Lesson 4: Page 1 — the answer snapping into the demonstrated shape.
Which examples — ruling tasks out
There is a second way to read few-shot: the examples help the model infer which task you mean, out of all the tasks it could perform. By Bayes' rule the odds of a task update with evidence: P(task | examples) ∝ P(examples | task) · P(task). The key point is that an example only teaches something if it can rule a task out. Three PENALTY examples fit “always give a penalty” and “actually judge the incident” equally — they cannot tell them apart. A single NO PENALTY example is impossible under “always give a penalty”, so that reading collapses and the real task wins.
P(task | examples) ∝ P(examples | task) · P(task) — Bayes
ExampleSay the model might be doing “echo PENALTY”, “judge the incident”, or “rate confidence”. Three clear penalties are consistent with all three — no help. A PENALTY and a NO PENALTY cannot come from “echo PENALTY”, so its probability drops to ~0 and “judge the incident” takes over.
In Lesson 4: Page 3 — why boundary examples teach more.
Diminishing returns
Reliability does not climb in a straight line; it approaches a ceiling. Each example closes the same fraction of the gap that is left, so the absolute gain shrinks every time. That fixed-fraction rule is the unlocking identity — a geometric decay of the gap, gap(n) = (1 − f)·gap(n−1), which in closed form is an exponential approach. R0 is the zero-shot level, R∞ the ceiling few-shot can reach, and k how fast you get there.
gap(n) = R∞ − reliability(n) = (R∞ − R0)·e−k·n
reliability(n) = R∞ − (R∞ − R0)·e−k·n
reliability(n) = R∞ − (R∞ − R0)·e−k·n
ExampleWith R0 = 0.25, R∞ = 0.94, k = 0.8: n=1 → 0.63, n=2 → 0.80, n=3 → 0.88. The jumps are +38, +17, +8 points — each about 55% of whatever gap was left. By n=5 you are at 0.93, a whisker under the ceiling.
In Lesson 4: Page 2 — the reliability meter that shot up, then flattened.
Linear cost & where to stop
Every example is more text, so the token cost is a straight line: tokens(n) = base + per·n. Now put the two curves together. The benefit of one more example is its marginal gain — the shrinking step k(R∞−R0)e−k·n — while the marginal cost is constant, per tokens, every single time. Early on the gain dwarfs the cost; the sweet spot is the n where the next example's gain stops being worth its tokens. Past that you pay a straight-line cost for a benefit that has flattened to nothing.
tokens(n) = base + per·n — linear
gain(n) = (R∞−R0)·e−k(n−1)·(1−e−k) shrinks · cost step = per flat
gain(n) = (R∞−R0)·e−k(n−1)·(1−e−k) shrinks · cost step = per flat
Examplebase = 120, per = 60. Example 1 buys +38 points for 60 tokens; example 5 buys +2 points for the same 60. Same price, a fraction of the value — which is why two or three is usually where you stop.
In Lesson 4: Page 2 — the cost bar climbing while the meter stalled.
Where to take it next.
Each idea here is a doorway. If you want to keep going, these are the next words to search — each is the natural step up from what you just saw:
Bayes' theoremupdating the odds of a hypothesis as each example arrives — the engine behind “which task”.
In-context learningthe research name for a model learning from examples placed in the prompt, without any training.
Exponential decay & half-lifethe shape of the diminishing-returns curve — the same maths as cooling coffee or fading echoes.
Marginal analysisweighing the gain of one more against its cost — the economist's tool for “where to stop”.
None of it is needed to finish the course — but each deepens the picture of what few-shot is really doing.
Quick check
You add a 6th, 7th and 8th example to the prompt and answers barely improve. This is…