The Third Umpire · Lesson 1 — The Replay ← Course

The replay — how a model decides.

It works like a cricket video umpire: weigh every clue, then commit to a call.

When a call is too close for the on-field umpires, cricket sends it upstairs — to a third umpire watching the replays. Did the batter nick the ball through to the keeper, or not? No one clue answers it, so the umpire weighs several at once: the deviation (did the ball change course as it passed the bat?), snicko (a sound trace — a sharp spike means contact), hot-spot (a heat camera — a bright mark where ball met bat), and the bat–pad gap (so a noise off the pad isn't mistaken for an edge).

The clues rarely agree cleanly. The umpire weighs them together and always commits — OUT or NOT OUT — surely when they line up, barely when they don't. Upstairs there is no "leave it": a call has to be made.

A large language model (or LLM) — the AI behind tools like ChatGPT — decides the same way, only its call is a single word. Given the text so far, it weighs everything it learned in training, puts odds on every word that could come next, and commits to one — then does it again for the word after that. The same instinct as the umpire, made one word at a time.

Review the three close calls and watch the umpire weigh the evidence — then see what happens when the clues disagree.

Send an appeal upstairs. Watch the umpire weigh the evidence.
VS AS RI
Evidence · the umpire weighs all of it
Snicko —·
Hot-spot —·
Deviation —·
Bat–pad —·
Read by an umpire shaped by seasons of past cricket — that experience is baked in, not looked up.
AWAITING DECISION

You hand over the inputs. The umpire decides.

You don't write the weighing — you assemble the evidence and ask. The umpire weighs it with an eye shaped by seasons of cricket, and hands back a call, along with the odds it put on that call. 

# Hand the umpire the evidence — you don't code the weighing.
verdict = umpire.review(
    snicko="faint sound",
    hotspot="no mark",
    deviation="no clear deviation",
    bat_pad="small gap",
)

# verdict.call → "OUT"   ·   verdict.confidence → 0.54   (the odds on that call)

Weigh everything, commit to one call.

Three deliveries, three different bundles of evidence. A spike alone didn't decide it — the umpire weighed the snicko against the hot-spot, the deviation, and the bat–pad gap, and always committed: a confident call when they agreed, a barely-there one when they didn't. It never left the decision unmade.

An LLM chooses its next word the same way. It weighs the words so far, puts odds on every candidate, and commits to one — sure when the odds pile onto a front-runner, only just when they're split. Then it repeats, word after word.

Next: the odds →
▸ Go deeper — how the umpire actually decides optional

It's a distribution, not a single verdict

When the umpire renders a call, it isn't picking from a fixed menu of two answers. Underneath, the model produces a probability for every possible next token, and the “decision” you see is simply the most likely one. The confidence number in this lesson — 0.96 for the thick edge, 0.54 for the feather — is exactly that: the share of probability sitting on the top option. A tall, lonely spike means the model is sure; a near-even split across options means it isn't, which is why the feather comes back as a barely-there call rather than a confident one.

Weighed against training, not looked up

Where do those probabilities come from? From patterns the model absorbed across an enormous amount of text during training — the umpire's accumulated experience, baked into its weights. It is not consulting a database at decision time, and it has no specific past match it can replay. (Looking things up live — pulling the actual archive footage — is a separate capability called retrieval, and it gets its own lesson later.) This is the difference between instinct shaped by experience and checking the records.

Why “probabilistic” matters

Two consequences follow, and both become later lessons. Because the answer is sampled from a distribution, the same appeal can produce different calls depending on how greedily you pick — that's the temperature dial. And because the top option is merely the most probable, not the verified truth, a confident-looking answer can still be wrong — that's the root of hallucination.