The Third Umpire · Lesson 1 — The Odds ← Course

The odds — a probability distribution.

A decision isn't one fixed answer — it's whichever answer the umpire judges most likely.

Ask whether it will rain tomorrow and a good forecaster won't say yes or no — they'll hand you a number: a 70% chance. That figure is a probability, a way of putting a number on how likely something is, running from 0 (it never happens) up to 1, or 100% (certain). At 50% it could honestly go either way.

The third umpire decides the same way. It doesn't lock onto a verdict straight away. For each possible call — OUT or NOT OUT — it weighs the evidence and assigns a probability: maybe 96% OUT against 4% NOT OUT, or a far closer 54% against 46%. The verdict you finally see — the call — is simply whichever option came out highest.

That top figure is also the confidence you met in The Replay. When one option towers over the other — 96% versus 4% — the umpire is sure, and confidence is high. When the two sit nearly level — 54% versus 46% — the call still gets made, just barely, and confidence is low.

Usually the call is that tallest bar. But the umpire needn't always take it: a model can also draw from the odds, picking each option in proportion to its height. That's why the very same appeal, asked twice, can come back differently — and how adventurously it draws is a dial you'll meet later.

This is the heart of how an LLM reaches every decision. It never simply “knows” the answer — it scores how likely each one is, then commits: usually to the top, sometimes to a draw. And a real model isn't weighing two options but spreading its odds across a vocabulary of tens of thousands.

Drag the evidence from beaten to edged and watch the two probabilities shift. Find the point where they sit so close the call is barely more than a coin toss.

Drag the evidence, or pick an appeal. Watch the odds — and the call.
beaten edged
OUT — but only just

Ask for the odds, not just the answer.

The model hands back a probability for each option. The reply is the top one; the confidence is its share. Read the spread and you can see how sure the umpire really is. 

result = umpire.review(evidence, return_probs=True)

result.probs        # {'out': 0.58, 'not out': 0.42}
result.call         # 'out'   (the top of result.probs)
result.confidence   # 0.58
result.sample()     # 'not out'   (drawn from the odds — can differ each time)

Underneath, every call is a set of odds.

A call is never a lone verdict — it's a set of odds, and the verdict is just the tallest bar. When that bar towers over the rest the call is easy and confidence is high; when the odds sit close, the umpire commits anyway, low confidence and all. The single number you met in The Replay was always the height of that top bar.

Every LLM answer is built this way: a spread of odds, with the reply usually the top bar. Hold onto that picture — it's where the next ideas begin: drawing more or less adventurously, and how a tall bar can still be wrong.

Next: frame by frame →
▸ Go deeper — where the odds come from optional

From scores to probabilities

For each step, the model emits a raw score — a logit — for every option in its vocabulary. A function called softmax turns those raw scores into clean probabilities that add up to 1 — the full set of odds across the options (its formal name is a probability distribution). The reply is usually the option at the top of those odds, and what we've been calling “confidence” is simply that top probability. Put plainly: confidence here means the model's predicted probability for an answer — not a statistical confidence interval, which is a different idea.

Confidence is not the same as correctness

A high probability means the patterns the model learned point strongly to that answer — not that the answer is true. Models can be confidently wrong or hesitant when right; how well a model's stated confidence matches its real accuracy is called calibration. This gap is the seed of hallucination, a few lessons on.

The odds can be tuned

Those odds aren't fixed before the model picks from them — how sharply they're peaked can itself be turned up or down, which is what makes answers steadier or more varied. That's a whole lesson later; for now, just know the odds can be tuned.

One set of odds per token

A real answer is many tokens long, and the model produces a fresh set of odds for each one, commits to a token, then does it again. The confident-looking sentence you read is a chain of these small bets — each word the most likely pick from its own little set of odds.