The Improviser · Lesson 1 — The Odds ← Course

The odds — a probability distribution.

The next note isn't one fixed answer — it's whichever note the model judges most likely, out of many.

Think of a weather forecast. It doesn't flatly promise rain or shine — it says something like a 70% chance of rain. That figure is a probability: a way of putting a number on how likely something is, running from 0 (never) up to 1, or 100% (certain). A 50% chance means it could honestly go either way.

A soloist chooses the same way. Finishing a phrase, no single note is fixed in advance — but some are far likelier than others. The ear weighs each candidate and settles on a spread: maybe 78% for the note that resolves the line, the rest split among a handful of others. The note you actually hear is usually whichever scored highest.

That is also where the confidence number from the last page came from: it is just that highest probability. When one note towers over the rest — 78% against a scatter of small odds — the choice is sure. When several notes sit nearly level — 24%, 22%, 20% — no note is clearly more likely, so the pick is a close one and confidence is low.

There's one more twist. The note you hear is usually the top bar. But the model doesn't have to take the top bar every time — it can also draw from the odds, picking each note in proportion to its height. That's why the same phrase, played twice, can resolve differently. How daring that draw is is a dial you'll meet later.

This is the heart of how an LLM reaches every decision. It never simply "knows" the answer — it rates how likely each option is, then commits to one: usually the most likely, sometimes drawn from the odds. Where our soloist weighs six notes, a real model scores its whole vocabulary — tens of thousands of possible next tokens — every single step.

Slide the harmony from wide open to pulled home and watch the odds over the notes shift. Find the point where they sit so close that the model stops being sure — then hear it. The phrase resolves four times over, each landing drawn from the odds: a sure model keeps coming home to the same note; an unsure one wanders.

Slide the harmony, or pick a preset. Watch the odds — and the note.
odds over the next note
wide open pulled home
NO CLEAR NOTE

Ask for the odds, not just the note.

The model hands back a probability for every candidate note. The note it plays is the top one; the confidence is its share. Read the spread and you can see how sure the ear really is. 

step = model.predict_next(played, key, return_probs=True)

step.probs        # {'C': 0.78, 'E': 0.08, 'G': 0.06, ...}
step.note         # 'C'   (the highest bar)
step.confidence   # 0.78

Underneath, every note is a set of odds.

When one note towers over the rest, the choice is easy and the confidence is high. When the odds sit close together, no note is decisive — the phrase could go several ways. The single number you met in The Next Note was always the height of the tallest bar.

Every LLM answer works this way: a set of odds across the options, with the answer you see sitting on top. Hold onto that picture — it's where the next ideas begin: how the phrase is chopped into pieces before any of this runs, and how a tall bar can still be the wrong note.

Quick check · your ear
Three notes come back at 24%, 22% and 20%. What does that spread tell you?
Next: note by note →
▸ Go deeper — where the odds come from optional

From scores to probabilities

For each step, the model emits a raw score — a logit — for every note in its vocabulary. A function called softmax turns those raw scores into clean probabilities that add up to 1 — the full set of odds across the options (its formal name is a probability distribution). The note it plays is usually the one at the top of those odds, and what we've been calling "confidence" is simply that top probability. Note the wording: confidence here means the model's own predicted probability — not a statistical confidence interval, which is a different idea.

Confidence is not the same as correctness

A high probability means the patterns the model learned point strongly to that note — not that it is the "right" note. Models can be confidently wrong or hesitant when right; how well a model's stated confidence matches its real accuracy is called calibration. This gap is the seed of hallucination — the wrong-note-played-with-conviction — a few lessons on.

The odds can be tuned

Those odds aren't fixed before the model draws from them — how sharply they're peaked can itself be turned up or down, which is what makes playing steadier or more daring. That's a whole lesson later; for now, just know the odds can be tuned.

One set of odds per note

A real solo is many notes long, and the model produces a fresh set of odds for each one, commits to a note, then does it again. The flowing line you hear is a chain of these small bets — each note the most likely pick from its own little set of odds.