The training ground — predicting the next word.
The model only ever guesses the next word — so why does that add up to something intelligent?
Everything you've seen so far is the model doing one small thing over and over: looking at the words so far, placing odds on what comes next, and picking one. (Each "word" here is really a token — the chunks from the last page; we'll just say "word".) That sounds far too simple to produce a thoughtful answer. So here is the real puzzle — how does guessing one word at a time turn into something that seems to understand?
The answer: to guess the next word well, you have to know a surprising amount. Finish "the capital of France is ___" and you need a fact. Finish "she tripped on the step and ___" and you need to know how the world works. "2 + 2 = ___" needs arithmetic; "he was furious, so he ___" needs a sense of how people behave. Each blank quietly demands real knowledge.
A young footballer at the training ground doesn't memorise every move they face — by drilling thousands of them, they slowly soak up the patterns of the game, until the right response becomes instinct. A model learns the same way. During a phase called training, it reads an enormous amount of human writing and does nothing but try to predict the next word, nudging itself a little whenever it's wrong.
Do that across billions of sentences and you can't get good by memorising — you're forced to absorb the regularities underneath: grammar, facts, cause and effect, the shape of an argument. That hard-won sense of how things fit together is where the apparent intelligence comes from.
Watch a few reps — each one is a "guess the next word". See what the model must already know to play it.
One move, learned from reading.
The model only ever predicts the next token. Nobody wrote these answers in by hand — it picked them up from text, the way a player learns the game by playing it.
model.predict_next("The capital of France is") # → "Paris" (one token) # then it adds that token and calls predict_next again — word after word. # never hard-coded — learned from billions of next-word guesses
Predicting well is learning how it works.
Five reps, five different kinds of knowing — a fact, then cause and effect, then arithmetic, then how people act, then how words relate. None of it was programmed in. The only way to get good at "guess the next word", across everything people have written, is to take on how language, the world, and reasoning actually behave.
That's where the intelligence comes from. Like a footballer on the training ground, the model never memorised every move; by drilling enough of them it learned the patterns underneath — and those patterns are what make each small bet land somewhere sensible.