The Third Umpire · Lesson 2 — The Maths ← Course
Optional · go further

The maths — tokens and attention.

The little bit of arithmetic under Lesson 2 — why a window fills on schedule, and why a longer one costs so much more than a short one.

None of this is required to go on. It is the arithmetic Lesson 2 leaned on, set down in one place: tokens add up, history grows in a straight line, and — the interesting one — the cost of paying attention grows with the square of the length.

The first two are just addition. The third is the one worth your time: it explains why a long window is expensive to run, not merely large.

Start with that third idea — drag the context length and watch the attention cost race ahead of the text. Doubling the words does not double the work.

Drag the context length. Text grows in a line; the attention step grows with the square.
context length — n (tokens)
n2,000
→ growth vs 1,000 tokens
text · n×1
attention · n²×1

The formulas, in one place.

Each is written the way you'd meet it in a textbook, with a plain-language line above and where it turned up in this lesson below.

The context budget
Everything in a prompt is measured in one unit — tokens — so the parts simply add up, and the total can never exceed the window W. The useful move is to rearrange it with subtraction: the history you can keep is whatever's left after the fixed costs come off the top, H ≤ W − (S + M + R). Spend more of the window on a long system prompt or a long reply, and there's less room for the conversation. It's one budget, shared by all of it.
W ≥ S + H + M + R — window holds all four
Hmax = W − S − M − R — room left for history
ExampleW = 2,000, system = 400, your message = 200, reply = 400. Then Hmax = 2000 − 400 − 200 − 400 = 1,000 tokens. At ~250 a turn, that's 4 turns before the oldest must go.
In Lesson 2: The Shared Budget — the 2,000-token bar.
History grows in a straight line
The history is just the running total of every turn so far — add a turn, add its tokens. The Σ symbol means “add them all up,” and when the turns are about the same length k it collapses to plain multiplication: k + k + … (t times) = k·t. That straight-line growth is why you can predict exactly when the window fills, and why trimming arrives on a schedule rather than by surprise.
H(t) = Σi=1t ki — sum of all turns
H(t) ≈ k · t — when turns are equal-sized
Examplek = 250 tokens a turn, cap = 1,000. It fills when 250·t = 1,000, so t = 4. Turn 5 pushes the oldest turn out of the window.
In Lesson 2: turns stacking up in the bar until one drops off.
Attention is quadratic — n²
To read a passage, the model's attention step compares tokens pairwise. With causal masking, each token attends to itself and every earlier token: 1 + 2 + … + n = n(n+1)/2, about n²/2 — half the full n × n grid, but still growing with the square of n. So double the context and the attention step does roughly four times the work. That is a large part of why a longer window is expensive; the window's ceiling itself comes from how the model was trained and how much memory a long conversation costs to serve, not from this step alone.
comparisons ≈ n²/2 — with masking, each token attends to itself and the ones before it
1 + 2 + … + n = n(n+1)/2 — that exact count
attention(2n) / attention(n) ≈ 4 — 2× the text, 4× the attention work
Examplen = 1,000 tokens → about 500,000 comparisons. Go to n = 2,000 → about 2,000,000 — twice the text, four times the attention work.
In Lesson 2: why a bigger window costs more — the reason it isn't a free setting.

Where to take it next.

Each idea above is a doorway. If you want to keep going, these are the next words to search — each is the natural step up from what you just saw:

Big-O notationthe shorthand for how cost grows with size — O(n) for the budget, O(n²) for attention.
Self-attentionthe mechanism whose every-token-sees-every-token wiring is where the n² comes from.
The KV cachethe standard trick that stops the model re-reading the whole history on every new token.
Sparse & linear attentionresearch aimed at beating the n² wall so windows can stretch — FlashAttention, sliding windows, and more.

None of these are needed to finish the course — but each one turns “why does a longer window cost so much?” into “here's how modern models stretch it anyway.” That trail, from a simple sum to the cost of attention, is most of what separates a small context from a million-token one.

On to Lesson 3 →
▸ Go deeper — what this page does and doesn't cover optional

Two different kinds of arithmetic

The budget and the history are exact bookkeeping — addition you could do by hand, counting tokens. The n² is a different sort of claim: it's about cost, not the token count. It tells you how the work scales as the context grows, which is what makes a longer window so much more expensive to run.

Clever engineering softens it

Real systems don't pay the naïve n² in full. Caching past keys and values, sparse and windowed attention, and other tricks cut the bill substantially. But the underlying scaling is why a longer window is a real engineering feat, not a free setting you can simply turn up.

If you do want to go there

The search terms are: Big-O notation, self-attention, the KV cache, and FlashAttention. Fair warning — like the network maths in Lesson 1, that path is a course (or three) in itself.