The Video Assistant · Lesson 2 — The Shared Budget ← Course

The shared budget — sharing the context window.

The window holds far more than your question — and it all has to fit at once.

On the last page, the window held video frames, and the oldest one fell out when it filled up. A real conversation works the same way — except the window has to hold much more than just your latest question.

On every turn, four things share that one window: the instructions you gave the model at the start (its “system prompt”), the whole chat so far, your new message, and the space it needs to write a reply. All four have to fit together.

And the window never gets any bigger. A long chat simply fills it up. Once it's full, the oldest parts of the conversation are dropped to make room — just like those frames sliding off the screen.

Add a few turns and watch the window fill. Keep going, and you'll see the earliest turn get pushed out.

Three voices — and every exchange piles into the same window.
System · the captain's briefing You are the video assistant referee. Give a one-line verdict.
Send the referee's check and see the verdict come back.
Add a turn. Watch the four parts share one fixed window.
one context window · 2,000 tokens
system prompt conversation so far your message reply (reserved)

It's all one list — and it all costs tokens.

Every turn, you send the model the same kind of list: the system prompt, the history, and your new message. The whole thing must fit the window, with room kept for the reply. 

messages = [
    {"role": "system", "content": briefing},   # set once, sent every turn
    *history,                                  # every past turn — this keeps growing
    {"role": "user", "content": new_message},  # your latest message
]

# the whole list must fit the window; trim the oldest history if it won't
reply = var.chat(messages, max_output_tokens=400)   # reply space, reserved up front

One window, shared by everything.

The instructions, the whole chat so far, your new message, and the space for the reply all share one window. It never gets bigger — it just fills up, and the oldest turns drop out to make room.

So a big part of building with an LLM is choosing what to keep in the window — and what to drop, shorten, or look up only when you need it.

Next: does it remember you? →
▸ Go deeper — managing the budget optional

Reserve room for the reply

The model also needs space to write its answer, and that space is carved out of the same window before it begins. That's the max_output_tokens in the code above. Cram the window with input and you can leave too little room for the model to respond.

When it won't fit: trim, summarise, retrieve

Three moves keep a long conversation inside the window: drop the oldest turns (what you just saw); replace a long history with a short running summary; or store everything outside the window and fetch only the relevant bits on demand. That last one — retrieval — is powerful enough to earn its own lesson later.