The Third Umpire · Lesson 10 — Drawing Stumps ← Course

Drawing stumps — knowing when to stop.

The hardest step in the loop is the one that ends it.

Every turn, the agent faces the same fork: enough, or go again? Both wrong answers cost. Stop too early and the verdict rests on too little. Never stop and the loop runs on, burning time and money and sometimes circling the same ground for no new evidence.

So agents stop on a condition, set in advance: enough evidence to be confident, a limit on turns, a budget or time ceiling. “I have enough” has to be a rule the loop can check — not a feeling it might never reach.

Give the same review four different stop rules and see where each one leaves it.

Choose a stop rule
same appeal · different rule for when to quit

A stop is a decision, not an accident.

Confidence is the stop you want — quit the moment the evidence settles it. But confidence may never arrive, so real agents pair it with a hard backstop: a cap on turns, a budget, a timeout. One rule to stop well, another to stop at all.

A loop with no exit is the classic agent failure — endless calls, mounting cost, no verdict. Deciding up front what “done” means is what keeps an autonomous agent from running away with your time and money.

Next: Lesson 11 →
▸ Go deeper — how agents decide “done” optional

Confidence, but calibrated

“Stop when confident” only works if the agent's confidence tracks reality. A model sure of a shaky answer will stop too soon; one that never feels sure will never stop. This is why the confidence stop is almost always paired with a hard limit.

Hard caps as a safety net

A maximum number of turns, a token or dollar budget, a wall-clock timeout — these do not aim for a good answer, they guarantee the loop ends. When the primary stop fails, the cap catches it and hands back the best answer so far, or an honest “could not determine”.

Loops that circle

Without care, agents repeat themselves — calling the same tool with the same arguments, re-deriving the same dead end. Detecting no-progress (the state has not changed) is another stopping signal: if the last step taught it nothing, going again probably will not either.

Stop and ask

Sometimes the right stop is not a verdict but a question. A well-built agent can halt and hand back to a person — “I need the ball-tracking data I cannot access” — rather than fabricating an answer to satisfy its own loop. Knowing when to stop includes knowing when to ask.