The Third Umpire · Lesson 4 — Bad Calls ← Course

Bad calls — garbage in, garbage out.

The model copies your examples faithfully — including their mistakes and their mess.

Examples work precisely because the model imitates them. But imitation cuts both ways: it mirrors whatever you show it, and it never pauses to ask whether your examples were any good in the first place.

Show it examples in three different styles, and the answers come back in three different styles. Show it one example with a wrong verdict, and it will repeat that mistake on a fresh appeal without a flicker of doubt. The model is not judging your examples — it is following them.

This is the flip side of the whole lesson: garbage in, garbage out. Clean, consistent, correct examples give clean, consistent, correct answers. Sloppy or wrong ones give exactly that back.

Feed the umpire three kinds of examples — clean, messy, and one with a wrong call — and watch its verdict pick up whatever you put in.

Same appeal each time. Change the quality of the examples and watch the answer follow.
the examples you show
↓ same appeal: “a faint edge to the keeper” ↓
the umpire's call

It imitates — it doesn't audit.

The call is drawn straight from the shape of the examples. Nothing in the code checks that they are correct or consistent — that part is on you. 


      

    


    

      
Garbage in, garbage out.

      

        Examples are instructions by demonstration, and the model is a faithful mimic.
        Inconsistent examples make inconsistent answers; a single wrong example teaches a
        wrong rule the model will repeat with full confidence. It copies the flaw as
        happily as the pattern.
      

      

        Treat your examples like code you are shipping: keep them correct, keep them in
        one consistent shape, and read them twice. A few clean examples beat a pile of
        sloppy ones every time.
      

      

        Next: the maths →
        
      

    


  


  

    

      ▸ Go deeper — keeping examples clean optional
      

        
It imitates, it does not evaluate

        

          A model has no way to know that one of your examples is wrong. To it, every
          example is simply “this is how it is done.” It will reproduce a mistake with the
          same confidence as a correct call — often more, because the example made the
          wrong answer look normal.
        

        
One bad example poisons the pattern

        

          It only takes a single mislabelled example to bend the rule. When few-shot
          results go strange, the first thing to check is not the model or the wording —
          it is whether one of your examples is quietly wrong.
        

        
Consistency is reliability

        

          If you want answers in a fixed shape, every example must be in that exact shape:
          same fields, same order, same style. Any variation you show becomes variation you
          get back. Uniform examples are the cheapest reliability you can buy.
        

        
Review examples like code

        

          Because examples steer behaviour as strongly as instructions do, they deserve the
          same care: version them, review them, and test the prompt after changing them.
          This is where good prompting quietly turns into the discipline of checking your examples against the calls you already know are right — a habit the course leans on much later.