The Third Umpire · Lesson 6 — The Phantom Call ← Course

The phantom call — hallucinations.

Sometimes the model hands you a fluent, confident answer that is pure invention. That is a hallucination — and a few changes to how you ask make it far rarer.

A hallucination is a confident answer to a question the model cannot actually answer — something false stated as fact, fluent and often dressed in convincing detail. It is not lying: the model does not know it is wrong. It simply produced the continuation its training made most likely.

Why does it happen? Recall Lesson 1: the model only ever predicts the next word, and it always produces something fluent. Crucially, it has no “no data” setting — no way to return a blank. When it lacks the real answer, the most natural-sounding continuation is often a natural-sounding invention. A gap is the commonest trigger, but a leading question or a strong pattern can coax a fabrication out of a model that “knew” better — and the confident tone is no signal, since it reads the same whether the model knows or is guessing.

Picture an umpire required to rule on every ball. He never saw the edge, but instinct fires and the finger goes up — no deceit, just a reflex making a call where there was no evidence. That is the phantom call. The model has even less choice: fluent text is the only thing it can produce, so a gap comes out as fluent invention rather than “I couldn't tell.”

The fixes are not about trusting harder. You can (1) give it permission to say “I don't know”, (2) hand it the actual evidence to read from instead of asking it to recall — the big one — and (3) ask it to cite a source you can check. Lowering temperature (Lesson 5) trims the odd unlucky token, but not a confident fabrication; grounding it in evidence is the biggest lever.

Ask the umpire something it cannot know, watch it invent an answer — then switch on the safeguards one by one and watch the phantom call give way.

A question with no record behind it.
Step 1 of 4
you ask the umpire
safeguards
✓allowed to say “not sure”
✓given the real record to read
✓must cite its source
↓
the umpire answers

The safeguards are just how you call it.

Each safeguard is a change to the request, not to the model. The biggest one — handing over the record — is a whole technique of its own, coming next. The record goes where everything else goes: into the context window from Lesson 2, paying the same budget. 


      

    


    

      
It cannot not answer — so a gap comes out as an answer.

      

        A model always produces fluent text, so a gap in its knowledge — or a leading
        question — surfaces as a confident, plausible fabrication, never a blank and never
        a warning. And its own certainty is no guide: a made-up answer can carry just as
        high a probability as a true one.
      

      

        The fix is in the setup, not in trusting harder: let it abstain, hand it the real
        evidence to read from (retrieval — the next lesson), and ask it to cite so you can
        check. And for anything that matters, verify.
      

      

        Next: the footage bank →
        
      

    


  


  

    

      ▸ Go deeper — why it happens and what actually helps optional
      

        
“Confident” tells you nothing

        

          The fluency and certainty in the wording are generated the same way whether the
          model knows the answer or is inventing it — there is no honesty dial that leaks
          into the prose. Even the model's internal probability can be miscalibrated, so a
          made-up answer can read every bit as assured as a correct one. Never treat tone
          as evidence.
        

        
Grounding is the biggest lever

        

          The most reliable systems don't ask the model to recall facts from memory at all.
          They fetch the relevant documents and have the model answer from those, which is the whole of the next lesson. A model reading off
          the record hallucinates far less than one recalling from training.
        

        
Give it an exit

        

          Simply telling the model it is allowed to say “I don't know” measurably cuts down
          invented answers. Models often fabricate because the prompt implicitly demands an
          answer and leaves no acceptable way to decline. Offer one.
        

        
It can still be wrong — so verify

        

          None of these make a model perfect; they lower the rate, not the possibility. For
          anything that matters, check the cited source yourself. A citation can be invented too; its value is that a fake one is easy to catch. Measuring how often a model
          goes wrong, and on what, is its own discipline, and a later lesson of its own.