The Improviser · Lesson 6 — The Phantom Note ← Course

The phantom note — hallucination.

A soloist who never heard the change still names a chord, and sounds sure of it. When a model does the same, we call it a hallucination: fluent, confident, and wrong.

A hallucination is when the model states something false as if it were fact — confidently, fluently, often with convincing detail. It isn't lying; it doesn't know it is wrong. It simply played the notes its training made most likely.

Why does it happen? Recall Lesson 1: the model only ever picks the next note, and it always produces something that flows. It has no reliable internal alarm for "I don't know," and no default way to voice one. Unlike a database, which can answer "no match found," a language model has no empty-handed option — so when the real answer is missing, the most natural-sounding continuation is often a natural-sounding invention. Gaps are the commonest cause, but a leading question can pull one out of a model that "knew" better.

Picture a soloist called on to name the chord under a bar he's never seen. He didn't hear the change, but his ear fires anyway and he names one, believing it — no deceit, just a reflex filling a silence. That is the phantom note. The model has even less choice: fluent continuation is all it makes, so a gap comes out as a confident wrong note rather than an honest "I couldn't tell."

The fixes aren't about trusting the model harder. You can (1) give it permission to say "I'm not sure," and (2) — the big one — put the actual chart in front of it to read from, instead of asking it from memory. Lowering temperature (Lesson 5) trims the odd unlucky token, but not a confident fabrication; grounding it in the chart is the real lever.

Ask the soloist a chord it can't know, hear it play a confident wrong one — then add the safeguards one by one and watch the phantom note resolve.

No safeguards yet. Listen to the soloist name a chord it was never shown.
you ask the soloist
safeguards · tap to add or remove
↓
the soloist answers

The safeguards are just how you call it.

Each safeguard is a change to the request, not to the model. The biggest one — handing over the chart — is a whole technique of its own, coming next. The chart goes where everything else goes: into the context window from Lesson 2, paying the same budget. 


      

    


    

      
It cannot not play — so a gap comes out as a note.

      

        A model always produces music that flows, so a gap in its knowledge — or a leading question — comes out as
        a confident, plausible fabrication — not a rest and not a warning. The tone tells
        you nothing about whether it is true — and neither does the model's own confidence:
        a made-up chord can carry just as high a probability as a real one.
      

      

        The fix is in the setup, not in trusting harder: let it abstain, and put the
        real chart in front of it to read from (retrieval — the next lesson). And for
        anything that matters, verify.
      

      

        Next: why it happens →
        
      

    


  


  

    

      ▸ Go deeper — why it happens and what actually helps optional
      

        
"Confident" tells you nothing

        

          The fluency and certainty in the phrasing are generated the same way whether the
          model knows the answer or is inventing it — there is no honesty dial that leaks
          into the notes. Even the model's internal probability can be miscalibrated, so a
          made-up chord can ring out every bit as assured as a correct one. Never treat
          tone as evidence.
        

        
Grounding is the biggest lever

        

          The most reliable systems don't ask the model to recall facts from memory at all.
          They fetch the relevant pages and have the model answer from those, which
          is the whole of the next lesson. A model reading off
          the chart hallucinates far less than one recalling from training.
        

        
Give it an exit

        

          Simply telling the model it is allowed to say "I don't know" measurably cuts down
          invented answers. Models often fabricate because the prompt implicitly demands an
          answer and leaves no acceptable way to decline. Offer one.
        

        
It can still be wrong — so verify

        

          Neither of these makes a model perfect; they lower the rate, not the possibility. For
          anything that matters, check the source yourself. Measuring how often a model
          goes wrong, and on what, is its own discipline, and a later lesson of its own.