The Improviser · Lesson 7 — The Same Step ← Course

The same step — meaning has directions.

Embeddings don't just cluster similar things. Their directions carry meaning too — so you can do arithmetic with them.

On the last page, close meant similar — two takes that mean the same thing land near each other. There's a second, deeper thing going on: the directions in the space mean things too. Move a fixed distance in a fixed direction and you make the same change in meaning every time — and because these are just numbers, you can add and subtract those moves.

The famous demonstration was king − man + woman ≈ queen. In music the same trick reads saxophone − jazz + blues ≈ electric guitar. Here's exactly why it works.

Give every word a tiny position built from just two numbers — two axes. The x-axis (across) says which genre it belongs to: jazz, blues, bluegrass… spread left to right. The y-axis (up) says what kind of word it is — a bare style name like jazz or blues sits at the bottom (0), and that style's signature instrument sits at the top (10). So “jazz”, just the name, is at 0, while the saxophone that defines it is at 10. Call that y-value the iconic-lead score, and every word becomes one point at (x, y) = (genre, iconic-lead).

The same holds for every style. On the plot, blues sits further along the x-axis, on the floor at 0; the electric guitar — blues's signature instrument — sits directly above it at 10. Bluegrass has its banjo overhead, reggae its organ, and so on: each style on the floor, its defining instrument straight up above it.

Now the sum. Starting at the saxophone, − jazz then + blues changes only the x-value (the genre) — it slides you sideways, out of jazz's column and into blues's — while the y-value (iconic-lead) stays pinned at 10. Land there and you're at the point sitting above blues at height 10: its signature lead, the electric guitar.

The point isn't the toy numbers. It's that an instrument is not a separate axis — every word is one point, and “the signature lead of a genre” is a single direction (straight up the iconic-lead axis) that points the same way from every genre. Follow it from anywhere and the arithmetic lands.

Pick a genre and watch the same step — saxophone − jazz + that genre — land on its own signature lead.

The step from jazz to another genre is the same step from saxophone to that genre's signature lead.

Add and subtract the vectors, take the nearest.

The analogy is literally vector arithmetic on the embeddings, then a nearest-neighbour lookup — the same “find the closest point” from the last page.


      

Relationships are directions.

Closeness told us two things mean the same. Directions tell us how two things differ — and because the differences are just vectors, they add and subtract. “The signature lead of a genre” is one such direction, and it points the same way whichever genre you start from.

Meaning laid out as geometry: near points are alike, and consistent relationships become consistent steps you can do arithmetic with. That is the real reason a search by meaning works so well.

Next: cutting the tape →
Go deeper — how literally to take this optional

The axes aren't really labelled

The clean “x = genre, y = signature” split is a teaching simplification. A real embedding has hundreds of numbers, and no single one means “genre” or “signature” — those ideas are smeared across many dimensions at once. What is genuine is that consistent directions exist in the space, even though they don't line up with individual axes. The 2D picture just flattens all that so the step is visible.

It's a tendency, not a law

The examples are cherry-picked. Two instruments are never at exactly the same “signature-ness”, so saxophone − jazz + blues lands near electric guitar, not dead on it — and “electric guitar” is simply the closest real point to that spot. That is why it is written “≈”, and why you take the nearest neighbour rather than an exact hit.

Word vectors vs passage vectors

This tidy arithmetic is clearest with word embeddings — one vector per word, the classic king/queen result. The passage embeddings RAG actually uses place whole sentences in the space, where the property that does the real work is plain “near = similar”, not neat analogies. The analogy is the intuition; retrieval leans on the closeness.