News for hackers. Stuff that matters.
Re:King – man and woman is queen; but why? by lblock on Tuesday January 20, 2026 @07:01AM (#46690925)
Isn't the whole linear operations thing not really true and even less so now in embeddings from transformer based models? I remember reading a blog about this but cannot find it anymore. This is the closest thing i could find now: https://mikexcohen.substack.com/p/king-man-woman-queen-is-fa...
Re:King – man and woman is queen; but why? by wongarsu on Tuesday January 20, 2026 @08:06AM (#46691422)
I just tried it on qwen3-embedding:8b with a little vibe-coded 100 line script that does the obvious linear math and compares the result to the embeddings of a couple of candidate words using cosine similarity, and it worked exactly as expected. Same 22 candidates for both questions

king - man + woman ≈ queen (0.8510)

   Top similarity
   0.8510   queen
   0.8025   king
   0.7674   princess
   0.7424   woman
   0.7212   queen Elizabeth
Berlin - Germany + France ≈ Paris (0.8786)

    Top similarity
    0.8786   Paris
    0.8309   Berlin
    0.8057   France
    0.7824   London
Sure, 0.85 is not an exact match, and if I dump an entire dictionary in there it might be worse, but the idea very much works
Re:King – man and woman is queen; but why? by mjhay on Tuesday January 20, 2026 @07:54AM (#46691321)
Very much so. It was a little less untrue with older word embedding models* but that kind of semantic linearity never was a thing in practice. Word embedding models try to embed semantically similar words close to each other, but that does not imply linearity at all.

*with transformer models, it is pretty much not even wrong.