Re:King – man and woman is queen; but why?
by lblock on Tuesday January 20, 2026 @07:01AM
(#46690925)
Isn't the whole linear operations thing not really true and even less so now in embeddings from transformer based models?
I remember reading a blog about this but cannot find it anymore. This is the closest thing i could find now:
https://mikexcohen.substack.com/p/king-man-woman-queen-is-fa...
[ Parent
| on HN ]
Re:King – man and woman is queen; but why?
by wongarsu on Tuesday January 20, 2026 @08:06AM
(#46691422)
I just tried it on qwen3-embedding:8b with a little vibe-coded 100 line script that does the obvious linear math and compares the result to the embeddings of a couple of candidate words using cosine similarity, and it worked exactly as expected. Same 22 candidates for both questions
king - man + woman ≈ queen (0.8510)
Top similarity
0.8510 queen
0.8025 king
0.7674 princess
0.7424 woman
0.7212 queen Elizabeth
Berlin - Germany + France ≈ Paris (0.8786) Top similarity
0.8786 Paris
0.8309 Berlin
0.8057 France
0.7824 London
Sure, 0.85 is not an exact match, and if I dump an entire dictionary in there it might be worse, but the idea very much works
Re:King – man and woman is queen; but why?
by mjhay on Tuesday January 20, 2026 @07:54AM
(#46691321)
Very much so. It was a little less untrue with older word embedding models* but that kind of semantic linearity never was a thing in practice. Word embedding models try to embed semantically similar words close to each other, but that does not imply linearity at all.
*with transformer models, it is pretty much not even wrong.