News for hackers. Stuff that matters.

A comment on What Are Transformer Models and How Do They Work?. Parent comment

Re:What Are Transformer Models and How Do They Work? by riffraff on Saturday April 15, 2023 @01:50AM (#35577963)
Isn't "adding" the same as "moving closer" ?

E.g. the vector for "bank" is mid-way between the geographical and financial meaning, "bank + money" is closer while "bank + river" if further away.

[ Parent | on HN ]
Re:What Are Transformer Models and How Do They Work? by jaidhyani on Sunday April 16, 2023 @12:48PM (#35591663)
It depends on the values of the vectors. (4, 4) + (3, 3) results in a new vector (7, 7) which is further away from both contributing vectors than either one was to each other originally. Additionally, negative coefficients are a thing.
[ Parent | on HN ]