Re:King – man + woman is queen; but why?
by stephencanon on Saturday January 07, 2017 @05:57PM
(#13347176)
Right, I don't deny that you get something sort of vector-ish and some operations that make intuitive sense some of the time. That's cool enough on its own! There's no need to pretend that we've actually constructed a vector space. Just say that we've arranged words on a hypersphere with an inferred distance metric and call it a day.
(Also, by "scalar product" I meant "scalar multiplication"--the product of a scalar and a vector, not the dot product. It's pretty clear how to make some sense out of the dot product, but it's pretty hard to make consistent sense out of scalar multiplication. Apologies for being unclear.)
Re:King – man + woman is queen; but why?
by moultano on Saturday January 07, 2017 @09:56PM
(#13347988)
There isn't a word at every point in the space. Is that your objection to calling it a vector space? I think of every point in space as representing subtle gradiations of concepts, only some of which are instantiated by words.
For differences - see this section: http://p.migdal.pl/2017/01/06/king-man-woman-queen-why.html#....
I mention also that multiplying a word by a factor (for PMI compression) results in a word of similar meaning, just being more characteristic (bear in mind that for other models in can be related e.g. to word frequency or other properties).
But you are right, that there are some problems with linear structure. Some of them were brought to me be Omer Levy (a researcher in this subject). I think an article that summarises it the best (or rather: shows empirically that it does not always work as intended) is:
- Tal Linzen, Issues in evaluating semantic spaces using word analogies, https://arxiv.org/abs/1606.07736