Re:LexVec, a word embedding model written in Go that outperf...
by IshKebab on Wednesday July 27, 2016 @12:07PM
(#12173982)
Has anyone done any work on handing words that have overloading meanings? Something like 'lead' has two really distinct uses. It's really multiple words that happened to be spelt the same.
[ Parent
| on HN ]
Re:LexVec, a word embedding model written in Go that outperf...
by glup on Wednesday July 27, 2016 @12:58PM
(#12174469)
Google "word sense induction" or "word sense disambiguation". Intuitively, distributional information of the same sort that is used to derive representations for different word types in W2V or LexVec is useful for distinguishing word senses. Two (noun) senses of lead, two senses of bat, etc. are pretty easy to distinguish on the basis of a bag of words (or syntactic features) around them. Other words are polysemous: they have multiple related senses (across the language, names for materials can be used as containers; animal name for the corresponding food--but with exceptions). For some high frequency words it's a crazy gradient combination of polysemy and homonymy: 'home' for can refer to 1) a place someone lives 2) the corresponding physical structure 3) where something resides (a more 'metaphorical' sense), among other things. Obviously an individual use of a word has a gradient relationship to these senses, and speakers differ regarding what they think the substructure is (polysemous or homonymous, hierarchical or not, etc.). I've been working in my PhD on a technique to figure this out, but people clearly use a lot of information that isn't available in language corpora alone (e.g. intuitive physics).
Re:LexVec, a word embedding model written in Go that outperf...
by rspeer on Wednesday July 27, 2016 @01:57PM
(#12175101)
It's tricky because we don't have good ground truth on what different word senses there are. (WordNet is not the final answer, especially as it separates every metaphorical use of the same word into its own sense.)
My experience is that you can distinguish word senses, but it seems the data isn't good enough to improve anything but a task that specifically evaluates that same vocabulary of word senses.
I see a sibling comment with link to spaCy's sense2vec, which uses the coarsest possible senses -- one sense for nouns, one sense for verbs, one sense for proper nouns, etc. It's a start.
Re:LexVec, a word embedding model written in Go that outperf...
by lmmlzxx on Thursday July 28, 2016 @12:57AM
(#12178392)
Actually "lead" has many, many more than two distinct senses.
Re:LexVec, a word embedding model written in Go that outperf...
by riyadparvez on Wednesday July 27, 2016 @01:43PM
(#12174934)
Well, there is Sense2Vec: https://github.com/spacy-io/sense2vec