News for hackers. Stuff that matters.
Re:Understanding word vectors by RoboTeddy on Friday March 02, 2018 @02:43PM (#16504933)
> we needed first to find the right sense for each word in a phrase and only then assign the vector

Interesting! I wonder if you could you e.g. arbitrarily split a word into some number of symbols, e.g. two, and each time you're going to apply a training update, only apply the update to one of the symbols -- perhaps initially choosing the symbol facing the greatest loss (forcing the symbols apart in vector space), and then eventually switching over to picking the symbol with the smallest loss (letting each settle onto its own precise meaning)?

There are no comments on this comment yet.