Re:Show HN: Sense2vec model trained on all 2015 Reddit comments
by syllogism on Tuesday February 16, 2016 @03:45AM
(#11108610)
Let's say we want to know how usage differs in one subreddit vs another. If you just train two entirely separate models, you end up with two vector spaces. The meanings within each space are entirely relative --- the absolute positions obviously aren't significant. You can try to learn a mapping, but the transform is not necessarily linear. (Interesting empirical question there...)
What you need is a 'seam' that connects the two vector spaces. I would do it like this.
Train a single model, with the words decorated by their subreddit. So you have combat:/r/gaming and combat:/r/history as different tokens. Then you have shared tokens which aren't decorated in this way.
Re:Show HN: Sense2vec model trained on all 2015 Reddit comments
by ppod on Tuesday February 16, 2016 @07:57AM
(#11109505)
There's a fantastic blog post that does this with male and female ratemyprof reviews:
http://bookworm.benschmidt.org/posts/2015-10-30-rejecting-th...
If you also trained a model on a different news aggregator's comments, would it be possible to "match up" the meaning vector spaces and see differences in what meanings each community ascribes to words?
Additionally, could you determine sentiments from the positions of the words in the meaning space? One example, looking for underlying assumptions, like associations of the words 'black person' and 'criminal'. Or idk, man + sex = player, but woman + sex = slut.
Would it be possible to go higher level and see how much a corpus agrees with "free markets are good" based on its word positions?
It seems like word2vec has the potential to bring Sapir-Whorf to a whole new level.