News for hackers. Stuff that matters.
Re:Show HN: Sense2vec model trained on all 2015 Reddit comments by syllogism on Tuesday February 16, 2016 @03:45AM (#11108610)
Let's say we want to know how usage differs in one subreddit vs another. If you just train two entirely separate models, you end up with two vector spaces. The meanings within each space are entirely relative --- the absolute positions obviously aren't significant. You can try to learn a mapping, but the transform is not necessarily linear. (Interesting empirical question there...)

What you need is a 'seam' that connects the two vector spaces. I would do it like this.

Train a single model, with the words decorated by their subreddit. So you have combat:/r/gaming and combat:/r/history as different tokens. Then you have shared tokens which aren't decorated in this way.

There are no comments on this comment yet.