Show HN: Word2Bits – Quantized Word Vectors

Re:Show HN: Word2Bits – Quantized Word Vectors by maxlam on Tuesday March 20, 2018 @04:05AM (#16626720)

Interesting, definitely need to think about debiasing. Seems like it won't really work straight out of the box since it'd destroy the 1 bit-ness of the vectors.

Though if only a few of the vectors are de-biased then you can still save a lot of space since all the other vectors are still represented using 2 numbers (while the de-biased vectors are represented using the full range of 32 bit numbers).

[ Parent | on HN ]

Re:Show HN: Word2Bits – Quantized Word Vectors by rspeer on Tuesday March 20, 2018 @04:10AM (#16626739)

There may be a way to build it into the loss function so that it happens before the quantization, right?

(and holy crap, look how fast the HN conservatives are getting to my comment)

[ Parent | on HN ]

Re:Show HN: Word2Bits – Quantized Word Vectors by hnuser1234 on Tuesday March 20, 2018 @05:33AM (#16627040)

Literally everything is either Left or Right. Anyone who disagrees with you is definitely a sexist conservative. No other possibility. The center is a black hole.

[ Parent | on HN ]

Re:Show HN: Word2Bits – Quantized Word Vectors by maxlam on Tuesday March 20, 2018 @04:17AM (#16626763)

Hm not sure, would need to think more about this -- definitely an interesting idea though!

[ Parent | on HN ]

Re:Show HN: Word2Bits – Quantized Word Vectors by rspeer on Tuesday March 20, 2018 @04:26AM (#16626789)

Thanks for being willing to discuss it. And sorry if I'm complicating your task.

But any interesting release of NLP data has the potential to affect the way the field progresses, so take it as a compliment that I consider this an interesting release of NLP data. That's why I'm asking you to actively consider the downstream effects of word vectors and find out if you can make them better.

[ Parent | on HN ]

Re:Show HN: Word2Bits – Quantized Word Vectors by maxlam on Tuesday March 20, 2018 @04:36AM (#16626834)

Thanks for the compliments! And thanks also for bringing up the debiasing! It's interesting to see what people care about and any conversation that leads to new ideas is a plus.

[ Parent | on HN ]