News for hackers. Stuff that matters.
Re:Building your own deep learning computer is 10x cheaper t... by p1esk on Wednesday September 26, 2018 @01:01PM (#18077967)
22,000-wide output sourced by a 4096-wide embedding

You will want to use hierarchical outputs in this case. Take a look at Hinton's 'Knowledge Distillation' paper.

Re:Building your own deep learning computer is 10x cheaper t... by oneshot908 on Friday September 28, 2018 @04:47PM (#18096928)
Sure, that's a nice approximation, and it will reduce performance somewhat IMO, it'd be nice to quantify how much, no?