News for hackers. Stuff that matters.
Re:Single Headed Attention RNN by 1maginary on Wednesday November 27, 2019 @09:41AM (#21648169)
You just have to love Stephen Merity.

His work on QRNN's saved me quite a bit of time and money when I was doing my undergrad dissertation on language models.

This SHA-RNN seems to have surfaced from a similar line of thinking that spawned the QRNN.

Re:Single Headed Attention RNN by technics256 on Wednesday November 27, 2019 @11:25AM (#21649223)
Are qRNNs still used much?
Re:Single Headed Attention RNN by polymorph1sm on Wednesday November 27, 2019 @10:24PM (#21654368)
check out MultiFiT [0] from fastai, it uses QRNN for speed.

[0] http://nlp.fast.ai/classification/2019/09/10/multifit.html