Re:Single Headed Attention RNN
by technics256 on Wednesday November 27, 2019 @11:25AM
(#21649223)
Are qRNNs still used much?
Re:Single Headed Attention RNN
by polymorph1sm on Wednesday November 27, 2019 @10:24PM
(#21654368)
check out MultiFiT [0] from fastai, it uses QRNN for speed.
[0] http://nlp.fast.ai/classification/2019/09/10/multifit.html
His work on QRNN's saved me quite a bit of time and money when I was doing my undergrad dissertation on language models.
This SHA-RNN seems to have surfaced from a similar line of thinking that spawned the QRNN.