anyBodyCanDoDL
Re:Yann LeCun, Geoffrey Hinton and Yoshua Bengio win Turing Award
on Wednesday March 27, 2019 @01:52PM (#19503754)
That's very ignorant to say that attention or convolution is anywhere close to the expressivity of RNNs(LSTM). Instead of reading a few cherry-picked results from a model with extremely tuned hyperparameters, pick any random set of tasks and experiment yourself. Based on my experience, LSTM always performs better than all unless you brutely search over hundreds of hyperparameters configs.