Transformer: A Novel Neural Network Architecture for Language Understanding

Re:Transformer: A Novel Neural Network Architecture for Lang... by mykeliu on Thursday August 31, 2017 @08:24PM (#15145195)

I'm a novice when it comes to neural network models, but would I be correct in interpreting this as a convolutional network architecture with multiple stacked encoders and decoders?

[ Parent | on HN ]

Re:Transformer: A Novel Neural Network Architecture for Lang... by knowtheory on Thursday August 31, 2017 @08:28PM (#15145212)

Nah, the paper explicitly states that their system is not recurrent nor convolutional:

> To the best of our knowledge, however, the Transformer is the first transduction model relying entirely on self-attention to compute representations of its input and output without using RNNs or convolution.

[ Parent | on HN ]