News for hackers. Stuff that matters.

A comment on Ask HN: Can someone ELI5 Transformers and the “Attention is all we need” paper. Parent comment

Re:Ask HN: Can someone ELI5 Transformers and the “Attention ... by saynay on Wednesday May 17, 2023 @02:41PM (#35979472)
I believe the other big thing is it allowed you to parallelize the training as well, so you could split those colossal training sets across many machines in a way you could not do with LSTMs.
[ Parent | on HN ]
Re:Ask HN: Can someone ELI5 Transformers and the “Attention ... by throwawaymaths on Wednesday May 17, 2023 @02:50PM (#35979564)
I don't really believe this because in theory it's possible to split up training of arbitrary neural nets using decoupled neural interfaces (not that it's easy mind you) but at the point where you have openAI scale resources the bottleneck to achieving that is the difficulty of Python.
[ Parent | on HN ]
Re:Ask HN: Can someone ELI5 Transformers and the “Attention ... by golly_ned on Thursday May 18, 2023 @01:17AM (#35984357)
What’s a neural interface?
[ Parent | on HN ]
Re:Ask HN: Can someone ELI5 Transformers and the “Attention ... by throwawaymaths on Thursday May 18, 2023 @07:42AM (#35986605)
https://arxiv.org/abs/1608.05343
[ Parent | on HN ]