News for hackers. Stuff that matters.

A comment on Why the deep learning boom caught almost everyone by surprise. Parent comment

Re:Why the deep learning boom caught almost everyone by surp... by scotty79 on Wednesday November 06, 2024 @10:27AM (#42063764)
Do transformer architecture and attention mechanisms actually give any benefit to anything else than scalability?

I though the main insights were embeddings, positional encoding and shortcuts through layers to improve back propagation.

[ Parent | on HN ]
Re:Why the deep learning boom caught almost everyone by surp... by valzam on Thursday November 07, 2024 @01:34AM (#42074043)
When it comes to ML there is no such distinction though. Bigger models == more capable models and for bigger models you need scalability of the algorithm. It's like asking if going to 2nm fabs has any benefit other than putting more transistors in a chip. It's the entire point.
[ Parent | on HN ]