Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on 3Blue1Brown: Visualizing Attention, a Transformer's Heart. Parent comment
I thought the general consesus was: "transformers allow neural networks to have adaptive weights".
As opposed to the previous architectures, were every edge connecting two neurons always has the same weight.
EDIT: a good video, where it's actually explained better: https://youtu.be/OFS90-FX6pg?t=750&si=A_HrX1P3TEfFvLay
There are no comments on this comment yet.
I thought the general consesus was: "transformers allow neural networks to have adaptive weights".
As opposed to the previous architectures, were every edge connecting two neurons always has the same weight.
EDIT: a good video, where it's actually explained better: https://youtu.be/OFS90-FX6pg?t=750&si=A_HrX1P3TEfFvLay