News for hackers. Stuff that matters.

A comment on 3Blue1Brown: Visualizing Attention, a Transformer's Heart. Parent comment

Re:3Blue1Brown: Visualizing Attention, a Transformer's Heart by mjburgess on Monday April 15, 2024 @06:27AM (#40038661)
> they are lot more scalable context length wise

Sure, we're agreeing. I'm just being less specific.

[ Parent | on HN ]
Re:3Blue1Brown: Visualizing Attention, a Transformer's Heart by YetAnotherNick on Monday April 15, 2024 @07:51AM (#40039279)
Scalable as in loss wise scalable, not compute wise.
[ Parent | on HN ]