News for hackers. Stuff that matters.
Re:3Blue1Brown: Visualizing Attention, a Transformer's Heart by blt on Monday April 15, 2024 @03:03PM (#40044424)
As a computer scientist, the "differentiable hash table" interpretation worked for me. The AIAYN paper alludes to it by using the query/key/value names, but doesn't explicitly say the words "hash table". I guess some other paper introduced them?

There are no comments on this comment yet.