News for hackers. Stuff that matters.
Re:Ask HN: Can someone ELI5 Transformers and the “Attention ... by visarga on Wednesday May 17, 2023 @03:12PM (#35979848)
It's not really doing hashing, which is random, but instead it makes meaningful connections, like connecting an adjective to its noun, or a pronoun to a previous reference of the name. That is why it is called "Attention is all you need" and not "Hashing is all you need". Attention includes some meaning, it is attentive to some aspect or another.
Re:Ask HN: Can someone ELI5 Transformers and the “Attention ... by throwawaymaths on Wednesday May 17, 2023 @04:41PM (#35981005)
Hashing does not have to be random, it just has to compress the domain. Modulo prime number is a perfectly valid hash for some use cases.