News for hackers. Stuff that matters.
Re:Ask HN: Can someone ELI5 Transformers and the “Attention ... by throwawaymaths on Wednesday May 17, 2023 @02:58PM (#35979662)
Correct, but note that if you subject a standard hash table algo to AD it won't magically become a transformer. (Hashes in the "normal construction" are discrete functions and thus aren't really continuous or differentiable, neither are lookup tables)

There are no comments on this comment yet.