News for hackers. Stuff that matters.
Re:Ask HN: Can someone ELI5 Transformers and the “Attention ... by Sai_ on Wednesday May 17, 2023 @02:07PM (#35979057)
You must be from a planet with very long years!

There is no way I can even begin to digest what you have said in your comment.

Re:Ask HN: Can someone ELI5 Transformers and the “Attention ... by nil-sec on Wednesday May 17, 2023 @02:52PM (#35979587)
Sorry maybe I should have added more explanation. One way to think about attention, which is the main distinguishing element in a transformer, is as an adaptable matrix. A feedforward layer is a matrix with static entries that do not change at inference time (only during training). The attention mechanism offers a way to have adaptable weight matrices at inference time (this is implemented by using three different matrices, K,Q & V called keys query and value in case you want to dig deeper).