News for hackers. Stuff that matters.

A comment on Ask HN: Can someone ELI5 Transformers and the “Attention is all we need” paper. Parent comment

Re:Ask HN: Can someone ELI5 Transformers and the “Attention ... by oneearedrabbit on Wednesday May 17, 2023 @02:14PM (#35979163)
I think in your notation it should have been:

y=Wx_0

y=W(x)x_0

[ Parent | on HN ]
Re:Ask HN: Can someone ELI5 Transformers and the “Attention ... by nil-sec on Wednesday May 17, 2023 @02:30PM (#35979353)
I guess I was more thinking about self attention, so yes. The more general case is covered by your notation!
[ Parent | on HN ]