News for hackers. Stuff that matters.
Re:People tricking ChatGPT “like watching an Asimov novel co... by visarga on Friday December 02, 2022 @02:48PM (#33835854)
> matrix multiplication to Relus (just a max(x,0))

Transformers famously employ the Softmax activation inside the attention matrix. Very rare to see Softmax anywhere other than the final layer.

There are no comments on this comment yet.