News for hackers. Stuff that matters.

A comment on Deep Learning in JavaScript. Parent comment

Re:Deep Learning in JavaScript by jsight on Friday March 29, 2024 @11:25AM (#39865209)
If you have the option to disable the mask, isn't it then a generic nn.Block?
[ Parent | on HN ]
Re:Deep Learning in JavaScript by GaggiX on Friday March 29, 2024 @12:56PM (#39866286)
The ability to disable the mask in nn.MultiHeadSelfAttention, then having nn.DecoderBlock and a nn.EncoderBlock.
[ Parent | on HN ]
Re:Deep Learning in JavaScript by eduardoleao052 on Friday March 29, 2024 @02:07PM (#39867149)
I think that’s it. I’ll probably add that soon
[ Parent | on HN ]
Re:Deep Learning in JavaScript by eduardoleao052 on Friday March 29, 2024 @11:43AM (#39865425)
It would be with some simple tweaks. For instance, the current block does not support Cross-Attention, just Self-Attention.
[ Parent | on HN ]