Deep Learning in JavaScript

Re:Deep Learning in JavaScript by GaggiX on Thursday March 28, 2024 @07:36PM (#39858779)

The nn.Block should probably be renamed to nn.DecoderBlock as it's not very clear if it's supposed to be an encoder or a decoder block, also an option to disable the mask from attention. That said, very cool project.

[ Parent | on HN ]

Re:Deep Learning in JavaScript by eduardoleao052 on Thursday March 28, 2024 @07:47PM (#39858875)

You are absolutely right! Will solve that as soon as possible. Thanks for the feedback!

[ Parent | on HN ]

Re:Deep Learning in JavaScript by jsight on Friday March 29, 2024 @11:25AM (#39865209)

If you have the option to disable the mask, isn't it then a generic nn.Block?

[ Parent | on HN ]

Re:Deep Learning in JavaScript by GaggiX on Friday March 29, 2024 @12:56PM (#39866286)

The ability to disable the mask in nn.MultiHeadSelfAttention, then having nn.DecoderBlock and a nn.EncoderBlock.

[ Parent | on HN ]

Re:Deep Learning in JavaScript by eduardoleao052 on Friday March 29, 2024 @02:07PM (#39867149)

I think that’s it. I’ll probably add that soon

[ Parent | on HN ]

Re:Deep Learning in JavaScript by eduardoleao052 on Friday March 29, 2024 @11:43AM (#39865425)

It would be with some simple tweaks. For instance, the current block does not support Cross-Attention, just Self-Attention.

[ Parent | on HN ]