News for hackers. Stuff that matters.

A comment on Kolmogorov-Arnold Networks. Parent comment

Re:Kolmogorov-Arnold Networks by gautam5669 on Wednesday May 01, 2024 @01:58AM (#40219995)
This is the first thought came to my mind too.

Given its sparse, Will this be just replacement for MoE.

[ Parent | on HN ]
Re:Kolmogorov-Arnold Networks by samus on Thursday May 02, 2024 @10:02AM (#40236445)
MoE is mostly used to enable load balancing since it makes it possible to put experts on different GPUs. This isn't so easy to do with a monolithic, but sparse layer.
[ Parent | on HN ]