Re:Ultrafast machine learning on FPGAs via Kolmogorov-Arnold...
by zipy124 on Wednesday June 10, 2026 @05:46AM
(#48473871)
Can I just say that this is extremely impressive work for a master's level thesis. Incredible work and I hope you manage to continue fulfilling your fantastic potential in your career!
Regarding your point that "90% of the benefit of KANs can be gained from a small variety of function shapes": even within the B-spline basis, the shapes are quite uniform. Much of the actual benefit of scaling up the basis size comes from learning more complex, piecewise-polynomial activation functions. Scaling up the number of basis functions (i.e. more granular intervals) also increases locality and allows the activation function's value across different parts of the domain to be learned semi-independently. (There obviously is a tradeoff here with overfitting.)
The number of basis functions (G+S) is largely what determines how expressive the activation is, as it relates to your point: "you could have a representation that scales from a standard relu perceptron though KANs to something with weighted inputs and fancy weighted activation functions."