Re:Multidirectional joint distribution neurons reducing to KAN
by jarekd on Saturday May 11, 2024 @10:40AM
(#40328386)
Which benchmarks for multidirectional neurons? To compare with which approaches?
Multidirectional are biological neurons, but I don't know how to compare with them?
Re:Multidirectional joint distribution neurons reducing to KAN
by cs702 on Saturday May 11, 2024 @10:49AM
(#40328423)
Can you show the world this can be made to work for, say, a toy benchmark like MNIST classification?
---
To be 100% clear: My question about practical application today is orthogonal to the question about whether this research is worth pursuing!
Re:Multidirectional joint distribution neurons reducing to KAN
by jarekd on Saturday May 11, 2024 @11:17AM
(#40328608)
(Multidirectional) biological neural networks are no longer superior in MNIST benchmark ... but e.g. consciousness, or being able to learn from single examples.
And no, recreating it is not a task a single person can complete.
Re:Multidirectional joint distribution neurons reducing to KAN
by cs702 on Saturday May 11, 2024 @01:23PM
(#40329436)
Alright. I've added you preprint to my reading list, so I can take a closer look at this.
Re:Multidirectional joint distribution neurons reducing to KAN
by jarekd on Saturday May 11, 2024 @03:28PM
(#40330115)
Just represent joint density for each neuron as a linear combination - then you can inexpensively propagate in both directions e.g. as E[X|Y,Z] or E[Y,Z|X] by substituting and normalizing ... the formulas turn out quite simple - could be hidden in dynamics of (bidirectional) biological NN ...
And for pairwise distribution becomes ~KAN, which turned out quit successful ... so we are talking about its extension: adding more possibilities, like triplewise dependencies and multidirectional propagation.
There are a lot of ideas that are clever and seem promising... but fail to perform well on such benchmarks.
Is there a github repo with code available?