Re:Multidirectional joint distribution neurons reducing to KAN
by cs702 on Saturday May 11, 2024 @10:37AM
(#40328367)
When I ask if this has been tested, I meant as an ANN on conventional benchmarks. Sorry if that wasn't clear.
There are a lot of ideas that are clever and seem promising... but fail to perform well on such benchmarks.
Is there a github repo with code available?
Re:Multidirectional joint distribution neurons reducing to KAN
by jarekd on Saturday May 11, 2024 @10:40AM
(#40328386)
Which benchmarks for multidirectional neurons? To compare with which approaches?
Multidirectional are biological neurons, but I don't know how to compare with them?
Re:Multidirectional joint distribution neurons reducing to KAN
by cs702 on Saturday May 11, 2024 @10:49AM
(#40328423)
Can you show the world this can be made to work for, say, a toy benchmark like MNIST classification?
---
To be 100% clear: My question about practical application today is orthogonal to the question about whether this research is worth pursuing!
Re:Multidirectional joint distribution neurons reducing to KAN
by jarekd on Saturday May 11, 2024 @11:17AM
(#40328608)
(Multidirectional) biological neural networks are no longer superior in MNIST benchmark ... but e.g. consciousness, or being able to learn from single examples.
And no, recreating it is not a task a single person can complete.
Re:Multidirectional joint distribution neurons reducing to KAN
by cs702 on Saturday May 11, 2024 @01:23PM
(#40329436)
Alright. I've added you preprint to my reading list, so I can take a closer look at this.
Re:Multidirectional joint distribution neurons reducing to KAN
by jarekd on Saturday May 11, 2024 @03:28PM
(#40330115)
Just represent joint density for each neuron as a linear combination - then you can inexpensively propagate in both directions e.g. as E[X|Y,Z] or E[Y,Z|X] by substituting and normalizing ... the formulas turn out quite simple - could be hidden in dynamics of (bidirectional) biological NN ...
And for pairwise distribution becomes ~KAN, which turned out quit successful ... so we are talking about its extension: adding more possibilities, like triplewise dependencies and multidirectional propagation.
However, it degenerates to ~KAN if restring to pairwise dependencies (can consciously add triplewise and higher), and gives many new possibilities, like multidirectional propagation, of values or probability distributions, with novel additional training approaches like through tensor decomposition.