Re:How do neural networks learn?
by QuadmasterXLII on Tuesday March 19, 2024 @06:27AM
(#39754363)
If a paper provided general tools that let me sit down with pen and paper and derive that SWIGLU is better than RELU, and batch normalization trains 6 pi /root 5 times faster than no normalization, etc, and I could get the derivation right before running the experiment, then I wpuld believe that I understood how neural networks train
[ Parent
| on HN ]
Re:How do neural networks learn?
by SiempreViernes on Tuesday March 19, 2024 @06:34AM
(#39754401)
Pretty much this, so long as you can't make any prediction ahead of time of the approximate performance of a architecture on a dataset one shouldn't claim they have a good understanding of how a network "learns".
This doesn't make sense as a requirement - you make no mention of the data. "How it works" will always depend, in part, on the data, you can't fully separate the algorithm from the data if you want to answer questions about why learning works the way it does. IMO.