News for hackers. Stuff that matters.

A comment on Why neural networks struggle with the Game of Life (2020). Parent comment

Re:Why neural networks struggle with the Game of Life (2020) by DrScientist on Friday May 17, 2024 @08:52AM (#40389349)
This may be a silly question but - so rather than train a big network and hope a subnetwork wins the lottery - why not just train a smaller network with multiple runs with different starting weights?
[ Parent | on HN ]
Re:Why neural networks struggle with the Game of Life (2020) by MalphasWats on Friday May 17, 2024 @09:08AM (#40389487)
Largely for the same reason people don't "just win" the lottery by buying every possible ticket (any more).
[ Parent | on HN ]
Re:Why neural networks struggle with the Game of Life (2020) by scarmig on Friday May 17, 2024 @06:23PM (#40394705)
The larger network contains exponentially more subnetworks. 10x the size contains far more than 10x subnetworks (although it'd also take more than 10x as long to train).
[ Parent | on HN ]
Re:Why neural networks struggle with the Game of Life (2020) by DrScientist on Tuesday May 21, 2024 @01:14PM (#40431149)
Ah ok - so you are saying the explosion of subnetworks is higher than the explosion of training time - leading to a win.

In this case a positive benefit of combinatorial complexity.

[ Parent | on HN ]