Re:The boundary of neural network trainability is fractal
by ks1723 on Tuesday February 20, 2024 @06:32PM
(#39448386)
Just to add to this, only the two learning rates are changed, everything else including initialization and data is fixed. From the paper:
Training consists of 500 (sometimes 1000) iterations of full batch steepest gradient descent. Training is performed for a 2d grid of η0 and η1 hyperparameter values, with all other hyperparameters held fixed (including network initialization and training data).
Training consists of 500 (sometimes 1000) iterations of full batch steepest gradient descent. Training is performed for a 2d grid of η0 and η1 hyperparameter values, with all other hyperparameters held fixed (including network initialization and training data).