Re:Techniques for Training Large Neural Networks
by kristjansson on Thursday June 09, 2022 @01:59PM
(#31684586)
> starting
This is a strange comment to see. The techniques in TFA have been at the heart of a the last few years of progress in large-scale language models, image generation, etc?
Re:Techniques for Training Large Neural Networks
by rg111 on Friday June 10, 2022 @05:14AM
(#31691772)
Dude, parallelization is exactly how non-trivial neural networks are trained since the beginning.
Parallel processing as in SIMD architecture is what makes Deep Learning possible now.
Parallel computing as in stacking graphics card is how literally everything is done everywhere- from big tech companies to not-so-well-funded data science departments in unknown unis.
You are so so off.
Re:Techniques for Training Large Neural Networks
by ShamelessC on Thursday June 09, 2022 @04:36PM
(#31686506)
Strange comment. May need to edit something? I can't parse it - parallel (e.g. CUDA) techniques have been popular since AlexNet.
I hope we start seeing more massively parallel training strategies (most likely with GPUs under the hood still)