News for hackers. Stuff that matters.
Re:Techniques for Training Large Neural Networks by superb-owl on Thursday June 09, 2022 @01:41PM (#31684333)
I'm really excited to see folks starting to talk about parallelizing machine learning. The conversation has been dominated by GPU-friendly techniques - a classic example of "everything looks like a nail when you have a hammer".

I hope we start seeing more massively parallel training strategies (most likely with GPUs under the hood still)

Re:Techniques for Training Large Neural Networks by kristjansson on Thursday June 09, 2022 @01:59PM (#31684586)
> starting

This is a strange comment to see. The techniques in TFA have been at the heart of a the last few years of progress in large-scale language models, image generation, etc?

Re:Techniques for Training Large Neural Networks by rg111 on Friday June 10, 2022 @05:14AM (#31691772)
Dude, parallelization is exactly how non-trivial neural networks are trained since the beginning.

Parallel processing as in SIMD architecture is what makes Deep Learning possible now.

Parallel computing as in stacking graphics card is how literally everything is done everywhere- from big tech companies to not-so-well-funded data science departments in unknown unis.

You are so so off.

Re:Techniques for Training Large Neural Networks by ShamelessC on Thursday June 09, 2022 @04:36PM (#31686506)
Strange comment. May need to edit something? I can't parse it - parallel (e.g. CUDA) techniques have been popular since AlexNet.
Re:Techniques for Training Large Neural Networks by Tostino on Thursday June 09, 2022 @04:49PM (#31686666)
I parsed that as meaning something more like BOINC to distribute the work and piece together the final result.
Re:Techniques for Training Large Neural Networks by mr_toad on Thursday June 09, 2022 @07:30PM (#31688280)
Like TensorFlow or PyTorch?