GPUs for Deep Learning in 2023 – An In-depth Analysis

Re:GPUs for Deep Learning in 2023 – An In-depth Analysis by tambre on Wednesday January 18, 2023 @03:42PM (#34432795)

I would be surprised if it did. But you probably shouldn't do professional work on GPUs that lack ECC memory.

[ Parent | on HN ]

Re:GPUs for Deep Learning in 2023 – An In-depth Analysis by nl on Wednesday January 18, 2023 @06:03PM (#34434384)

The lack of ECC memory is almost certainly not a factor. If you can train at FP8 your model will recover from a single flipped bit somewhere.

[ Parent | on HN ]

Re:GPUs for Deep Learning in 2023 – An In-depth Analysis by Loranubi on Wednesday January 18, 2023 @10:50PM (#34436494)

I mean you could even view bit flips as a regularization technique like dropout...

[ Parent | on HN ]

Re:GPUs for Deep Learning in 2023 – An In-depth Analysis by dahart on Thursday January 19, 2023 @12:31AM (#34437040)

Yeah I hear it’s common practice now to avoid synchronizing GPU training kernels in order to speed things up, and it has positive regularization benefits and little downside.

[ Parent | on HN ]