News for hackers. Stuff that matters.
Re:AlphaFold2 at CASP14: “It feels like one’s child has left... by aftbit on Saturday December 12, 2020 @01:39PM (#25400394)
If computation is the bottleneck, one might imagine that crowdsourced computing like Folding@Home and BOINC could offer an opportunity for less resourced teams to compete.
Re:AlphaFold2 at CASP14: “It feels like one’s child has left... by rcxdude on Sunday December 13, 2020 @11:21AM (#25407606)
Such resources are only really good for a particular kind of distributed computing: mostly the kind which is easily verifiable, and easily broken down into small embarrassingly parallel chunks which need a lot of computation done relative to their size.

Training large machine models is almost the opposite of this: the datasets are huge and don't split up easily, and iterations (which touch the entire model and a chunk of the dataset) are relatively fast. Even with dedicated hardware a lot of the challenge in getting these systems to work effectively is in moving the data around between different compute nodes. Something like folding@home just will not function for such a workload.

Re:AlphaFold2 at CASP14: “It feels like one’s child has left... by rsfern on Saturday December 12, 2020 @03:02PM (#25401131)
While interesting, I think this model is only viable for a certain subset of computational workloads.

AF2 took like two weeks of computation on 200 TPUs to train. I’m not sure if they used a data parallel or model parallel distributed training approach, but I don’t imagine either would scale particularly well to a globally distributed network of worker machines in a folding@home analogue...