Re:AlphaFold2 at CASP14: “It feels like one’s child has left...
by aftbit on Saturday December 12, 2020 @01:39PM
(#25400394)
If computation is the bottleneck, one might imagine that crowdsourced computing like Folding@Home and BOINC could offer an opportunity for less resourced teams to compete.
Re:AlphaFold2 at CASP14: “It feels like one’s child has left...
by rcxdude on Sunday December 13, 2020 @11:21AM
(#25407606)
Such resources are only really good for a particular kind of distributed computing: mostly the kind which is easily verifiable, and easily broken down into small embarrassingly parallel chunks which need a lot of computation done relative to their size.
Training large machine models is almost the opposite of this: the datasets are huge and don't split up easily, and iterations (which touch the entire model and a chunk of the dataset) are relatively fast. Even with dedicated hardware a lot of the challenge in getting these systems to work effectively is in moving the data around between different compute nodes. Something like folding@home just will not function for such a workload.
Re:AlphaFold2 at CASP14: “It feels like one’s child has left...
by rsfern on Saturday December 12, 2020 @03:02PM
(#25401131)
While interesting, I think this model is only viable for a certain subset of computational workloads.
AF2 took like two weeks of computation on 200 TPUs to train. I’m not sure if they used a data parallel or model parallel distributed training approach, but I don’t imagine either would scale particularly well to a globally distributed network of worker machines in a folding@home analogue...
In physics, we routinely have author lists of hundreds to thousands of authors on the big experiments. When resources are pooled to make a measurement, author lists can be huge.
University groups can absolutely compete with teams from industry, especially in consortia. The big difference I see between industry and academia in this context at the moment is that academics have to submit a grant proposal and compete for 6-12 months with scientists nationwide for $1M in compute-credits, whereas someone inside a company looking to flex its ML muscles may be able to requisition the same with an email.