News for hackers. Stuff that matters.
Re:Alphafold by dekhn on Thursday July 15, 2021 @03:35PM (#27849083)
many of these resources are available, it's mostly that academic scientists don't have the time, money, or expertise to manage large datasets. However, the community has maintained high quality MSA database for decades and that's exactly the work that DM drafted off.
Re:Alphafold by gnufx on Thursday July 15, 2021 @05:24PM (#27850310)
> academic scientists don't have the time, money, or expertise to manage large datasets

I may be cynical about general expertise, as a support person, but large datasets have long been stock in trade of areas I'm more or less familiar with, whether "large" is TBs or PBs like CERN experiments. (When I were a lad, it was what you could push past the tape interface in a few days -- data big in cubic feet...)

Re:Alphafold by dekhn on Thursday July 15, 2021 @05:48PM (#27850534)
Tape is worthless except for archival purposes (and it's not particularly good). it should not be the constraint on the dataset (IE, any important dataset should already be in live serving with replication).

Very few players wrangle petabytes effectively. Many players have petabytes, but they're just piles of disorganized data that couldn't be used for training ML. Moving petabytes is still a huge pain and few folks have proficiency in giving ML algorithms high performance access to the data.

Re:Alphafold by pas on Friday July 16, 2021 @02:29AM (#27853532)
The linked GH repo's readme says that it needs to download ~430GB and it takes ~2.5TB unzipped.