News for hackers. Stuff that matters.

A comment on Qri: A global dataset version control system built on the distributed web. Parent comment

Re:Qri: A global dataset version control system built on the... by ktpsns on Saturday March 23, 2019 @02:40PM (#19471722)
> scientific data primarily at a larger scale (100 GB to petabytes)

Buying hard discs (100TB for a few 10kEUR a few years ago) is a real investion in our institute. As far as I understood, with distributed storages each participant volunteers to share his disc to store his (and other) data. Here's the devil's advocate: Why should I share my expensively bought disc space with you?

[ Parent | on HN ]
Re:Qri: A global dataset version control system built on the... by DocSavage on Saturday March 23, 2019 @03:38PM (#19471992)
Some institutions won't pay for others. In our space, big non-profit science institutions like Janelia and Allen Brain foot the bill for making the data available. Depending on the utility of the data, Amazon (https://aws.amazon.com/opendata/public-datasets/) or Google (https://cloud.google.com/public-datasets/) could also handle the cost of storing and distributing the data.

With versioned data, you could leverage the largesse of the big institutions to provide the base data, and then only the deltas for the children versions need be handled by users making changes.

[ Parent | on HN ]