News for hackers. Stuff that matters.
Re:Dat – A git-like tool for large datasets by danieldk on Saturday April 26, 2014 @02:36AM (#7650633)
For those interested in a solution that you can use now, iRODS may be interesting:

https://irods.org/

iRODS is a data storage system that provides replication (e.g. on local filesystems, S3, and HPSS. Deployments at different organizations can be linked, to make (part of) the namespace of one company be visible to that of another company. It has a permission system on namespaces.

But one of the nicest features is that it has a built-in C-like trigger language. You can specify triggers for certain namespaces. E.g., for research, it is often necessary to pointers to data (URLs) that are permanent and can be published in e.g. papers. For this reason EUDat[1] has triggers on namespaces that automatically create PIDs[2].

But since the trigger language is very complete, you can do nearly anything with it.

Note: I am not involved in iRODS anyway, I just know a couple of colleagues who use it for safe replication, sharing, etc.

[1] http://www.eudat.eu/ [2] http://www.pidconsortium.eu/

There are no comments on this comment yet.