Re:Show HN: Joule, a Git-like workflow for your Datasets
by phanaton on Sunday November 03, 2019 @04:21PM
(#21436700)
What advantages over Git LFS does it offer?
Re:Show HN: Joule, a Git-like workflow for your Datasets
by Allezxandre on Sunday November 03, 2019 @06:12PM
(#21437428)
Git LFS is limited by your hosting solution. If you use Gitlab, for instance, your repository cannot exceed 10 Gb of storage space (including code and other ressources). And on Github I believe that limit is 1 Gb.
Many Deep Learning datasets that deal with pixel-data grow past the 10 Gb mark.
I built this service to solve that: instead of hosting the datasets yourself, you just push them using Joule, and the users that download your code will only need to run a single command to download your code.
EDIT: here’s a sample Git repository with a dataset hosted on Joule: https://gitlab.com/joule-host/that-state-of-the-art-research...