Re:Show HN: Quilt – manage data like code
by eternauta3k on Friday July 14, 2017 @06:36PM
(#14773754)
> If I wanted to share my data, wouldn't I rather use something more accessible?
Still, it's good inspiration. Maybe I'll make a github repo with my city's open datasets loaded into python.
There are some important differences from git: https://news.ycombinator.com/item?id=14772036
It's not about people not being able to load their data, but about accelerating the loading with serialization and about whether or not people want to focus on data cleaning, or have the cleaning done once and then available for posterity.
Quilt is, in my view, as open as git or GitHub. The de/serialization code is all open source, and uses an open format (Parquet). Parquet is accessible (and more optimal than text files) for things like Presto DB, Hive, etc.
> Quilt is, in my view, as open as git or GitHub
Can I run something that hosts a quilt repo on my own server?
Yes. In progress and the first on-prem installs are up and running. Happy to chat further: aneesh at quilt data dot io.
Anyway, do I get this right: They expect users to be experts in data analysis but not being able to load the data into whatever software they use? They want me to share data and to offload my data into their walled garden that can be accessed only via their service? If I wanted to share my data, wouldn't I rather use something more accessible?