Re:Show HN: Quilt – manage data like code
by fredcash25 on Friday July 14, 2017 @01:53PM
(#14771772)
How is this different/the same as the DAT project (https://datproject.org/ and https://github.com/datproject)?
[ Parent
| on HN ]
Dat is a distributed transport layer for raw data. Quilt is a centralized (your infrastructure or ours) transport and consumption layer for virtualized data. As such we'll be able to, for example, run efficient queries across all of Quilt, allow users to import data the same way (no data prep scripts) across a variety of platforms, etc.
Re:Show HN: Quilt – manage data like code
by lmeyerov on Saturday July 15, 2017 @12:56PM
(#14777607)
I recently encountered https://data.world/datanerd/inc-5000-2016-the-full-list and was impressed by the combo of a pandas-friendly client library & centralized online tier with bells & whistles. Feels closer to that direction.
There's obviously friction in this space -- this weekend I'm playing with databricks spark notebooks and public data on S3 and the data prep will still be annoying -- so I'm looking forward to design innovation in it.
I'd like to chat more about this. Quilt uses S3 and I think we could make the process of getting data into Databricks much simpler. Drop me a line if you'd like to discuss aneesh at quiltdata dot io.