News for hackers. Stuff that matters.
Re:Show HN: Quilt – manage data like code by akarve on Friday July 14, 2017 @02:37PM (#14772170)
Data cleaning is so necessary. `build.yml` already supports a limited set of feature (through pandas). In addition to custom data transformations, any "out of the box" cleaning functions you'd like? In the spirit of dplyr? We've looked at e.g. scikit feature for normalization, 1-of-n encoding, etc.
Re:Show HN: Quilt – manage data like code by zitterbewegung on Friday July 14, 2017 @06:26PM (#14773701)
How about deduplication. Also table / dictionary lookup . And string replacement and regex replacement .
Re:Show HN: Quilt – manage data like code by akarve on Saturday July 15, 2017 @11:16AM (#14777058)
Got it. If you'd like me to ping you once we have custom build hooks: aneesh at quiltdata dot io. Have you tried Luigi or Bonobo for data cleaning?
Re:Show HN: Quilt – manage data like code by zitterbewegung on Saturday July 15, 2017 @12:49PM (#14777568)
No I haven't I think I just did it myself I'll look into that .