Re:CSVfix is a tool for manipulating CSV data
by michaelmior on Tuesday November 12, 2013 @06:00PM
(#6722019)
csvkit is awesome. My only complaint was that it correctly parsed a file and left my confused when I had half the number of rows I expected because I didn't realize each row contained a line break.
Re:CSVfix is a tool for manipulating CSV data
by georgehotelling on Tuesday November 12, 2013 @06:35PM
(#6722240)
I've had some success cleaning up CSV data in the past using OpenRefine[0] (née Google Refine, and Freebase Gridworks before that). It is a really powerful tool for getting data in a consistent format.
Re:CSVfix is a tool for manipulating CSV data
by eli on Wednesday November 13, 2013 @09:09AM
(#6725199)
That reminds me, there's actually a funky old Microsoft skunkworks project that lets you query CSVs and Apache logfiles and all sorts of stuff via SQL: http://www.microsoft.com/en-us/download/details.aspx?id=2465 I have no idea if it even still runs on current windows versions.
Re:CSVfix is a tool for manipulating CSV data
by RexRollman on Tuesday November 12, 2013 @06:24PM
(#6722176)
Way off topic but one of my favorite iOS apps is CSVtouch. It works well with .csv files and it lets me keep my data is an open format (although it can only view files and not edit them).
[0] https://github.com/onyxfish/csvkit
[1] https://github.com/jehiah/json2csv
[2] https://github.com/stedolan/jq
https://github.com/dbro/csvquote
Apply it first, then do the normal processing with GNU coreutils and you'll cover most use cases.
[0] http://openrefine.org/
I want a way to use Open Refine and export the code to Python.
I'm frequently surprised by how popular that project remains:
It takes some time to get used to the workflow, but it's very powerful and does a great job making messy data usable.
[1]https://code.google.com/p/google-refine/
http://technet.microsoft.com/en-us/scriptcenter/dd919274.asp...