News for hackers. Stuff that matters.
Re:TSV Utilities: Command line tools for large, tabular data... by austinjp on Sunday September 01, 2019 @04:03AM (#20851076)
Thanks for this. Looks perfect for my needs. I've got to analyse a hideous generated CSV file with thousands of columns. I was planning to load it into SQLite3 but it has too many columns for import. I was also unpleasantly surprised to find that the column names were too long for PostegresQL too. Python + pandas can handle it, but a tool for quick interactive exploration is just what I need. Hopefully visidata will provide!
Re:TSV Utilities: Command line tools for large, tabular data... by IanCal on Sunday September 01, 2019 @05:43AM (#20851389)
If you still hit size issues, you might be able to throw things through xsv first (list columns, select just a few columns and pipe out, get stats on contents of columns, samples, etc). It's one of my other goto things https://github.com/BurntSushi/xsv