https://news.ycombinator.com/item?id=28708145
...but if your files are CSV, there is a CSV extension for gawk
@include "csv"
BEGIN { CSVMODE = 1 }It's funny searches for awk CSV seem to yield a bunch of SO questions where the answers are increasingly cumbersome regexes instead of this extension.
Of course, you can't count of this extension being widely installed, but it's great for my own desktop.
awk -f ./ucsv.awk -e '{print $5}'
Also this> 4,"hello, \"world\"",6
Is incorrect per https://tools.ietf.org/html/rfc4180 so you should just fix it with a sed -i 's/\\"/""/g' and then just parse as normal.
I just end up using Python/Perl but I do have a soft spot for awk so it would be cool if good support was built-in.
XML is somewhere in the middle--I've seen some horrible abuses of CDATA sections way back when--but at least there are accepted ways to prove what's invalid.
In principle:
cat textfile.csv | csvquote | awk -f myprogram.awk | csvquote -u > output.csv
Also works for other text processing tools like cut, sed, sort, etc.
col1,col2,col3
1,2,3
4,"hello, \"world\"",6
"7 buckets",,9
To get the usual awk experience with this very common file format, exactly the type of thing you want to parse with awk, you first need to install gawk, then use a big FPAT regex that needs to be adjusted for any new CSV variant.
I would love to see awk with "CSV mode", where it intelligently handles formats like this if you just pass a flag. I think awk would do well to differentiate itself with excellent 2d dataset parsing functionality, but at least catchup up to the average scripting language would be great.
I'm half expecting someone to say "just pass -csv it does what you want" and if so I'll be very excited.