News for hackers. Stuff that matters.
Re:Modernizing AWK, a 45-year old language, by adding CSV su... by rgoulter on Thursday May 12, 2022 @05:01AM (#31351116)
I'd be curious to see a comparison with the csvkit suite. https://csvkit.readthedocs.io/en/latest/index.html
Re:Modernizing AWK, a 45-year old language, by adding CSV su... by benhoyt on Thursday May 12, 2022 @05:29AM (#31351354)
That's written in Python and uses the "agate" library which uses Python's built-in "csv" module. I did a couple of simple benchmarks against Python's csv module in the article: https://benhoyt.com/writings/goawk-csv/#performance (Go/GoAWK is a bit more than 3x as fast)

I also did a quick test using csvcut to pull out a single field, and compared it to GoAWK. Looks like GoAWK is about 4x as fast here:

  $ time csvcut -c agency_id huge.csv >/dev/null

  real 0m25.977s
  user 0m25.240s
  sys 0m0.424s
  $ time goawk -i csv -H -o csv '{ print @"agency_id" }' huge.csv >/dev/null

  real 0m6.584s
  user 0m7.434s
  sys 0m0.480s
Re:Modernizing AWK, a 45-year old language, by adding CSV su... by rgoulter on Thursday May 12, 2022 @05:57AM (#31351569)
I meant more in terms of user/developer experience.

I imagine there's some use case where AWK will be able to come up with output that csvkit can't.

But for simple cases, csvkit's invocation is easier to remember.

Re:Modernizing AWK, a 45-year old language, by adding CSV su... by benhoyt on Thursday May 12, 2022 @08:21PM (#31361714)
Ah, true -- that would be a good comparison to do, or section to add the the docs (csv.md in the repo). I'll add it to my to-do list!