News for hackers. Stuff that matters.
Re:Running Awk in parallel to process 256M records by Zeebrommer on Wednesday June 03, 2020 @03:05AM (#23400744)
I am often impressed by the things that can be done with these old-school UNIX tools. I'm trying to learn a few of them, and the most difficult part are these very implicit syntax constructions. How is the naive observer to know that in bash `$(something)` is command substitution, but in a Makefile `$(something)` is just a normal variable? With `awk`, `sed` and friends it gets even worse of course.

Is the proper answer 'just learn it'? Are these tools one of these things (like musical instruments or painting) where the initial learning phase is tedious and frustrating, but the potential is basically limitless?

Re:Running Awk in parallel to process 256M records by avar on Wednesday June 03, 2020 @03:15AM (#23400800)
Some of it you pickup or remember from the context of the file you're looking at, but you really should take the time to read the manuals for these tools from cover to cover at some point if you're making extensive use of them. In the case of GNU bash & make: https://www.gnu.org/savannah-checkouts/gnu/bash/manual/bash.... & https://www.gnu.org/software/make/manual/make.html
Re:Running Awk in parallel to process 256M records by MaxBarraclough on Wednesday June 03, 2020 @05:46AM (#23401668)
Shoutout to the lesser-known GNU Recutils. Not sure how well it would fare in this particular instance, but it seems relevant.

https://www.gnu.org/software/recutils/

https://en.wikipedia.org/wiki/Recfiles