News for hackers. Stuff that matters.

A comment on What you need may be “pipeline +Unix commands” only. Parent comment

Re:What you need may be “pipeline +Unix commands” only by zaphar on Sunday March 03, 2019 @09:18AM (#19294513)
Just a few weeks ago I was working with an 11TB dataset which I processed with just unix command line tools told on a single Linux VM.

You can get a lot of milage out of parallel xargs and other of the shelf tools.

[ Parent | on HN ]
Re:What you need may be “pipeline +Unix commands” only by srean on Sunday March 03, 2019 @11:11AM (#19295116)
Indeed! Gnu prallel, xargs, make -j, netcat, mawk/gawk (with C extensions if needed), jq, coreutils, textutils can get insane amount of stuff done.
[ Parent | on HN ]