News for hackers. Stuff that matters.
Re:On Counting (2017) by mfontani on Thursday May 24, 2018 @06:47AM (#17142753)
Reminds me quite a bit of https://adamdrake.com/command-line-tools-can-be-235x-faster-...

… in which, though, the author at least parallelises the counting.

Re:On Counting (2017) by zxcmx on Thursday May 24, 2018 @08:15AM (#17143199)
There is a nearly-as-simple solution involving lambda (which would not require running a kafka cluster) but meh.
Re:On Counting (2017) by ILMostro7 on Thursday May 24, 2018 @07:06AM (#17142849)
Bash 5.0? Parallel execution
Re:On Counting (2017) by lmilcin on Thursday May 24, 2018 @08:35AM (#17143308)
Counting unique occurences isn't easy if you are given data the way it was described in the article and you have a requirement for exact count.

But there are things you can do to help if you are willing to redesign it a bit.

You could partition the data up front, as it is being written. Then you could send partitions to be processed in parallel (and you still could do this with the shell command!)