Re:When your data doesn’t fit in memory: the basic techniques
by squiggleblaz on Tuesday November 12, 2019 @05:23AM
(#21513006)
Because I feel the other answers, though correct, are entirely missing out on clarity:
The first iteration looks like this:
<file head -n10 | grep foo | tr baz qux
The second iteration looks like this:
<file gzcat | head -n10 | grep foo | tr baz qux
It makes the change much easier, since the first thing written is the first thing that happens. In `gzcat < file`, the logical first step - reading and streaming file - is now the physical second step. Like a German sentence, whenever you want to prepend the file, you have to maintain an item in second position.
The first iteration looks like this:
<file head -n10 | grep foo | tr baz qux
The second iteration looks like this:
<file gzcat | head -n10 | grep foo | tr baz qux
It makes the change much easier, since the first thing written is the first thing that happens. In `gzcat < file`, the logical first step - reading and streaming file - is now the physical second step. Like a German sentence, whenever you want to prepend the file, you have to maintain an item in second position.