News for hackers. Stuff that matters.
Re:When your data doesn’t fit in memory: the basic techniques by dyanaraps on Monday November 11, 2019 @05:03PM (#21509383)
Useless use of `cat`. :)

    wc -l file
To hide filename:

    wc -l < file
Re:When your data doesn’t fit in memory: the basic techniques by scarejunba on Monday November 11, 2019 @05:13PM (#21509456)
I'm actually pretty much against UUoC. The cat example is much more resilient to modification. For instance, there are usually natural modifications considering filtering etc.

Anyway, since he said number we probably want `nl` not `wc`.

Separating pipe-setup from processing is sound engineering to me and makes things much easier. For instance as I iterate, I will have cat piped to head piped to filter but eventually I'll take the head out and run the whole thing. That's a trivial ^W versus moving around arguments and worrying about arg order, etc.

Re:When your data doesn’t fit in memory: the basic techniques by navaati on Monday November 11, 2019 @05:26PM (#21509570)
That's right regarding directly giving the filename for the program (here wc) to open itself.

However using the input redirect is fine, especially if you format it that way (which is exactly the same thing):

  <file wc -l
Then, adding another step is as natural as with cat:

  <file grep meh |wc -l
Re:When your data doesn’t fit in memory: the basic techniques by thestoicattack on Monday November 11, 2019 @05:34PM (#21509641)
Useless use of `wc` :)

    <file grep -c meh
Re:When your data doesn’t fit in memory: the basic techniques by scarejunba on Monday November 11, 2019 @06:19PM (#21509990)
25 years using shells and I never thought to place input redirection at the front. Well, thank you!
Re:When your data doesn’t fit in memory: the basic techniques by herpderperator on Monday November 11, 2019 @05:36PM (#21509667)
I am confused. What's wrong with `wc -l < file | ... | ...`?

I know `< file wc -l | ... | ...` is identical because the redirection is done before the command is executed, but what does putting it in front help with?

Re:When your data doesn’t fit in memory: the basic techniques by squiggleblaz on Tuesday November 12, 2019 @05:23AM (#21513006)
Because I feel the other answers, though correct, are entirely missing out on clarity:

The first iteration looks like this:

<file head -n10 | grep foo | tr baz qux

The second iteration looks like this:

<file gzcat | head -n10 | grep foo | tr baz qux

It makes the change much easier, since the first thing written is the first thing that happens. In `gzcat < file`, the logical first step - reading and streaming file - is now the physical second step. Like a German sentence, whenever you want to prepend the file, you have to maintain an item in second position.

Re:When your data doesn’t fit in memory: the basic techniques by emj on Monday November 11, 2019 @05:52PM (#21509789)
You can easily change that first command with muscle memory. That's why I use cat $file| but I guess <$file would work too.
Re:When your data doesn’t fit in memory: the basic techniques by emmelaich on Monday November 11, 2019 @07:12PM (#21510283)
As per James Taylor's comment above, using cat means you can easily replace it with head e.g. for testing.

And not change anything else in the pipeline.

Re:When your data doesn’t fit in memory: the basic techniques by james_s_tayler on Monday November 11, 2019 @06:26PM (#21510034)
I hate when people nitpick this stuff. What if we just straight prefer the first one?
Re:When your data doesn’t fit in memory: the basic techniques by BeetleB on Monday November 11, 2019 @08:04PM (#21510558)
There's a whole Wikipedia section for this:

https://en.wikipedia.org/wiki/Cat_(Unix)#Useless_use_of_cat

Re:When your data doesn’t fit in memory: the basic techniques by james_s_tayler on Tuesday November 12, 2019 @03:32PM (#21517837)
It all makes a lot more sense now. I'm glad I can add demoggification to my vocabulary.
Re:When your data doesn’t fit in memory: the basic techniques by mkl on Tuesday November 12, 2019 @06:05AM (#21513201)
I usually use cat, because I often build a pipeline by starting with

  head filename | ...
then just change "head" to "cat" when I get it working, or grep if I want to check parts of a file, etc.
Re:When your data doesn’t fit in memory: the basic techniques by Izkata on Tuesday November 12, 2019 @08:08AM (#21513744)
Likewise here, it maintains the left-to-right direction and "one operation per | gap |"
Re:When your data doesn’t fit in memory: the basic techniques by tomp on Tuesday November 12, 2019 @08:01AM (#21513695)
Bad code is code smell.

In this particular case, people (like me) would wonder, "what's the point" and then go searching for the details of how `cat` and `wc` work, assuming that there's might be a reason the original developer wrote this code, as opposed to the simpler `wc -l` (i.e. Chesterton's Fence).

Re:When your data doesn’t fit in memory: the basic techniques by Zelizz on Wednesday November 13, 2019 @05:27PM (#21530372)
People usually use it because piped input is a known interface, you don't have to go digging through a man page or help page to figure out what flag is needed to accept input from a file. "Data flows from left-to-right" is also very intuitive, and you don't need to know the specifics of how your shell interprets a command line to figure out where to put the input redirection in the case of a more complicated shell command.
Re:When your data doesn’t fit in memory: the basic techniques by james_s_tayler on Tuesday November 12, 2019 @03:28PM (#21517791)
The point is I personally find it more ergonomic. Just like how I'll take a slightly longer route to avoid turning directly across traffic or avoid a particularly dangerous intersection.

Also, context is really important. Not that people on the internet ever stop to think about what the context is that the person is writing the code in. I don't write bash scripts that run mission critical workloads. I just ad-hoc type stuff into my shell to produce output I need for stuff at the time.

Re:When your data doesn’t fit in memory: the basic techniques by calvinmorrison on Monday November 11, 2019 @10:28PM (#21511295)
Well one reason is programs being smart (dumb) about output if they think that dev stdout is a dev console. Using an extra cat eliminates this.
Re:When your data doesn’t fit in memory: the basic techniques by shermanyo on Monday November 11, 2019 @07:29PM (#21510375)
or just:

  cat -n file
Re:When your data doesn’t fit in memory: the basic techniques by messe on Tuesday November 12, 2019 @04:56AM (#21512901)
Or even:

    nl file