News for hackers. Stuff that matters.
Re:Bash redirections cheat sheet by dllthomas on Monday September 10, 2012 @04:59PM (#4502231)
One thing worth noting is that "redirection of streams" is a poor metaphor.

If I have two streams,

  1 ---------- 1'

  2 ---------- 2'
and I redirect the 1>2

  1 ----
        \
  2 ---------- 2'
it seems to work okay.

But if you have three,

  1 ---------- 1'

  2 ---------- 2'

  3 ---------- 3'
and try "redirecting 1 to 2 then 2 to 3", the natural reading would be

  1 -
     \
  2 -----
         \
  3 ---------- 3'
But that's not what you get, because you're not "redirecting streams" - you're copying file descriptors. This is partially fixed by informing people to do the operations backwards, but there are corner cases where that breaks down, and "copying file descriptors in the order they appear on the line" never does because that's genuinely what's going on.

  1 -> 1'
  2 -> 2'
  3 -> 3'
foo 1>2 2>3

This is "run foo copying fd 2 to 1 and 3 to 2"

which gives

  1 -> 2'
  2 -> 3'
  3 -> 3'
Re:Bash redirections cheat sheet by wethesheeple on Monday September 10, 2012 @08:32PM (#4503078)
Is there a typo: s/genuinely/not geniuinely

I guess what you're explaining is the process that happens as the kernel reserves fd's? And when you start your program there isn't necessarily any guarantee that a particular fd will be available, except for 0,1,2, right?

Re:Bash redirections cheat sheet by dllthomas on Monday September 10, 2012 @08:46PM (#4503117)
Copying file descriptors is genuinely what's going on, so understanding it that way always works.

Edited for clarity:

The way file descriptors work is that the kernel maintains, for each process, a table mapping from numerical file descriptors to structures describing the open file/socket/device associated with that descriptor.

When the shell is setting up to spawn a new process (or subshell, for that matter), it processes the redirections in order, left to right. For redirections of the form "A>&B" this is a call to dup2(B, A), which has the kernel copy the entry in the table at B over the entry at A.

Incidentally, you will note the direction of the symbol doesn't matter when cloning file descriptors: 5>&7 and 5<&7 mean the same thing - you're just copying a file descriptor. It does matter when opening a file, as the file will be opened for reading or writing before before being dup2'ed over the requested descriptor. Having said that, of course use the correct symbol - it's good documentation.

Re:Bash redirections cheat sheet by wethesheeple on Monday September 10, 2012 @10:12PM (#4503413)
Thanks for the this. In my opinion, there's too little written about passing fd's (use of dup2). It seems like a technique that is more useful than just as part of shell redirection. Safer than popen()?
Re:Bash redirections cheat sheet by dllthomas on Tuesday September 11, 2012 @01:34PM (#4506719)
I'm pretty sure popen is just a wrapper around this stuff. It's basically,

pipe: get two fds connected by a pipe

fork: create a child process

dup2: move the read fd of the pipe to 0 (stdin) in the new process

exec: run the program in the child process

There's some cleanup but that's the gist - popen isn't a syscall (or reasonably close to one), so must necessarily rely on other stuff to get its work done where it actually interfaces with the kernel.

Re:Bash redirections cheat sheet by wethesheeple on Tuesday September 11, 2012 @03:00PM (#4507166)
That makes perfect sense. It's the fork+exec that makes it dangerous (e.g. in the CGI context). This is something I've been wondering about and you have just provided a jolt of clarity. Many thanks, again.
Re:Bash redirections cheat sheet by dllthomas on Tuesday September 11, 2012 @03:17PM (#4507282)
Happy to help :)