It's all fun and games until someone [XOFF]

Re:It's all fun and games until someone [XOFF] by Too on Sunday June 21, 2015 @03:27AM (#9752548)

This reminds me of a problem we had with running processes from python with subprocess.POpen. Some times after running the program for some time it just froze, none of us could figure out why and we reviewed the source of the invoked program and debugged it over and over again without finding anything that could get locked up. Eventually we found out that the problem was not in the invoked program, it was within python; piping the stdout into POpen was done to a fixed size buffer that was only being read/cleared after the invoked program exits, and since it can't continue to run it will never exit, stdout was getting blocked so just by writing to it the program froze. The solution to the problem was to start a new thread that kept reading the piped stdout and move it to another buffer. Today the documentation has a big warning about this behavior and a convenience-wrapper, communicate(), that does the threaded read for you but back then figuring this out was not exactly obvious.

[ Parent | on HN ]

Re:It's all fun and games until someone [XOFF] by d0mine on Sunday June 21, 2015 @09:11AM (#9753279)

Just to be clear, the issue is not in Python: the deadlock will happen if you won't consume the pipe whatever language you use to spawn a subprocess -- the buffer is not in your application -- it is the OS pipe buffer.

I imagine your broken code did something like:

  #XXX WRONG, DON'T DO IT
  p = Popen(cmd, stdout=PIPE)
  p.wait() #XXX deadlock is possible if cmd generates enough output
  output = p.stdout.read()

To avoid the deadlock, it is enough to swap the last two statements:

  p = Popen(cmd, stdout=PIPE)
  output = p.stdout.read() 
  p.wait()

that you would normally write as:

  output = check_output(cmd) # 2.7+

If you need to consume more than one pipe separately then you should consume them concurrently and therefore you need threads or async.io.

`Popen.communicate()` can read from both pipes concurrently (as you've mentioned) and it is always available. It was available even in Python 2.4 that introduced the subprocess module.

[ Parent | on HN ]

Re:It's all fun and games until someone [XOFF] by pjungwir on Monday June 22, 2015 @03:04PM (#9760481)

Yep, I've hit this in Perl and Ruby and even once wrote a class to do something like Python's `communicate`. It's especially trappy if you want to run a command and both send stuff to its stdin and read stuff from its stdout. Bash makes it so easy, but it's a fraught with danger anywhere else!

[ Parent | on HN ]

Re:It's all fun and games until someone [XOFF] by swinglock on Sunday June 21, 2015 @12:53PM (#9754048)

Funny, I fixed a bug caused by exactly this recently in a program in production that had been humming along nicely until the output of another utility had grown larger than it ever was in testing or previously in production, causing the deadlock. The big warning in the Python docs sure was helpful in finding the problem.

[ Parent | on HN ]