News for hackers. Stuff that matters.
Re:So Pythonistas, you want to get rid of the GIL... by amirmansour on Friday July 06, 2012 @03:24AM (#4206740)
I'm actually interested to know what the HN community does to achieve parallelism in Python? Do you use multiprocessing module, Stackless python, PP module (parallel python)?
Re:So Pythonistas, you want to get rid of the GIL... by dagw on Friday July 06, 2012 @03:38AM (#4206777)
Either multiprocessing or shifting the bits that really benefit from parallelization out to C/C++ tend to cover the vast majority of my needs. Also numpy and numexpr does a lot of the multiprocessing stuff for me, so in many cases I don't even have to think about it.
Re:So Pythonistas, you want to get rid of the GIL... by Nursie on Friday July 06, 2012 @04:22AM (#4206884)
That's usually the point at which I decide the project could really use being rewritten in C.

If I need performance badly enough that I'm considering making major structural changes in order to utilise multi-process or find some other way around the GIL, I probably ought to be considering using a language that supports what I'm trying to do. Of course my python projects tend to be fairly small.

Re:So Pythonistas, you want to get rid of the GIL... by smeg on Friday July 06, 2012 @04:27AM (#4206897)
Or just write the fast bits in C, call it from Python via ctypes and get the GIL unlocked and parallelism for free.
Re:So Pythonistas, you want to get rid of the GIL... by Nursie on Friday July 06, 2012 @04:32AM (#4206906)
If my projects were large enough to make that a more attractive option, sure, that's a great plan.

Usually when I've hit this situation I'm writing something simple but processor intensive, and it's easier for me (12 years commercial parallel/distributed C programming, amateur python enthusiast) to go back to home ground at this point.

Re:So Pythonistas, you want to get rid of the GIL... by dbaupp on Friday July 06, 2012 @03:47AM (#4206787)
For sufficiently new version of Python (3.2+), concurrent.futures[1] is pretty nice for its use-case, but otherwise multiprocessing has been quite good.

[1]: http://docs.python.org/py3k/library/concurrent.futures.html

Re:So Pythonistas, you want to get rid of the GIL... by cageface on Friday July 06, 2012 @06:02AM (#4207111)
If the problem at hand isn't easily solved with a multi-process approach then this would probably be the time I'd start taking a hard look at another language.

Scala, Go, Clojure etc make this so much easier that I think a language jump is reasonable.

Re:So Pythonistas, you want to get rid of the GIL... by willvarfar on Friday July 06, 2012 @03:29AM (#4206752)
I use workers (sometimes multiprocessing, sometimes I do my own starting of them).

Often I have use-cases where my own shared memory system works well http://williamedwardscoder.tumblr.com/post/13363076806/buffc...

> I was writing a custom video editing program in Python (interesting choice of language for that problem) and I wanted to cache decoded frames

Re:So Pythonistas, you want to get rid of the GIL... by blitzprog on Friday July 06, 2012 @04:02AM (#4206832)
I'm using QThread for PyQt GUI applications. However the GIL makes it impossible to execute _Python_ code in parallel. I can understand the reasons for that, however it doesn't change the fact that it makes my application inherently slower and is therefore something I dislike.

Apart from that I really like Python.

Re:So Pythonistas, you want to get rid of the GIL... by smeg on Friday July 06, 2012 @04:24AM (#4206888)
Same here. As a nice side effect, QThread somehow makes signals work again, so you can Ctrl-C a MT Python program (you still have to set a handler for SIGINT but at least it works).

For this reason, I actually use QThread in console apps aswell.

Re:So Pythonistas, you want to get rid of the GIL... by sateesh on Friday July 06, 2012 @04:33AM (#4206911)
If you are using QThread for console apps and are not using any other Qt libraries, isn't your console app has an unnecessary dependency on Qt ?
Re:So Pythonistas, you want to get rid of the GIL... by lmm on Friday July 06, 2012 @06:42AM (#4207198)
Qt's split up into separate libraries; QtCore is pretty much a general portable utility library like glib or APR, and it's perfectly reasonable for a console app to depend on it.
Re:So Pythonistas, you want to get rid of the GIL... by smeg on Friday July 06, 2012 @04:40AM (#4206925)
Well I guess it is "unnecessary" to the extent it is unnecessary to be able to Control-C your programs. Personally I like my programs to respond to signals.
Re:So Pythonistas, you want to get rid of the GIL... by heretohelp on Friday July 06, 2012 @04:10AM (#4206848)
multiprocessing or rewrite in something else, depending on the nature of the problem.