News for hackers. Stuff that matters.
Re:Global Interpreter Lock, or how to kill it by jerf on Wednesday June 29, 2011 @03:45PM (#2710923)
This seems to be conflating two levels of concurrency. One is at the user level, where the user may want to write a function that atomically removes an element from one list and adds it to another, and one is at the interpreter level, where it needs to be able to complete all operations involved in, say, adding the element to the list, without another thread coming in and stomping on the first thread, with all the associated hazards.

At the interpreter level, as the article mentions, it suffices to lock the structures down as Jython does. STM would just seem to add overhead that isn't necessary.

At the user level, STM + imperative(/uncontrolled effects) is basically a known failure. A lot of effort has been spent on it, with people with similar levels of control over their VM (like the C# attempt), and it just doesn't really work.

If you've got the control necessary to automatically STM things, you might as well just equally-automatically copy the Jython-style locking. STM is "nifty" but I don't see that it actually adds anything useful. Either that or you have to rigidly control effects in user code, and that's not Pythonic in any sense whatsoever (philosophical or practical).

Re:Global Interpreter Lock, or how to kill it by Locke1689 on Wednesday June 29, 2011 @04:14PM (#2711053)
At the user level, STM + imperative(/uncontrolled effects) is basically a known failure. A lot of effort has been spent on it, with people with similar levels of control over their VM (like the C# attempt)

Agreed. The only project that I know of that's even close to a "working" implementation of STM is Haskell and that's because Haskell doesn't have uncontrolled stateful code.

Re:Global Interpreter Lock, or how to kill it by swannodette on Wednesday June 29, 2011 @04:23PM (#2711109)
Clojure is designed for the user level and offers some unique advantages over Haskell's STM [1]:

  * MVCC snapshot avoiding transactions restarts 
    on read invalidation.
  * Ensure references on read-writes provides a 
    kind of manual control over resource acquisition order.
  * Has explicit commute which reduces retries on 
    commutative writes.
[1] http://stackoverflow.com/questions/4560605/how-does-clojure-...

EDIT: I didn't read the OP closely enough. Yes grafting fine-grained STM onto a imperative language hasn't born much fruit.

Re:Global Interpreter Lock, or how to kill it by william42 on Wednesday June 29, 2011 @04:45PM (#2711222)
Isn't Clojure still mostly functional and mostly based on persistent data structures?
Re:Global Interpreter Lock, or how to kill it by calebmpeterson on Wednesday June 29, 2011 @05:50PM (#2711540)
Yes very much so...and then some.
Re:Global Interpreter Lock, or how to kill it by tomp on Wednesday June 29, 2011 @04:48PM (#2711235)
> At the interpreter level, as the article mentions, it suffices to lock the structures down

That's a sufficient, but not necessary condition. The idea of STM is to provide just the necessary protection, and nothing more, so that even multiple threads can manipulate the same structure simultaneously, as long as they don't manipulate the same properties/parts of this structure.

Having said that, I don't claim that the way (P|J)ython is written, there is any unnecessary protection going on, furthermore, to fully ensure that just the necessary protection is on, a lot of code may have to be rewritten.

Re:Global Interpreter Lock, or how to kill it by fijall on Wednesday June 29, 2011 @04:17PM (#2711073)
"If you've got the control necessary to automatically STM things, you might as well just equally-automatically copy the Jython-style locking."

So far thinking about how to really do that automatically failed. Do you feel like providing an idea?

Re:Global Interpreter Lock, or how to kill it by jerf on Wednesday June 29, 2011 @05:27PM (#2711438)
Failed for CPython or failed for PyPy? I'm just going on the claim that PyPy is in a position to automatically add STM, per that blog post. If in fact it can't automatically add locks, it isn't going to be automatically adding STM either and this is all a moot discussion.
Re:Global Interpreter Lock, or how to kill it by jholman on Wednesday June 29, 2011 @10:07PM (#2712205)
Adding fine-grained locks without risking deadlock is pretty tricky. Perhaps too tricky for automatic addition.

As I (shallowly) understand it, STM doesn't introduce deadlock threats, so it requires less-amazing feats to add it automatically.

Note, again, that the FA does admit that using STM will perform more poorly than fine-grained locking, and he only hopes that it will scale well enough that extra cores will pay for it.

As I understand it, one of the reasons for the GIL is that it makes it easier to write C extensions (especially as compared with fine-grained locking, which is not friendly to extensions), and I'm not sure, but I think the STM scheme preserves this property?