Going from pure callback passing to yield-based cooperative multitasking increases clarity and removes a lot of opportunities for mistakes, so in practice it ends up being a win - but just like callback passing, you can still hit a point where it's too complicated or you're having to do too much.
It's only natural that going from custom task schedulers built on 'yield' to language-native mechanisms for cooperative multitasking improves things further. Even that, though, will likely not 'solve the issue', just make it considerably easier to tackle for individual applications. There's no One True Scheduler or One True Task Primitive, at least as far as I know: everyone seems to want something slightly different.
@producer
def prod():
for i in range(0,100):
yield i
@producer
@consumer
def middleman():
for i in (yield):
yield i
@consumer
def cons():
for i in (yield):
print i
Producers can yield values and consumers can use yield to get their input. I will fully admit that code, abstracted away by those decorators, is not the prettiest and took a minute to understand when writing. The biggest gotcha was with both a producer and consumer. You have to be very careful to capture the return value of the generator.send(input) call as it ends up being the item of the first yield. Also, pure consumers will immediately throw a StopIteration exception after sending the input. I'm very glad, however, that I stumbled upon using generators in this way as it results in a very clean implementation at this level. The internals are a little dicey, however, which I think is what you're eluding to.I use the functions in multiprocessing.Process objects as the target by first binding them to pipes and also without binding them to anything, in which case they use stdin/stdout.
Implicit stack coroutines (like greenlets) can always be built on top if necessary, but they tend to require horrible things like monkey patching to work together with existing synchronous, blocking code.
that's not true. monkey patching is an optional feature that you can use with greenlet-based libraries, but you don't have to.
compare that to inlineCallbacks, where you don't have that option.
Co-routines solve cooperative multitasking. Which is why they're now being discussed for python in a fashion that doesn't rely on C-stack hacks.