News for hackers. Stuff that matters.
Re:Some thoughts on asynchronous Python API design in a post... by quotemstr on Monday November 07, 2016 @12:12AM (#12888951)
> threads are still expensive in bulk

They don't have to be. First of all, even ordinary threads are more efficient than you might think. On a really awful low-end Android 4.1 device, I can pthread_create and pthread_join over 5,000 threads per second. On a real computer, my X1 Carbon Gen4, I can create and join over 110,000 threads per second. (And keep in mind that each create-join pair also forces two full context switches.)

For most applications, performance of regular threads is perfectly adequate. In these environments, the maintainability and debuggability advantages of using plain old boring threads makes it really hard to justify using something exotic.

But suppose you do have big performance requirements: you can still use normal-looking threaded code. There's a difference between how we represent threads in source code and how we implement them. It's possible to provide green, userspace-switched threads without requiring "await" and "async" keywords everywhere. GNU Pth did it a long time ago, and there are lots of other fibers implementations.

> the CPU has to shuffle a lot of data every time you switch

Any green-threaded system (with or without explicit preemption points) also does context switches! Such a system maintains in user space a queue of things to work on: as the system switches from one of these work items to another, it's switching contexts! You have the same kind of register reloading and cache coldness problems that switching thread contexts has. There's no particular reason that you can do it much better than the kernel can do it, especially since switching threads in the same address space is pretty efficient.

Re:Some thoughts on asynchronous Python API design in a post... by int_19h on Monday November 07, 2016 @02:29AM (#12889476)
The problem with all green thread implementations that I know of is that they're language and/or framework-specific. So the moment you start using them, you get the same set of problems as using setjmp/longjmp in C across the boundaries of foreign code - it either just blows up spectacularly, or at the very least violates invariants because the interleaving code is not aware that someone's pulling the rug from under it.

This can only be solved by standardizing a fiber API and (per platform) ABI, and by forcing all libraries in the ecosystem to be aware of fibers if their behavior differs with threads in any way (e.g. if TLS and FLS are distinct).

Callbacks (and hence promises), on the other hand, work with what we already have, and are trivially passed across component boundaries as a simple function pointer + context pointer, or some suitable equivalent expressible in C FFI. For example, I can take an asynchronous WinRT API (which returns a future-like COM object), and wrap it in a Python library that returns awaitable futures; with neither WinRT being aware of the specifics of Python async, nor with Python aware of how WinRT callbacks are implemented under the hood. On the other hand, if WinRT used Win32 fibers for asynchrony, Python would have to be aware of them as well.

Re:Some thoughts on asynchronous Python API design in a post... by anonymoushn on Monday November 07, 2016 @12:39PM (#12892987)
I expect you can also use a callback that switches greenlets, or one passes the values it got to lua's coroutine.resume.
Re:Some thoughts on asynchronous Python API design in a post... by int_19h on Monday November 07, 2016 @02:54PM (#12894395)
How can it switch green threads without breaking any foreign code currently on the stack? Consider what happens when said code holds an OS mutex, for example.

The only way I see this working is if your green threads roll their own stack on the heap, and switch that, without touching the OS stack. But then how is the result fundamentally different from promise chains? Their callbacks and captured state essentially form that very same green stack.

Re:Some thoughts on asynchronous Python API design in a post... by quotemstr on Monday November 07, 2016 @03:22PM (#12894680)
To start a fiber, you allocate some memory, set RSP to the end of that memory, set your other registers to some arbitrary initial state, and jump to your fiber routine. To switch fibers, you set RSP to some other block of memory, restore your registers, and set PC to whatever it was when you last switched away from that fiber. There's nothing magical, and it works with almost all existing code. If you hold a mutex and switch to a different fiber, the mutex stays held. How could it be otherwise?
Re:Some thoughts on asynchronous Python API design in a post... by int_19h on Monday November 07, 2016 @07:48PM (#12896675)
I was thinking of a situation where thread-aware but not fiber-aware code uses mutex to synchronize with itself, which breaks with fibers because they reuse the same thread, and the mutex is bound to that thread (so if another fiber tries to acquire that mutex, it's told that it already has it, and proceeds to stomp over shared data with impunity).

But upon further consideration, I realize that in this narrow scenario - where fibers are used in conjunction with callback-based APIs - this shouldn't apply, because you can't synchronize concurrent callback chains with plain mutexes, either.

Having said all that, are there any actual implementations that seamlessly marry fibers with callbacks? I don't recall seeing any real world code that pulled that off. Which seems to imply that there are other problems here.

Of note is that CLR tried to support fibers, and found it to be something that was actually fairly expensive. By extension, this also applies to any code running on top of that VM:

"If you call into managed code on a thread that was converted to a fiber, and then later switch fibers without involvement w/ the CLR, things will break badly. Our stack walks and exception propagation will rely on the wrong fiber’s stack, the GC will fail to find roots for stacks that aren’t live on threads, among many, many other things." (http://joeduffyblog.com/2006/11/09/fibers-and-the-clr/)

GC is a sticking point here, it seems - clearly it needs to be fiber-aware to properly handle roots in switched-out fibers.

Re:Some thoughts on asynchronous Python API design in a post... by anonymoushn on Monday November 07, 2016 @09:50PM (#12897204)
That all sounds correct to me. I'm not familiar with greenlet internals, but lua's stacks live on the heap and the whole situation ends up being similar to promise chains in terms of where your state is at runtime.
Re:Some thoughts on asynchronous Python API design in a post... by jeswin on Monday November 07, 2016 @02:21AM (#12889457)
Short replies coz on phone.

1. Async await is almost similar to normal looking threaded code. Just add await before a normal looking call.

2. A language could have chosen to make it "exactly the same" by auto inserting awaits, but then you don't get to say when you don't actually want to wait. Many times you don't.

3. I agree native threads are cheap. But you still have a) thread stacks and additional control structures, b) wouldn't you have to deal with things like processor affinity? I mean, either you/lib or the kernel. And the kernel already does it for you.

Re:Some thoughts on asynchronous Python API design in a post... by TimJYoung on Monday November 07, 2016 @08:00AM (#12890664)
Excellent discussion, and your points are all very much spot-on. I just wanted to add this re: Windows and fibers/threads because it is very much relevant to the conversation:

https://blogs.msdn.microsoft.com/larryosterman/2005/01/05/wh...

Re:Some thoughts on asynchronous Python API design in a post... by fzzzy on Monday November 07, 2016 @09:37AM (#12891261)
I don't believe you. Show me your code. I think you just completely made your numbers up.
Re:Some thoughts on asynchronous Python API design in a post... by quotemstr on Monday November 07, 2016 @11:34AM (#12892331)
Re:Some thoughts on asynchronous Python API design in a post... by fzzzy on Monday November 07, 2016 @01:16PM (#12893384)
Thanks!
Re:Some thoughts on asynchronous Python API design in a post... by RubyPinch on Monday November 07, 2016 @02:10PM (#12893912)
> On a real computer, my X1 Carbon Gen4, I can create and join over 110,000 threads per second.

is that C, or python's multithreading?