News for hackers. Stuff that matters.
Re:Goroutines, Nonblocking I/O, and Memory Usage by steveklabnik on Thursday January 05, 2017 @06:49PM (#13332599)
So, soon I will have some stuff to point you to with regards to how Rust is doing it, but we're not _quite_ there, so as a summary...

With Tokio, you write non-blocking IO stuff as "futures", similarly to promises in JS. These then get compiled down to what's effectively a state machine, and they run in an event loop on top of epoll/kqueue. That event loop is single threaded, and fully non-blocking.

I am not a super, super expert on these low-level details, but I believe that this should mean that it is closer to the C code than the Go code.

Re:Goroutines, Nonblocking I/O, and Memory Usage by dom96 on Friday January 06, 2017 @07:27AM (#13335769)
This sounds similar to what we have in Nim right now. There are multiple layers (from highest to lowest):

* Async await

* The async dispatcher (event loop)

* Selectors module

Each builds on top of the layers below it. The selectors module is an abstraction for the various system `select()` mechanism such as epoll or kqueue. If you wish you can use it directly, and some people have done so successfully[1]. But you must be aware that on Windows it only supports `select()`.

The async dispatcher implements the proactor pattern and also has an implementation of IOCP. It implements the proactor pattern for other OS' via the selectors module. Futures are also implemented here, and indeed all the procedures return a future.

The final layer is the async await transformation. This allows you to write asynchronous code in a synchronous style and works similar to C#'s await. The transformation works by converting any async procedures into an iterator.

This is also single threaded and fully non-blocking and does give you a lot of control. In theory you should be able to work with any of these layers interchangeably (as long as you only care about non-Windows platforms).

1 - http://goran.krampe.se/2014/10/25/nim-socketserver/

Re:Goroutines, Nonblocking I/O, and Memory Usage by steveklabnik on Friday January 06, 2017 @11:36AM (#13337664)
Yeah we map the IOCP model to a readiness one, so it's totally platform independent. Wonder if you all could do the same.
Re:Goroutines, Nonblocking I/O, and Memory Usage by dom96 on Friday January 06, 2017 @02:47PM (#13339554)
I believe we already do the same thing. As I've said, we use the proactor pattern and map it onto the readiness one.
Re:Goroutines, Nonblocking I/O, and Memory Usage by lobster_johnson on Thursday January 05, 2017 @06:56PM (#13332651)
Interesting solution!

I like futures, though cancelation has historically been a controversial topic there, too (at least in the JavaScript world; JS promises are not cancelable, which is problematic for HTTP requests [1]).

I do like that you guys chose to not make green threads a language-level construct (as Go did with "go" and goroutines), which means you have the option of supporting multiple models, for the programmer to choose according to their particular use case.

[1] https://github.com/whatwg/fetch/issues/27

Re:Goroutines, Nonblocking I/O, and Memory Usage by steveklabnik on Thursday January 05, 2017 @06:59PM (#13332675)
Cancellation falls out of the model: you drop the future, and it Just Works. (drop being the regular old Rust destructor stuff, which happens when it goes out of scope, or when you call std::mem::drop.)

Yup! The danger there is that the ecosystem may fragment, however, basically everyone is rallying around tokio, including the other people who were working on similar libraries, so in practice, this should be fine.