News for hackers. Stuff that matters.
Re:GoBGP: BGP Implemented in Go by 15155 on Monday April 18, 2016 @02:17PM (#11522010)
RE: the (or Rust) comment-

All of these services would be better suited for a language which has blessed async IO, concurrency, parallelism primitives. Go has these, Rust does not. pthreads are not the answer.

I much prefer Rust, but Go's stdlib and concurrency features far exceed that of Rust at the moment.

Re:GoBGP: BGP Implemented in Go by pcwalton on Monday April 18, 2016 @05:37PM (#11523347)
> All of these services would be better suited for a language which has blessed async IO, concurrency, parallelism primitives. Go has these, Rust does not. pthreads are not the answer.

Go does not have truly async I/O. It has a userspace (M:N) implementation of threaded, synchronous I/O. There is a distinction, and it's not an academic one.

Rust uses the kernel-level (1:1) implementation of threaded, synchronous I/O, with async I/O provided by mio if you want it.

There is no meaningful distinction between Go's language-level primitive channels and Rust's MPSC channels in the standard library. Not supporting generics is a good reason to put channels directly in the language, but that doesn't apply to Rust.

Additionally, I don't see how Go provides any parallelism primitives that Rust doesn't. In fact, Rust's parallelism (particularly data parallelism) libraries far exceed the capabilities of Go's, mostly because of SIMD and generics which allow you to build highly optimized data parallel abstractions. If I tried to parallelize the project I work on in Rust using Go's built-in primitives, it would be far slower than sequential.

Re:GoBGP: BGP Implemented in Go by tptacek on Monday April 18, 2016 @08:38PM (#11524125)
Can you be a bit clearer on the distinction here? It's "synchronous" in the sense that the code is straight-line synchronous, but threads are scheduled in part based on I/O events (IIRC, this used to be the primary way threads were scheduled).

Yes, every connection has a (small) thread stack associated with it. But in a "truly asynchronous" network program, every connection still has memory associated with it; it's just that the memory doesn't take the form of a procedure stack.

Re:GoBGP: BGP Implemented in Go by pcwalton on Monday April 18, 2016 @10:52PM (#11524595)
> Yes, every connection has a (small) thread stack associated with it. But in a "truly asynchronous" network program, every connection still has memory associated with it; it's just that the memory doesn't take the form of a procedure stack.

That's also true with 1:1 threading. It's just that the context switching is handled by the kernel.

Semantically, there's no difference between what Go does and what NPTL does. The difference is in implementation: Go does a lot of the work itself in userspace, while NPTL does the work in the kernel. (I say NPTL not to be pedantic but because there were pthreads implementations in Linux that used Golang-like schedulers. They were abandoned in favor of NPTL because the extra complexity was judged to not be worth it for small if any performance gains.)

You're right of course that you need per-connection state in any model. But with a state machine you can be much more compact than a call stack. Modern compilers (I doubt this includes Go 6g/8g, but haven't checked) will do stack coloring to reduce stack usage, but the overhead is still significant because compilers essentially always choose runtime performance of straight-line code over stack compactness wherever there's a tradeoff. State machine compilers, like C#'s async/await compiler, make the opposite choice, and as a result they can use less memory. Moreover, with state machines you can go the extra mile and really pack your state into a tiny fixed-size allocation you can allocate with a segregated fit arena. That's pretty much unbeatable for performance.

Re:GoBGP: BGP Implemented in Go by steveklabnik on Monday April 18, 2016 @02:34PM (#11522134)
In general, almost every stdlib will be "better" than Rust's, as we're taking an "anti-batteries included" approach.

And while Go does have great built-in stuff for a certain kind of concurrency, Rust's approach is more flexible and safer. It's a tradeoff, not a "far exceed" in my mind.

Re:GoBGP: BGP Implemented in Go by 15155 on Monday April 18, 2016 @02:42PM (#11522183)
Isn't the Rust approach the Ruby, Python, or C++ approach?: give pthreads, rely on the community to produce a fragmented, incompatible ecosystem.
Re:GoBGP: BGP Implemented in Go by steveklabnik on Monday April 18, 2016 @03:02PM (#11522316)
The Rust approach is, Rust's safety guarantees work for concurrency, but the details of that concurrency are left up to libraries, not the language. Since the safety is in the language, things are always safe, but you get the flexibility to do what you need.

You have to remember, Rust is a systems language. Which means that you need access to what the system gives you, and that means at least OS threads and both synchronous and asynchronous IO. We can't just decree "the world must use only aio and green threads", or we would be compromising Rust's fundamental design goals.

Re:GoBGP: BGP Implemented in Go by wolf550e on Monday April 18, 2016 @03:38PM (#11522557)
That's excellent and must be this way to allow programming microcontrollers or linux kernel modules in Rust, but you guys also need a blessed cross platform N:M threading async I/O sockets and files library, so people can write snmp/mail/web/etc servers/proxies/etc in Rust.
Re:GoBGP: BGP Implemented in Go by pcwalton on Monday April 18, 2016 @05:41PM (#11523364)
Few C implementations of these applications use M:N threading. M:N threading was tried early on in Linux's history and was abandoned for a reason. It is not a requirement for implementing those applications; in fact, it'll always be suboptimal from a performance point of view, especially in low-level languages like Rust. (Note that I'm not saying it's not fast enough for most applications, or that it was the wrong choice for Go, just that it's not optimal.)

I think the right solution is something like async/await to make truly asynchronous programming palatable. But in the meantime, 1:1 threading is really not that bad on Linux, because the kernel is very optimized.

Re:GoBGP: BGP Implemented in Go by jacobparker on Monday April 18, 2016 @07:05PM (#11523742)
(Off-topic) what do you think about marshalling for continuations from await? So the issue where on one hand it's confusing/surprising if you don't default to marshalling the continuation back to the thread (etc.) that created the continuation (e.g. the UI thread) but if you do then mixed blocking and async this can result in deadlocks (contention on the thread/etc.)

.NET defaults to this, you can opt out (which you usually want to do, at least for correctness) by doing await Foo().ConfigureAwait( continueOnCapturedContext: false )

EDIT: my etc.'s are weaseling around the word thread... more details about .NET here: http://blogs.msdn.com/b/pfxteam/archive/2012/04/12/async-awa...

Re:GoBGP: BGP Implemented in Go by steveklabnik on Monday April 18, 2016 @04:06PM (#11522779)
So, mio is certainly becoming the core library everyone uses for async I/O, but N:M threading is more complex, and doesn't have any libraries that are mature enough yet to start consolidating. I think AIO is more important than the threading model for this kind of thing, personally, but it's also not exactly my area of expertise.
Re:GoBGP: BGP Implemented in Go by 15155 on Monday April 18, 2016 @07:31PM (#11523846)
Can mio be pulled into the stdlib, then?

I fear a world of gevent/Twisted, EventMachine/Celluloid/base Ruby, Boost/a litany of other options.

Re:GoBGP: BGP Implemented in Go by pcwalton on Monday April 18, 2016 @08:26PM (#11524078)
I wouldn't mind pulling it into the standard library if we're sure it's mature enough to be ready. In the meantime, though, it's the de facto standard async I/O framework, and I don't see that changing anytime soon.
Re:GoBGP: BGP Implemented in Go by steveklabnik on Monday April 18, 2016 @09:08PM (#11524247)
Agree 100%.
Re:GoBGP: BGP Implemented in Go by pcwalton on Monday April 18, 2016 @05:43PM (#11523371)
That hasn't been the case. In practice, mio is the standard for everything asynchronous in Rust.

Adopting Go's approach would force either asynchronous I/O or M:N threading on everyone, which is unacceptable for Rust's goals.

Re:GoBGP: BGP Implemented in Go by im_down_w_otp on Monday April 18, 2016 @04:04PM (#11522765)
You want ponylang.org

Concurrency & parallelism primitives + safety.