News for hackers. Stuff that matters.
Re:Why Rust? [pdf] by acconsta on Wednesday September 09, 2015 @02:52PM (#10193407)
The Rust community makes a car:

"Check out our awesome new car! It makes driving 'easy and safe'!"

"Does it prevent crashes?"

"No, that's impossible! But here, look at our onboard computer that prevents many types of driving errors."

I like Rust. I want it to succeed. But I think the rhetoric gets ahead of the language sometimes, and not prioritizing higher level concurrency tools because you have the borrow checker is a mistake.

Re:Why Rust? [pdf] by kibwen on Wednesday September 09, 2015 @03:30PM (#10193756)
It's incorrect to say that Rust hasn't prioritized higher-level concurrency tools. Many of the design decisions over the previous years have been made with an eye towards supporting every concurrency paradigm.

For example, here's a post of Niko Matsakis' from Feb 2014 about changing a fundamental aspect of the borrow checker in order to better support data parallelism in the future: http://smallcultfollowing.com/babysteps/blog/2014/02/25/rust...

For another example, here's an RFC from Nov 2014 about fundamentally altering Rust's concurrency support in order to better support fork-join parallelism in the future: https://github.com/rust-lang/rfcs/blob/master/text/0458-send...

I'm also confused about your perception that the rhetoric gets ahead of the language. The type system does indeed make things easier and safer. When people ask what that means, we're eager to elaborate on the precise guarantees that Rust provides. Misleading people as to Rust's capabilities is not on the agenda.

Re:Why Rust? [pdf] by acconsta on Wednesday September 09, 2015 @04:13PM (#10194219)
>It's incorrect to say that Rust hasn't prioritized higher-level concurrency tools

I think it's a fair characterization given 1.0 shipped without them, and there doesn't seem to any timeline for standardization (please correct me if I'm wrong).

I don't think I have to tell you that concurrency is one of the most important challenges in modern programming. But all Rust gives you today (and for the foreseeable future) is a pthreads wrapper.

>I'm also confused about your perception that the rhetoric gets ahead of the language

The parent comment says verbatim "Rust goes out of its way to make it easy and safe to write multithreaded code". This apparently doesn't include preventing deadlocks, a problem that is common, hard-to-avoid, and difficult to recover from. Does that not strike you as a bit of an overreach?

Re:Why Rust? [pdf] by the_why_of_y on Wednesday September 09, 2015 @07:09PM (#10195263)
> deadlocks, a problem that is common, hard-to-avoid, and difficult to recover from

Deadlocks are by far the easiest concurrency problem to debug, since it's very obvious when your program is deadlocked, and in most cases a stack trace of the involved threads is sufficient to debug and fix a deadlock. Also, if your program is deadlocked it won't corrupt user data.

Rust's std::sync::Mutex is fortunately non-recursive, which makes it easier to find deadlocks during testing.

Data races are far worse since they may cause arbitrary effects at a later point in program execution, so they take a lot of developer time to track down and very likely lead to data corruption.

Re:Why Rust? [pdf] by acconsta on Wednesday September 09, 2015 @09:05PM (#10195627)
If deadlocks are so easy to fix why do they happen so often? Seriously, how many applications have you seen hang in your life? (not to discount the role of other race conditions in causing hangs, which the borrow checker also can't categorically prevent).
Re:Why Rust? [pdf] by neandrake on Thursday September 10, 2015 @12:04AM (#10196085)
I believe the comment reads that deadlocks are the "easiest concurrency problem" to debug -- but your question is moreso headed to the point that they are easy to introduce in a system.

From the example, from a deadlock the locked threads and their stacktraces can be determined, which will help in pointing towards where the cause of the problem is. Compared to situations where the problem occurs due to a data-race causing an unexpected/invalid state but a problem doesn't manifest until later. Fixing this type of problem is more problematic as usually any exception/stacktrace that might crop up does not relate any information as to how the state became invalid in the first place. I tend to see these types of bugs more than deadlocks and in my experience they're always more involved in debugging compared to deadlocks.

Re:Why Rust? [pdf] by GFK_of_xmaspast on Wednesday September 09, 2015 @11:35PM (#10196036)
Null pointer derefs are easy to find and fix and : yet.
Re:Why Rust? [pdf] by neandrake on Thursday September 10, 2015 @12:18AM (#10196123)
This is not always the case. Null pointer derefs are just indicators that the program got into an invalid state. Is the fix to provide some alternative pathway when the program got in a bad state or is it to prevent the bad state from happening in the first place? If the fix is to prevent the bad state, tracking down how that state occurred in the first place can often be time consuming in the case of data races.
Re:Why Rust? [pdf] by steveklabnik on Wednesday September 09, 2015 @04:48PM (#10194470)

  > But all Rust gives you today (and for the foreseeable future) is a pthreads
  > wrapper.
... Plus significant static guarantees about the safety of using said wrapper.
Re:Why Rust? [pdf] by acconsta on Wednesday September 09, 2015 @05:49PM (#10194863)
And that's a great achievement! If there were a Nobel prize for practical use of a type system, the Rust team would get it.

But overselling it ("Rust goes out of its way to make it easy and safe to write multithreaded code" except oh yeah it can deadlock at any time) is a mistake.

Re:Why Rust? [pdf] by kibwen on Wednesday September 09, 2015 @08:07PM (#10195450)
You act as though deadlocks in Rust are trivial and pervasive, but they aren't. It just makes no guarantees that deadlocks will not occur. Show me a language that statically prevents deadlocks and I'll be quite curious to check it out. In any case, writing concurrent code in Rust is safe and easy. I suggest you try it. :)
Re:Why Rust? [pdf] by acconsta on Wednesday September 09, 2015 @09:05PM (#10195624)
>You act as though deadlocks in Rust are trivial and pervasive, but they aren't

Writing nontrivial multithreaded code using thread and mutex primitives is very hard to get right (not only due to data races, but also race conditions and deadlocks). Has your experience been different?

>Show me a language that statically prevents deadlocks

OK, idiomatic Go and Erlang will never deadlock. Sure, you can use mutexes in Go, but unlike in Rust they aren't the only means of achieving high levels of concurrency (in fact, they are explicitly discouraged).

I find this attitude from the Rust community disheartening. Not everything needs to be statically verified to be useful.

Re:Why Rust? [pdf] by kibwen on Thursday September 10, 2015 @12:09PM (#10198860)
You really must not have done much concurrent programming if you think Go and Erlang won't deadlock. :P Throwing the "idiomatic" qualifier on there isn't a defense; "idiomatic Rust" doesn't deadlock either.
Re:Why Rust? [pdf] by acconsta on Thursday September 10, 2015 @03:10PM (#10199927)
Let me be more precise then. Goroutines enable a fan out pattern in which each task, even small ones, is a separate asynchronous routine. Tasks fan out from a coordinator routine and their results fan back in:

https://talks.golang.org/2012/concurrency.slide#46

This is great for a couple reasons:

It's automatically concurrent without explicit pooling and locking.

The code flow remains sequential. No callback hell!

It reduces concurrency errors. The locking pattern is naturally acyclic! Nothing races!

Ah, but Go has a big runtime, you say! We can't have that in Rust! Well, here's the same thing in C and in C++:

http://libmill.org/tutorial.html

http://www.boost.org/doc/libs/1_59_0/libs/coroutine2/doc/htm...

Coroutines are a wonderful tool for building concurrent applications, and I dearly hope we get them in Rust.

Re:Why Rust? [pdf] by Manishearth on Thursday September 10, 2015 @08:55PM (#10201448)
Forget to update that 3 after fiddling with the channels? Deadlock.

Forget to unlock a mutex when dealing with shared data? Deadlock (and Go doesn't have RAII mutexes, so it's very easy to do this).

The pattern you put forth is possible in Rust too. Use mio if you want goroutine-like efficiency. Nothing new.

Re:Why Rust? [pdf] by acconsta on Friday September 11, 2015 @02:36PM (#10205398)
>Forget to update that 3 after fiddling with the channels? Deadlock

To be fair, how long would that take to debug?

>Forget to unlock a mutex when dealing with shared data?

You'll note there are no mutexes. RAII mutexes are great (although less useful without exceptions). But the entire point of Go's concurrency model ("share by communicating") is that you don't need need to deal with them.

>Use mio if you want goroutine-like efficiency

For a personal project, I would. But will any commercial entity use an unstable, unportable library with a single maintainer for critical functionality in their app? Because that's the concurrent IO situation in Rust, now and for the foreseeable future.

Re:Why Rust? [pdf] by dbaupp on Friday September 11, 2015 @08:30PM (#10206820)
Your example is tiny and trivial, the same thing written with locks/whatever would be equally easy to debug. Problems in a more complicated application would be harder to debug, no matter if you use either channels or locks.

> But the entire point of Go's concurrency model ("share by communicating") is that you don't need need to deal with them.

Exactly the same thing works in Rust, and works "better": the lack of other sharing (except by message passing) is enforced at compile time. Rust ensures that other options are available with as much help for correctness as possible.

> But will any commercial entity use an unstable, unportable library with a single maintainer for critical functionality in their app?

Concurrent IO is inherently unportable, and mio has support for the major platforms (OSX, Linux and Windows, with tests run on all) so I don't know what you mean by that. mio won't be the first or last lib with a single maintainer that a commercial entity uses.

(Unstability is of course a perfectly reasonable criticism, and I'm sure it'll disappear as the library ages.)

Re:Why Rust? [pdf] by acconsta on Saturday September 12, 2015 @12:23AM (#10207210)
>Your example is tiny and trivial, the same thing written with locks/whatever would be equally easy to debug.

Write the version with explicit locks and condition variables and we'll see if it's as easy to debug. :P

>Exactly the same thing works in Rust, and works "better": the lack of other sharing (except by message passing) is enforced at compile time.

Which is one of the reasons why Rust can (should) eat Go's lunch. All it's missing are lightweight coroutines and concurrent IO.

And I didn't know Mio supports Windows now — that's good news! If Mio stabilizes and Rust gets lightweight coroutines (probably requiring compiler support), Rust could be the best of all worlds.

Re:Why Rust? [pdf] by dbaupp on Wednesday September 09, 2015 @10:45PM (#10195915)
Both of those languages will semantically deadlock: forget to send a message on a channel and a different thread will sit there waiting forever. It's not a mutex-deadlock, but it achieves the same thing. Neither of those languages protects against this. (And Rust has exactly that level of "deadlock" freedom: it has channels too.)

In any case, you seem to be ignoring what everyone is saying: Rust doesn't guarantee deadlock freedom, but it still tries to help. Mutexes can be an important building for some things, but they're not the final story. There's atomics and channels in the standard library right now, and now that 1.0 is released, there'll be a growth of even better abstractions.

One of the people working full time on Rust has a PhD in concurrent programming, and has it as a personal goal to make Rust great at it. You can see his initial work on his blog[1], and read his thesis which introduces "reagents" (something he has expressed interest in implementing in Rust)[2].

1.0 is the start for Rust, not the end.

[1]: http://aturon.github.io/blog/2015/08/27/epoch/

[2]: https://www.mpi-sws.org/~turon/turon-thesis.pdf

Re:Why Rust? [pdf] by acconsta on Thursday September 10, 2015 @12:12AM (#10196110)
Putting the word "semantically" in front of deadlock doesn't change its meaning:

https://en.wikipedia.org/wiki/Deadlock

Either way, redefining the word does nothing to make mutexes and native threads safer or easier to use in practice.

I can't emphasize this enough — data races are not the only kind of concurrent programming error, yet they are the only kind prevented by borrow checking. Go, Erlang, and Node have high level approaches to concurrency that reduce other categories of errors (in addition to providing massive performance benefits vs. naive native threading).

I think the time to add good concurrency abstractions to a language is before 1.0, but I'm in strong disagreement with the community there. And judging by this thread, concurrent IO isn't even on the core team's roadmap. This is not a good sign for a language that sells itself on concurrency!

Look at the situation 10 months ago. How much has it improved?

https://www.reddit.com/r/rust/comments/2l0a4b/do_rust_web_se...

Channels are a step in the right direction, but are pretty limited without coroutines (channels without coroutines are just synchronized queues). Atomics are cool, but they address a totally different problem. Reagents, well, let's see an implementation.

C++ has needed a successor for many years now, and Rust is the best candidate. But lofty claims notwithstanding, the Rust concurrency situation is pretty dreadful.

Re:Why Rust? [pdf] by dbaupp on Thursday September 10, 2015 @01:09AM (#10196224)
> Putting the word "semantically" in front of deadlock doesn't change its meaning:

And focusing on hangs in concurrent programs that are caused by using a data structure called "mutex" doesn't stop one getting exactly the same symptoms via your apparently-perfect channels.

One can use channels to get all four of the conditions required for a deadlock, especially Go's synchronous-by-default channels.

Any time you have a protocol of multiple tasks communicating with each other in some structured way, it's possible to break that protocol and hence have tasks sitting around waiting for messages that aren't coming. Especially in languages like Go/JavaScript/... which aren't powerful enough to model things like session types in their type system, e.g.

https://www.reddit.com/r/rust/comments/3jhd56/session_types_...

> I can't emphasize this enough — data races are not the only kind of concurrent programming error, yet they are the only kind prevented by borrow checking

Yes, that's exactly why the whole Rust community tries to be very careful about using "data race" when talking about concurrency in Rust.

However, it is the case that data races are the worst sort of concurrency bug: they are undefined behaviour and so can lead to arbitrary memory corruption, possibly only appearing a long way from the actual place with UB. Deadlocks and other problems are, by default, much more controlled in their failure modes.

Rust focuses on truly outlawing large classes of horrible problems: dangling pointers, iterator invalidation, data races (and all without requiring a garbage collector, although a GC barely helps with the latter two). It also tries to help with other problems with fewer guarantees, but even just being memory safe is a huge step up from widely used low-level languages (i.e. C/C++).

All the languages you mention are quite opinionated in their concurrency, imposing costs that Rust doesn't and can't, for its target space. (And, Go certainly doesn't provide any guarantees at all, not even data race freedom.)

> in addition to providing massive performance benefits vs. naive native threading

Important qualification: for IO bound tasks. Which is perfectly fine, but it needs to be understood.

> I think the time to add good concurrency abstractions to a language is before 1.0, but I'm in strong disagreement with the community there

What's so important about being pre-1.0? You seem focused on it, but I don't understand why. What benefit does Rust gain by delaying the release of 1.0 for months/years just to get good async IO support? Why is this particular pet feature any more important than everyone else's pet feature? (There have been so many requests: "why couldn't X make it into 1.0?")

If you're concerned about theoretical fragmentation of the ecosystem... that's not a problem in practice: mio is the standard.

> And judging by this thread, concurrent IO isn't even on the core team's roadmap

Wrong, it's very much on the roadmap, e.g. Alex Crichton (core team member) has been adding windows support to mio himself.

> Look at the situation 10 months ago. How much has it improved?

A lot. There's a burgeoning ecosystem built around mio.

---

In any case, Rust has been stable for barely 3 months. Be patient, and give it time for the concurrency story to blossom from the seeds that have been sown so far. Based on the experience so far, I'm pretty confident that Rust can easily be much better (i.e. more performant and reliable) than both Node and Go and even Erlang. (Of course, it may be syntactically less nice, since those languages bake it in deeply, while Rust is less opinionated.)

Re:Why Rust? [pdf] by acconsta on Thursday September 10, 2015 @04:20AM (#10196752)
>your apparently-perfect channels

I've never used Go, but it has some interesting concurrency ideas. So do Node and Erlang. I naively expect Rust to adopt the best ideas from each.

>One can use channels to get all four of the conditions required for a deadlock, especially Go's synchronous-by-default channels.

One can, yes. But it's pretty easy to avoid cyclic locking patterns when each request is handled by a separate goroutine, as is idiomatic. One thread per request in Rust will drag pretty quickly.

Yes, the borrow checker is an impressive achievement. But is it enough for Rust to succeed? Marketing yourself as safer C++ is what Java already did (with tremendous success) 20 years ago. And the market for systems languages has only shrunk since then (my phone runs Java).

>All the languages you mention are quite opinionated in their concurrency, imposing costs that Rust doesn't and can't, for its target space.

And yet Rust has already partially standardized channels. Finish the channels, add coroutines and you've implemented Go! (I'll note there are already coroutine implementations for C and C++, which do not limit their use as systems languages).

>(And, Go certainly doesn't provide any guarantees at all, not even data race freedom.)

Which, interestingly, hasn't hindered its ability to become a successful language! A lesson worth remembering.

>Wrong, it's very much on the roadmap, e.g. Alex Crichton (core team member) has been adding windows support to mio himself.

I mean... https://github.com/carllerche/mio/graphs/contributors?from=2...

Could be worse, could be better?

>Why is this particular pet feature any more important than everyone else's pet feature?

I don't think "good concurrency is a pet feature" is the winning argument here.

>I'm pretty confident that Rust can easily be much better (i.e. more performant and reliable) than both Node and Go and even Erlang.

Me too! But I'm not sure that's going to happen with a single core developer on Mio and no timeline for standardization.

Re:Why Rust? [pdf] by kibwen on Thursday September 10, 2015 @12:30PM (#10199017)

  > I've never used Go, but it has some interesting 
  > concurrency ideas. So do Node and Erlang. I naively 
  > expect Rust to adopt the best ideas from each.
This is where your naivete shows. Rust originally did have the same thread model as Go baked into the language and standard library, and it labored for years to find a usable compromise between Go's green thread model and the native threading model. And a compromise is indeed necessary, firstly because we don't just need another Go, and secondly because Go's threading model imposes horrific costs when trying to interoperate with non-Go code (literally thousands of times the overhead that you'd expect). For a language like Rust that intends to interoperate with the native ecosystem, that overhead is unacceptable. After about three or four complete redesigns and rewrites the entire green threading infrastructure was chucked to the curb. Fortunately, Rust is low-level enough that libraries like mio can pick up the slack on their own, and in the meantime libraries that don't need green threads don't have to pay the price.
Re:Why Rust? [pdf] by acconsta on Thursday September 10, 2015 @05:57PM (#10200806)
>Rust originally did have the same thread model as Go baked into the language and standard library

Ehhhhhh not quite. Go provides one threading API, and it's green threading. Rust tried to provide the both green threading and native threading using identical APIs. That was a unique and in retrospect quixotic decision. Most of the problems identified in the RFC stem from the unified API issue:

https://github.com/rust-lang/rfcs/blob/0806be4f282144cfcd55b...

I also understand Rust's implementation was backed by libuv, which, being designed for Node, was a poor fit for Rust:

https://plus.google.com/+nialldouglas/posts/AXFJRSM8u2t

You're the third person in this thread to tell me that Go-like concurrency requires a big runtime, and it remains false. Here are analogous concurrency implementations in C and C++:

http://libmill.org/tutorial.html

http://www.boost.org/doc/libs/1_59_0/libs/coroutine2/doc/htm...

And because of that fallacy, the future of concurrency in Rust is a one-man show, third party library. It's a tremendous loss.

Re:Why Rust? [pdf] by dbaupp on Thursday September 10, 2015 @07:21PM (#10201156)
The same approach can and does exist in Rust too, e.g. https://crates.io/crates/mioco .

(Concurrency in C and C++ is also third-party libraries... The whole point of languages like Rust and C++ is that powerful functionality like this can be built externally, so that different trade-offs can be made. Languages like Go and Node force one approach, and so when you need something outside it, you're forced to do something suboptimal.)

Re:Why Rust? [pdf] by acconsta on Thursday September 10, 2015 @08:08PM (#10201303)
>The same approach can and does exist in Rust

Without the documentation, stability, portability, quality guarantees, and compiler support (that's a big one — code generation for coroutines needs to be good) of a standard library.

>Concurrency in C and C++ is also third-party libraries

C++ is on track to standardize concurrent file and network IO. Draft specifications have already been published, and Microsoft shipped coroutines in VS 2015. It would be a damn shame if C++ got concurrent IO before Rust.

I would like to use Rust professionally, and I'm sure you do/would as well. But no one can possibly sell their boss on using a project with a single part time maintainer to provide critical functionality.

Re:Why Rust? [pdf] by dbaupp on Thursday September 10, 2015 @09:52PM (#10201582)
The C/C++ libraries you're holding up as examples get no compiler support.

That said, I do think a much better style of coroutines for Rust would be a C#-esque async/await transformation, converting stack-frames/local variables into an enum, allowing literally zero-cost coroutines (all the state is stored inline, no need to allocate a separate stack). Relevant issues:

- https://github.com/rust-lang/rfcs/issues/388

- https://github.com/rust-lang/rfcs/issues/1081

I'm pretty sure this is quite non-trivial to implement automatically.

---

C++ has had 20 years of stability, Rust only 3 months. Rust will get concurrent IO before C++ has on that time scale.

The goal with 1.0 was to stabilise enough of the language that people can start using it to write libraries that work into the forseeable future, allowing them to seriously explore the space of, for example, concurrent IO in Rust. Once enough exploration has been done (maybe you think enough has been done for async IO now), the functionality can start to become more official.

Re:Why Rust? [pdf] by acconsta on Friday September 11, 2015 @04:15PM (#10205924)
>The C/C++ libraries you're holding up as examples get no compiler support.

You can open VS 2015 today and use C++ coroutines backed by Microsoft (and their compiler, which is developed alongside their standard library).

And I am by no means saying that Goroutines are the final story in concurrent IO. Stackless coroutines in Rust would be a dream.

>C++ has had 20 years of stability, Rust only 3 months. Rust will get concurrent IO before C++ has on that time scale.

Concurrent IO is a hell of lot more important than it was in the 1990s, and the relative timescale is irrelevant for people choosing between Rust and C++ today (or Go, Scala, Clojure, C#, etc.).

>Once enough exploration has been done (maybe you think enough has been done for async IO now),

Exactly the opposite — I think the number of developers working on this (the Mio author plus Alex Crichton, maybe some offshoots) is far too few.

And the attitude I'm seeing from some core developers in this thread (concurrent IO is a "pet" feature that the community will someday deliver fully formed and ready for "blessing") is a huge disappointment.

Re:Why Rust? [pdf] by dbaupp on Friday September 11, 2015 @08:19PM (#10206801)
Rust is targeting more than one domain. Concurrent IO is not important in all domains. I'm sure the domains you work in need it a lot, but that's not the whole world. (This is all I mean by "pet feature".)

There are cross-cutting concerns that apply to everything (including concurrent IO libraries) that development work is focusing on. Rust doesn't need to eat Go's/Scala's/Clojure's... lunch 3 months after it was released, taking a year or two to settle in and branch out seems fine to me.

Re:Why Rust? [pdf] by acconsta on Saturday September 12, 2015 @12:12AM (#10207191)
>Rust is targeting more than one domain. Concurrent IO is not important in all domains.

Well, what domains is Rust targeting? Rust still isn't a good fit for embedded/kernel stuff without allocators and OOM handling (not to mention every architecture not targeted by LLVM).

That leaves userland applications, and how many applications don't need concurrent IO?

I'm definitely being impatient, and I'm sorry for that. It's just frustrating to see only one core member working on it.

Re:Why Rust? [pdf] by dbaupp on Saturday September 12, 2015 @01:57AM (#10207411)
The standard library is different to the language itself: the language certainly doesn't disallow allocators/OOM handling, it's just the design of std. Furthermore, the standard library is layered: there's `std` with various OS-required routines (IO, etc.) and `core` that is the core stuff that doesn't require any of that. Operating systems/embedded applications can generally use `core`. However, even that is better than C/C++, where one usually has a from-scratch "standard" library (which is possible to do with Rust too): not being able to use the compiler-bundled `std` doesn't seem like a point against Rust.

Many user-space applications don't do heavy network work, which is where concurrent IO is most necessary. E.g. games or a scientific simulation, even web-browsers don't need of concurrent IO (they're not trying to juggle thousands of connections).

Re:Why Rust? [pdf] by acconsta on Saturday September 12, 2015 @02:23PM (#10208824)
>Operating systems/embedded applications can generally use `core`.

Which doesn't, correct me if I'm wrong, provide a stable allocator API or OOM handling yet.

>E.g. games

When your client pings 400 servers, how is it doing that? How is the game server implemented?

>a scientific simulation

That runs on one machine, and doesn't do a lot of disk IO? (concurrent file IO matters too).

>even web-browsers

Open the Chrome dev tools "network" tab, and then open Gmail. How many requests did it make?

Again, concurrent IO is important.

Re:Why Rust? [pdf] by openasocket on Wednesday September 09, 2015 @05:25PM (#10194734)
FYI, detecting deadlocks in the general case is in-computable, whether at run time or compile time.
Re:Why Rust? [pdf] by heinrich5991 on Wednesday September 09, 2015 @07:03PM (#10195232)
Stuff like the borrow-checker in the general case is also not computable, only for a small subset of programs, e.g. Rust programs.
Re:Why Rust? [pdf] by kibwen on Wednesday September 09, 2015 @04:59PM (#10194547)

  > I think it's a fair characterization given 1.0 shipped 
  > without them, and there doesn't seem to any timeline for 
  > standardization (please correct me if I'm wrong).
The release of 1.0 was not an indication that the language was 100% complete, or that the stdlib was 100% comprehensive. The 1.0 release represented a stable foundation upon which to build an ecosystem. Going forward, the Rust developers absolutely do care about providing more concurrency primitives. Here's Aaron Turon's recent work on implementing epoch-based memory reclamation for implementing lock-free data structures: http://aturon.github.io/blog/2015/08/27/epoch/ . Of the two interns the Rust project was granted this summer, one of them spent the entirety of their time working on supporting SIMD in the language (http://huonw.github.io/blog/2015/08/simd-in-rust/) while the other spent their time specifying the behaviors that are allowed inside Rust's `unsafe` blocks, which includes taking a good long stare at Rust's memory model and all its concurrency primitives and making sure that they're sound. While we're on the topic of interns, you're also overlooking the `Arc` pointer in the stdlib, which is an intern project from more than three years ago which allows one to safely share memory between threads. Meanwhile, pcwalton (Rust core team member and full-time Servo developer) has been working on shmem support and multiprocess allocators for Servo. In addition, the Rust stdlib contained an implementation of fork/join prior to 1.0, but at the last minute the API was found unsound for a few edge cases and so it was deprecated and a working implementation was deferred until later (today there are at least two working reimplementations of this on crates.io with the unsoundness fixed).

  > This apparently doesn't include preventing deadlocks, a 
  > problem that is common, hard-to-avoid, and difficult to 
  > recover from. Does that not strike you as a bit of an 
  > overreach?
Not at all. Statically preventing data races is an enormous leap forward in the state of the art. Here's a quote from Matthias Felleisen of Northeastern University on teaching Rust to students without experience in concurrent programming: https://www.youtube.com/watch?v=JBmIQIZPaHY&feature=youtu.be...

"I had them program parallel programs in Rust, and as some of you know, Rust prevents race conditions with its type checking. [Note that he's incorrect here, Rust prevents data races, not race conditions in general.] I will admit, the first two weeks, because we used the beta release, was a mess. We couldn't understand the type error messages. But once we got over the hump, I was blown away, that these kids never had problems writing parallel programs in imperative style. There were no race conditions. The type system slapped their fingers. We taught them how to design, the type system enforced it, and lo and behold, I hate to admit this but I have to admit it, they just didn't have problems with this stuff."

Re:Why Rust? [pdf] by acconsta on Wednesday September 09, 2015 @06:25PM (#10195067)
SIMD, lock free data structures, and ARC pointers are great. But what's the timeline for concurrent IO (either async or coroutines)? Preferably one that is stable, portable, and efficient (mio isn't the first two).

>Statically preventing data races is an enormous leap forward in the state of the art

The borrow checker is a great achievement. But it doesn't prevent deadlocks. And because Rust doesn't provide higher level concurrency tools than threads and mutexes, deadlocks are going to be a significant problem in practice until such tools are standardized.

Re:Why Rust? [pdf] by burntsushi on Wednesday September 09, 2015 @06:35PM (#10195113)
> But it doesn't prevent deadlocks.

Could you describe how you've developed the misconception that Rust does prevent deadlocks? Whatever it is, we should try to fix it right away, because Rust has never tried to prevent deadlocks, and suggesting that it does is not good.

> And because Rust doesn't provide higher level concurrency tools than threads and mutexes

Rust's standard library has channels.

Re:Why Rust? [pdf] by acconsta on Wednesday September 09, 2015 @07:17PM (#10195296)
This is all a response to:

https://news.ycombinator.com/item?id=10192042

Reasonable people can disagree about what "safe" and "easy" mean, but I don't think Rust's concurrency primitives are either given the (very real, very bad) possibility of deadlock.

>Rust's standard library has channels.

Which are a good start, but only a partial solution without coroutines (and the other additions you mentioned).

Re:Why Rust? [pdf] by burntsushi on Wednesday September 09, 2015 @07:31PM (#10195344)
> but I don't think Rust's concurrency primitives are either given the (very real, very bad) possibility of deadlock.

OK. Then I don't know what to say. An infinite loop is a deadlock. Unless you have a programming language that can guarantee termination, you can't prevent deadlocks statically.

So yes, I think a brief marketing pitch of "safe and easy concurrency" is totally appropriate given the domain Rust is shooting for. In fact, it is one of Rust's strengths, so to not advertise it as such would be quite odd.

> Which are a good start, but only a partial solution without coroutines (and the other additions you mentioned).

OK. But you said "And because Rust doesn't provide higher level concurrency tools than threads and mutexes" which just isn't true. I'm trying to clear up what it is Rust does have. I'm not saying it has everything. Writing software takes time.

Re:Why Rust? [pdf] by acconsta on Wednesday September 09, 2015 @08:35PM (#10195543)
>An infinite loop is a deadlock

It isn't. I don't know what else to say:

https://en.wikipedia.org/wiki/Deadlock

Re:Why Rust? [pdf] by steveklabnik on Wednesday September 09, 2015 @02:58PM (#10193461)

  > not prioritizing higher level concurrency tools because you have the
  > borrow checker is a mistake.
Ahh, but to mix up your analogy here, the borrow checker is the engine, and the higher level tools are like building fancier cars. You have to get your foundations built before you can build higher-level things on top of them.

Now that we have a foundation, these things are starting to appear. See https://news.ycombinator.com/item?id=10131429, for example.

Re:Why Rust? [pdf] by acconsta on Wednesday September 09, 2015 @04:12PM (#10194204)
In general, there are many reasons (fragmentation, quality, portability, dependency management, compiler support) it's preferable to have concurrency tools standardized. Standardization is only going to get harder in the future.

Specifically, it's great that Rust has a capable atomics library. I'm sure we'll see many good concurrent data structures come from it. But that's not quite what I meant by concurrency tools. The thread primitives Rust offers are the same ones POSIX standardized twenty years ago. It would be very valuable to have something like OpenMP (parallel loops, etc.) and something for concurrent IO (either asynchronous or coroutine-based).

There's no reason Rust can't have the performance of C++ and the easy concurrency tools of Go or Erlang.

Re:Why Rust? [pdf] by steveklabnik on Wednesday September 09, 2015 @04:38PM (#10194407)
We do have libraries for asynchronous io, and co routines, they're just not ready yet. I absolutely agree with you that these things are useful, but the code needs to be written by someone. We're just at the stages where these things are getting good enough to try out.

(And the language can't have exactly the same stuff as Go or Erlang without adding a runtime, which is contrary to the goals of the project.)

Re:Why Rust? [pdf] by acconsta on Wednesday September 09, 2015 @05:45PM (#10194842)
That's great. I look forward to seeing them evolve and hopefully become standardized.

>(And the language can't have exactly the same stuff as Go or Erlang without adding a runtime, which is contrary to the goals of the project.)

Here is 90% of what Go gives you as a C library. No heavyweight runtime necessary:

http://libmill.org/

Re:Why Rust? [pdf] by burntsushi on Wednesday September 09, 2015 @06:58PM (#10195204)
> Here is 90% of what Go gives you as a C library. No heavyweight runtime necessary

Rust has channels in its standard library. Admittedly, they do not have the same functionality as Go's concurrency primitives. The two most significant omissions are probably a multi-producer/multi-consumer channel and a more complete (and stable) `select` construct.

There is also my `chan` library, which replicates Go functionality with respect to channels: http://burntsushi.net/rustdoc/chan/ --- Notably, you still have to use native threads.

Your `libmill` example is a coroutine library with a scheduler and everything. That is definitely too much runtime for standard Rust. However, there are people working on coroutines in Rust, on which something like `libmill` could be built: https://crates.io/search?q=coroutine

Re:Why Rust? [pdf] by acconsta on Wednesday September 09, 2015 @08:15PM (#10195484)
>That is definitely too much runtime

Is it incompatible with native threads? (no) Does it affect non-coroutine functions? (no) Does it have any effect whatsoever when not using the library? (no)

I encourage you to look over the code before dismissing it as "definitely too much runtime."

Re:Why Rust? [pdf] by burntsushi on Wednesday September 09, 2015 @08:43PM (#10195567)
> I encourage you to look over the code before dismissing it as "definitely too much runtime."

I encourage you to take a look at ongoing work on coroutines in Rust, which was the part of my comment where I didn't dismiss your point.

I think the portion of my comment you quoted was taken out of context.

Re:Why Rust? [pdf] by Manishearth on Wednesday September 09, 2015 @04:37PM (#10194399)
>It would be very valuable to have something like OpenMP (parallel loops, etc.) and something for concurrent IO (either asynchronous or coroutine-based).

https://crates.io/crates/mio for the async IO.

There was another library out there that provided tons of concurrency utilities, but I can't find it now. Parallel loop-like syntax should be easy with scoped_threadpool and a macro though.

Rust tries to avoid stuffing everything into the standard library. So the lack of utilities in the stdlib isn't an artifact of some ignorance of concurrency, it's because Rust doesn't want to keep everything in the stdlib. Rust keeps the basic framework for thread safety in the stdlib (though it doesn't need to, not exactly), but the rest is built upon by the community.

Re:Why Rust? [pdf] by 15155 on Wednesday September 09, 2015 @09:16PM (#10195657)
What happens when one has 10 different ways of handling async IO? Massive ecosystem fragmentation.

Ruby has EventMachine, the standard lib, and Celluloid. None of these are interoperable.

Python has the standard lib, Twisted, and a number of other projects. All have the same problems.

C++ has Boost, Asio, a few others. This is more along the lines of where Rust is headed.

C has /countless/ options. None of them are standard, and it's totally understood in such an old language.

---

Go has goroutines: it's a harmonious, unified ecosystem. I greatly dislike go, but this is one thing they do absolutely correctly.

Erlang has fully-preemptive multitasking (which the entire ecosystem is built upon).

Haskell has incredible parallelization and concurrency primitives built right into the stdlib.

Certain things belong in the stdlib. Async IO is definitely one of them. If not an implementation, a defined, common interface of some nature, to help prevent some of the fragmentation.

async/await, if I am not mistaken, will require compiler and borrow-checker support. This would be a good start.

Re:Why Rust? [pdf] by acconsta on Wednesday September 09, 2015 @06:21PM (#10195042)
>Rust tries to avoid stuffing everything into the standard library

That's a valid philosophy, but also one that leads to problems with fragmentation, quality, portability, dependency management, and compiler support.

Concurrent IO is surely important enough to standardize.

Re:Why Rust? [pdf] by burntsushi on Wednesday September 09, 2015 @06:51PM (#10195176)
> That's a valid philosophy, but also one that leads to problems with fragmentation, quality, portability, dependency management, and compiler support.

No, not necessarily. We would like for the standard library to remain minimal, but one of its important functions is to collect common interfaces to maximize interoperability between crates. This has worked well in practice so far.

Similarly, the standard library provides portable facades on top of platform specific APIs, for example, for performing IO. Crates can take advantage of this so that they can be portable themselves. Moreover, crates themselves can also provide portable facades over platform specific APIs, so I'm not convinced that this will be a problem in practice.

Dependency management is handled quite well by Cargo. It has been a wonderful tool to have at our disposal and is really the crux of what makes a small standard library possible.

Compiler support is a good argument, but one that I hope becomes weaker in time as we stabilize more functionality.

To be clear, I agree that a small standard library has its own downsides. In particular, quality is IMO on of the best arguments against a small standard library. Fortunately, we trying to mitigate this by adopting officially blessed libraries into the `rust-lang` organization: https://github.com/rust-lang/rfcs/blob/master/text/1242-rust... --- This allows us to avoid the problems with a big standard library (too much stuff that is hard to evolve because of stability) while still providing quality with crates that we promise to maintain.

Re:Why Rust? [pdf] by 15155 on Wednesday September 09, 2015 @09:18PM (#10195662)
> so I'm not convinced that this will be a problem in practice.

Historically, it's been a massive problem. See my other post. Rust is already seeing IO-related fragmentation.

C, C++, Ruby, Python, etc. are all massively-fragmented ecosystems regarding IO, concurrency, (safe) parallelism. I don't think we need to soil a great language (Rust) with these same mistakes.

Re:Why Rust? [pdf] by burntsushi on Wednesday September 09, 2015 @09:28PM (#10195693)
I was specifically speaking about the size of Rust's standard library. Python, at least, has a massive standard library, so I'm not sure how that's applicable here. (Or, at the very least, demonstrates that a big standard library isn't sufficient. What matters is what is in the standard library, which is not fundamentally incompatible with a small standard library.)

Also, none of those languages started out with a tool like Cargo.

I made a few other comments about mitigating this as well that should be considered.

Re:Why Rust? [pdf] by 15155 on Wednesday September 09, 2015 @09:35PM (#10195715)
I don't agree with the idea that Cargo is just going to magically make all fragmentation disappear, simply because it's convenient, good, and available. If this were true, I would suspect we'd have seen some of this fragmentation stop in these other ecosystems: it hasn't, despite excellent tooling.

I'm not suggesting a Python-esque stdlib. I'm suggesting a multi-threaded, cross-platform, (ideally, edge-triggered) event system, above which higher level primitives can be introduced and safely interoperate.

If Rust already has std/net, this is not that far of a gap to close. Granted, implementing a reactor or (higher level) green threads greatly affects the way your programs execute, in my opinion, the benefits of a "blessed way" would outweigh the problems.

Also, as with all of those languages with fragmented ecosystems: nothing is preventing a developer from implementing their own solutions, they're just heavily encouraged to be compatible.

Re:Why Rust? [pdf] by burntsushi on Wednesday September 09, 2015 @09:51PM (#10195777)
I never meant to imply that any one tool is a panacea. If this is an argument you think I'm making, then let's just squash that right now: I'm not. What I'm suggesting is that we have thought through this very problem and come up with a number of ways to mitigate it. We don't want to see the ecosystem fragmented. Cargo is one strategy. Blessed crates are another. In particular:

> To be clear, I agree that a small standard library has its own downsides. In particular, quality is IMO on of the best arguments against a small standard library. Fortunately, we trying to mitigate this by adopting officially blessed libraries into the `rust-lang` organization: https://github.com/rust-lang/rfcs/blob/master/text/1242-rust... --- This allows us to avoid the problems with a big standard library (too much stuff that is hard to evolve because of stability) while still providing quality with crates that we promise to maintain.

I could absolutely see one of these crates providing async IO. I am less sure of seeing it wind up in std. Some examples of crates that are currently on track to being blessed---but maybe never end up in std---are regex and rand.

I disagree with adding good package tooling after-the-fact and using its failure in preventing fragmentation as a reason for why Cargo will be ineffective. Overcoming inertia is hard. Having Cargo at the outset is a nice advantage we have working in our favor. We should acknowledge that.

> If Rust already has std/net, this is not that far of a gap to close.

It's a pretty big gap IMO, especially if you want to provide a common high level interface. std::net is a portable interface around platform specific APIs and not much else. Async IO is quite a bit more involved.

> I'm not suggesting a Python-esque stdlib. I'm suggesting a multi-threaded, cross-platform, (ideally, edge-triggered) event system, above which higher level primitives can be introduced and safely interoperate.

Note that my comment was specifically about arguing against a Python-esque stdlib, or rather, in favor of a small standard library. It was not meant to target omission of any one particular feature, which is what you seem to be focused on. (The criticism I responded to was not specific to async IO.)