This all depends on how threads are implemented. If they're scheduled preemptively then communication can be expensive, relatively speaking, because of the need for locking and atomic operations. But you can also schedule cooperatively in user space, just as Tokio does when serially resuming async tasks; or as Java's Project Loom does for its new "lightweight" threads.
Note that unlike JavaScript, Tokio and Project Loom can also run different tasks on different, preemptively scheduled threads. And while I don't know that much Rust, I imagine you're going to need to use either unsafe or Rc or maybe even Arc if you intend to share data between different Tokio tasks--i.e. data that doesn't fit the normal caller/callee borrow semantics.
The other part of the problem is space requirements. Usually where you have preemptively scheduled threads the stack space for a thread is allocated lazily as a function is called and faults in pages via the OS' virtual memory system, much like single-thread, single-stack processes in a preemptive process OS. This means the minimum space allocation for a thread is at least 2x the page size (e.g. 4096 * 2). But many times a thread of execution only goes a couple of function calls deep, with minimal amounts of function-local (i.e. stack-allocated) data. If you have 1 thread per network connection, with hundreds of thousands or millions of connections that overhead could be significant.
But this, too, is a function of the implementation. Goroutines in Go use normal heap memory for stacks, and the compiler emits code to grow and move threads automatically. Rust proponents will tell you that async functions don't require any runtime cost because the stack requirements can be calculated statically. But to calculate this statically you can't support recursive functions. And if you can statically calculate your space requirements for the hidden async state object, you could also statically calculate the stack size for a thread just the same.
So really what it all comes down to isn't whether "async" is better or worse than "threads" along any of these dimensions. Abstractly, all threading implementations are async, and all async implementations effectively implement threads (i.e. a data structure that encapsulates a program counter, local automatic storage, etc). The real reason you choose one over the other is external factors. For Rust that dominate factor is interoperability with native C ABIs, particularly native stack disciplines. Because Rust can't implement much magic in the lower layers of the runtime environment while maintaining the degree of interoperability with C, C++, and other language libraries (via the C ABI) that they're committed to, they have no choice but to put most of the instrumentation into the language itself. And this necessitates the async contortions, independent of any other preferences. Contrast that with Go, where calling into C is slightly more costly because they preferred to push more of the async/thread abstraction beneath the language syntax.
But perhaps what this tells us is that we should think about revisiting native stack disciplines and thread scheduling semantics. IIRC, Linux will soon get scheduler activations (i.e. ability for userland to efficiently switch execution to another specified kernel-visible thread). That's a small step in the right direction, and if it catches on more operating systems will adopt this--after having ditched them 20 years ago, ironically, before async network I/O became popular and when 1:1 thread scheduling became the preferred kernel model).
Unfortunately we're not there yet. Golang with GOMAXPROCS set to 1 comes close, but now I lose the ability to spawn real threads for expensive computation.
"Reqwest" uses the Tokio machinery, even for a blocking request. If you turn on "Trace" level logging, you can watch it start up a thread pool and go through a 35-step process, using all the async and futures machinery, to do one synchronous request. Log messages include "handshake complete, spawning background dispatcher task" and "signaled close for runtime thread (ThreadId(2))"
This seems excessive.
That's not right. http is supposed to be a common library of types for HTTP servers and frameworks (although developers of some competing frameworks have rejected it). It was never an HTTP client like make it sound like, and it's actually newer than hyper.
As for reqwest vs. hyper, the former offers synchronous wrappers over the async ones, easier TLS support and other niceties (compression, proxy support, cookies, WASM). It's high-level and easier to use, somewhat like requests over urllib3 in the Python world.
That split between "sync" and "async" was always there, for networking libraries. E.g. on C/C++ you find libs that run on libevent, or Boost.Asio, or other varieties, and they don't mix, so you end up spawning a seperate thread - exactly the same thing.
And this is, IMHO, how we should see Tokio + async: as a more ergonomic libevent.
Yeah I think it points to a culture problem. In some ways because dependency management is so easy with Cargo, I think it creates the temptation to just throw in some dependencies to make something work without truly understanding the overall complexity of what you're creating. Something very similar happens in the NodeJS world.
> it splits the ecosystem
This is something I've really noticed with Rust: it almost seems like there are really two things: Rust, and Rust+Tokio. I'm a bit ambivalent about Tokio being baked into so many libraries: I think it's great to have as an option, but once I decide to use one library built around Tokio, it imposes a lot of constraints about how the flow of control is going to work in my program.
A lot of what is not enjoyable about rust as a user is really nice when it's being imposed on people who are not you, whose work you're interfacing with.
If we all have the attitude "it's good/fast because it's rust" this is not going to lead to a lot of cruft making its way into the ecosystem.
Obviously, in the real world, this basically never happens. Performance is one thing that's basically always going to 'leak', so you still get people rewriting stuff in assembly, or making custom asics, because the abstractions that higher level languages offer are not perfect.
In a strongly typed language, with strong safety guarantees, I think there are less ways an abstraction can leak (for instance, by corrupting memory or whatever), so there's a correspondingly lower cost to pulling in a dependency than there would be if you were working in an unsafe language, or a dynamic one.
I also think if performance is the only way in which your dependency leaks implementation details, then it still makes sense to pull in a dependency first, profile, then swap out if necessary.
We absolutely have an NPM/leftpad culture in Rust.
Is that better or worse than C and C++ where dependencies are so painful that you end up reinventing the wheel most of the time? I honestly don't know.
On UNIX systems just using pkg-config and similar tools, or just adopt either conan or vcpkg, which contrary to cargo also support binary libraries out of the box.
Plus vendoring C and C++ libraries is not a dark science, only known by old druids.
Also with c/c++ style system dependencies, I feel like there are a lot of issues with things like version conflicts which are solved much more simply by a package manager like cargo.
I agree that it's functional, but to say relatively easy I think is a bit of a stretch.
Usually when compiling from source many libraries provide pkg-config configuration files on "make install".
Right now, I'm using the latest version of "reqwest" known to "crates.io." It's pulling in Tokio v0.2.23, not the new tokio v1.0.0. No surprise there, the new version only came out yesterday. So we'll see how the new version works at some time in the future.
It's good to get to version 1. The semantic versioning rules allow breaking changes without changing the first digit when the first digit is 0. Typical complaint on forums: "bignum happens to use rand internally, and it happens to only use version 0.5.0, with restrictions against using a higher version due to breaking changes." Rust still has many low-level crates at version 0.x.x, from "bytes" to "uuid". Reaching 1 indicates greater stability.
Rust makes version pinning feasible, e.g. by allowing multiple versions of the same package in a build (not all module systems have this feature!) but doesn't encourage it. You've identified a problem with using 0.x.y-versioned packages as dependencies (which means de-facto opting out of semver), but that's not a problem with Rust specifically; it could occur in any language.
It is up to the library authors to uphold its semantics.
Isn't it the case with pkg-config that everything is stored in a central location?
In any case, I think you can't seriously argue that the c/c++ dependency management solution is anywhere close to running `cargo build`/`cargo run` in terms of simplicity.
Specially the C++ community has seen lack of something like cargo as weaknesses and moved to sort it out.
I think this is the aspect of modern approaches async which I am most ambivalent about. One of the things I have learned about programming over the past 10 years is that I, as the programmer, really want to own the flow of control of my program. Once I hand that over to some other system, usually in the name of convenience, I will start to have issues which are difficult to understand and solve.
For instance, a while ago, I was working on a project which was using making heavy use of RXJava. One of my colleagues pushed a commit, and suddenly CI was failing on a unit test which passed when run locally. It turns out the problem was because the CI server was running tests with a different scheduler, so GC was happening at a different time, creating an NPE which didn't happen locally. Imo when you start to see unit tests behaving inconsistently based on a factor which is completely outside the actual code you yourself have written, this is a sign you are going down the wrong path.
I also wonder how much a lot of the buzz around async actually has to do with the fact that it's a bit brain-bending to wrap your head around at first, as compared to its actual utility. I think for a lot of us as programmers, we enjoy that feeling of understanding something difficult - like when you really get recursion for the first time - and we're attracted to the idea of really fundamentally new concepts being introduced to programming.
But it seems to me that async is one of those concepts which brings us farther away from actually programming the hardware, and puts a kind of middle-man between us and the CPU, and I'm not sure that has ever been a good thing.
More to your point, as I think you were using this as an analogy for the perils of giving up control: Rust's explicitness should entail all the semantics of your program, and hence async Rust makes you model out all the potentially-racy async interactions with Arc, Mutex, etc. The same middle-man (the borrow checker) who watches over your regular ol' sync code's memory-correctness, now expects extra constraints to be upheld for values passing through async-boundaries. And for me the whole point of Rust is that this correctness proof will do a better job than any person could, for any moderately sized program. So this is middle man you'd want between you and the CPU.
That said, your async runtime could definitely do shenanigans that screw up your nicely modeled program, but that would be a bug in that specific runtime. I haven't deeply read Tokio's source and even if I did, making a qualified judgment about it is beyond me.
An async runtime is a totally different animal. If you hand me a block of async rust code and ask me how it will execute, the answer I have to give is "it depends on the runtime". This is the disconnect I am talking about.
I'd be curious to hear about examples where the runtime did or would surprise you!
- So what if I am implementing a high-throughput, performance critical system which makes heavy use of async, and under certain circumstances the runtime I'm using falls off a performance cliff. It's going to be difficult to diagnose and solve this problem, because the critical path of my program actually winds through a library which is essentially a black box to me.
- What if I have two dependencies, and each one internally depends on a separate async runtime. And what if each of these runtimes is designed with the assumption that it is the main owner of system resources, like threads. There may be conflicts which are very difficult to understand but have effects on the performance of my program.
I think fundamentally, an issue with this type of "middleware" is that by its nature, an async runtime, like Tokio for example, has to be implemented with a lot of assumptions about how "the generic program" should optimally handle async. It may work great for the vast majority of use-cases, but fundamentally whenever you design a super general, abstract system like this you have to make tradeoffs.
In some ways Rust has taken probably the best possible approach to this, by making it modular and allowing you to bring your own runtime, but I think in practice, if the use of async continues to become pervasive in Rust and certain libraries get locked into certain ecosystems, it will not be so easy in practice to take advantage of that modularity.
I didn't know that cross "async" communication was cheaper, that does seem like a good selling point, but what exactly makes it cheaper? After all threads share the same address space, so you can just pass pointers around the same way you would within the same thread. I expected the overhead to be roughly similar.
Things can get cache-expensive if the code is running on different cores, but then again using all the hardware resources available is generally something you want to do if you care about performance.
And of course with threads it's harder to actually run single-core, you need to dedicated a specific core which brings operational complexity.
Maybe one can enforce this convention in the particular project, but there's no ecosystem-wide consensus on this, and in fact I don't want this to be consensus. I write blocking non-async functions every day. Why am I wrong to do so?
I feel like the "what color is your function" thing is incomplete. There are arguably 3 types of functions:
- Functions which do all their work synchronously and return without blocking
- Async functions which contain an internal state machine
- Functions which block on expensive IO or long computations
Mixing blocking functions and async functions in the same kernel thread leads to various performance disasters. Javascript is so meticulous about not having blocking IO in part because its basically impossible to tell from a function's signature whether it will block the thread. Lua has this problem - callback oriented lua feels like a natural fit for the language, but lots of 3rd party libraries are packed with blocking calls. Writing asyncronous lua feels like fighting a river. You have to constantly guard against calling blocking code, and most API docs won't tell you where they block.
This (in my opinion) not "wrong". At least not in general. There are instances where it might be more or less probematic though.
It's probably problematic if you already have a bunch of async code in the codebase, because other readers of the code are likley to expect blocking functions to be async.
It's maybe problematic for high performance or high scale code. Synchronous blocking functions are more likely to hit OS limits (file handles, network sockets, etc) than async code. If the code is obviously written from the ground up for high scale/performance, this is less likely to be a problem, but if it's proof of concept code that's likely to get pushed into production by over eager PMs as soon as it passes tests, it'll be worse.
It's possibly problematic if done in a language/frameworks where async is idiomatic - it'd be wrong to write using blocking functions in a nodejs codebase, because you'd be breaking other people expectations when reading/understanding the code.
Maybe a useful rule of thumb might be "if more than some number (perhaps 30 or 50%) of other people working on the code might think 'hang on, I'm gonna refactor this to use async', then maybe using a non-asyn blocking function was the wrong choice. That means it's _never_ the wrong choice for one person codebases. It means it's probably almost always a wrong choice in a javascript codebase. For everything else? "It depends". I'd always choose to go with "the principle of least astonishment" - do whatever other people who might be affected would expect you to wherever possible.
An async system which poisons the rest of my code to force async usage doesn't seem like it will scale to code leveraging multiple libraries and will likely fail at the first lib where the author decided not to bother. The beauty of coroutines in go and Java(soon) is that the async functionality remains local to the code that can make use of it - everyone else just sees a thread-like API.
In a codebase where concurrency is carefully controlled and constrained, an async system that gives you visibility into where the yield points are is very valuable: https://glyph.twistedmatrix.com/2014/02/unyielding.html .
You are quite correct. This happens.
That's the bit you'd use an async runtime for, in a (mostly) dedicated thread.
The heavy computations would be in other threads that aren't doing so.
What does 'blocking' mean? I would expect the definition of synchronous to be the exact opposite; i.e., a synchronous function must block the caller until the function has finished executing. For that matter, what is "regular logic"? The name implies there is some sort of "irregular logic" to contrast it with.
I get the feeling that the writing may be unclear because the concepts are themselves not well-defined.
With blocking I mean waiting. For I/O to complete, for time to pass, or for another task to complete something. In event-based programming, functions must not block. Async functions may seem to block, but they don't rely because a state machine is involved.
The upside of async over a simple event loop is, in my experience, when things become less simple, and you end up with hard-to-read little state machines all over the place.
With async, you can have your event loop, but the state machines are handled by the compiler. Code is like threaded code. That can be very convenient.
Threads, obviously, accomplish the same thing, and arguably more easily. But threads have a performance problem when they must interact heavily. Cross-thread communication is expensive. Single-threaded async task interaction is very cheap, comparatively (I use tokio only in single-threaded mode, as an event loop replacement; its multi-threaded scheduler performs terribly on serious I/O). I think the interaction problem is often more important than just the number of tasks.
As for the function coloring argument, I've started to see the async keyword as documentation. A non-async function is "regular logic", it must complete without blocking. An async function is a state machine; as such it must be part of a larger state machine (the event loop), and it can go down into smaller sub-state machines (i.e. call other async functions). If you make that distinction explicit, it all makes a lot of sense.