The upside of async over a simple event loop is, in my experience, when things become less simple, and you end up with hard-to-read little state machines all over the place.

With async, you can have your event loop, but the state machines are handled by the compiler. Code is like threaded code. That can be very convenient.

Threads, obviously, accomplish the same thing, and arguably more easily. But threads have a performance problem when they must interact heavily. Cross-thread communication is expensive. Single-threaded async task interaction is very cheap, comparatively (I use tokio only in single-threaded mode, as an event loop replacement; its multi-threaded scheduler performs terribly on serious I/O). I think the interaction problem is often more important than just the number of tasks.

As for the function coloring argument, I've started to see the async keyword as documentation. A non-async function is "regular logic", it must complete without blocking. An async function is a state machine; as such it must be part of a larger state machine (the event loop), and it can go down into smaller sub-state machines (i.e. call other async functions). If you make that distinction explicit, it all makes a lot of sense.

This all depends on how threads are implemented. If they're scheduled preemptively then communication can be expensive, relatively speaking, because of the need for locking and atomic operations. But you can also schedule cooperatively in user space, just as Tokio does when serially resuming async tasks; or as Java's Project Loom does for its new "lightweight" threads.

Note that unlike JavaScript, Tokio and Project Loom can also run different tasks on different, preemptively scheduled threads. And while I don't know that much Rust, I imagine you're going to need to use either unsafe or Rc or maybe even Arc if you intend to share data between different Tokio tasks--i.e. data that doesn't fit the normal caller/callee borrow semantics.

The other part of the problem is space requirements. Usually where you have preemptively scheduled threads the stack space for a thread is allocated lazily as a function is called and faults in pages via the OS' virtual memory system, much like single-thread, single-stack processes in a preemptive process OS. This means the minimum space allocation for a thread is at least 2x the page size (e.g. 4096 * 2). But many times a thread of execution only goes a couple of function calls deep, with minimal amounts of function-local (i.e. stack-allocated) data. If you have 1 thread per network connection, with hundreds of thousands or millions of connections that overhead could be significant.

But this, too, is a function of the implementation. Goroutines in Go use normal heap memory for stacks, and the compiler emits code to grow and move threads automatically. Rust proponents will tell you that async functions don't require any runtime cost because the stack requirements can be calculated statically. But to calculate this statically you can't support recursive functions. And if you can statically calculate your space requirements for the hidden async state object, you could also statically calculate the stack size for a thread just the same.

So really what it all comes down to isn't whether "async" is better or worse than "threads" along any of these dimensions. Abstractly, all threading implementations are async, and all async implementations effectively implement threads (i.e. a data structure that encapsulates a program counter, local automatic storage, etc). The real reason you choose one over the other is external factors. For Rust that dominate factor is interoperability with native C ABIs, particularly native stack disciplines. Because Rust can't implement much magic in the lower layers of the runtime environment while maintaining the degree of interoperability with C, C++, and other language libraries (via the C ABI) that they're committed to, they have no choice but to put most of the instrumentation into the language itself. And this necessitates the async contortions, independent of any other preferences. Contrast that with Go, where calling into C is slightly more costly because they preferred to push more of the async/thread abstraction beneath the language syntax.

But perhaps what this tells us is that we should think about revisiting native stack disciplines and thread scheduling semantics. IIRC, Linux will soon get scheduler activations (i.e. ability for userland to efficiently switch execution to another specified kernel-visible thread). That's a small step in the right direction, and if it catches on more operating systems will adopt this--after having ditched them 20 years ago, ironically, before async network I/O became popular and when 1:1 thread scheduling became the preferred kernel model).

Unfortunately we're not there yet. Golang with GOMAXPROCS set to 1 comes close, but now I lose the ability to spawn real threads for expensive computation.

"Reqwest" uses the Tokio machinery, even for a blocking request. If you turn on "Trace" level logging, you can watch it start up a thread pool and go through a 35-step process, using all the async and futures machinery, to do one synchronous request. Log messages include "handshake complete, spawning background dispatcher task" and "signaled close for runtime thread (ThreadId(2))"

This seems excessive.

That's not right. http is supposed to be a common library of types for HTTP servers and frameworks (although developers of some competing frameworks have rejected it). It was never an HTTP client like make it sound like, and it's actually newer than hyper.

As for reqwest vs. hyper, the former offers synchronous wrappers over the async ones, easier TLS support and other niceties (compression, proxy support, cookies, WASM). It's high-level and easier to use, somewhat like requests over urllib3 in the Python world.

That split between "sync" and "async" was always there, for networking libraries. E.g. on C/C++ you find libs that run on libevent, or Boost.Asio, or other varieties, and they don't mix, so you end up spawning a seperate thread - exactly the same thing.

And this is, IMHO, how we should see Tokio + async: as a more ergonomic libevent.

Yeah I think it points to a culture problem. In some ways because dependency management is so easy with Cargo, I think it creates the temptation to just throw in some dependencies to make something work without truly understanding the overall complexity of what you're creating. Something very similar happens in the NodeJS world.

> it splits the ecosystem

This is something I've really noticed with Rust: it almost seems like there are really two things: Rust, and Rust+Tokio. I'm a bit ambivalent about Tokio being baked into so many libraries: I think it's great to have as an option, but once I decide to use one library built around Tokio, it imposes a lot of constraints about how the flow of control is going to work in my program.

A lot of what is not enjoyable about rust as a user is really nice when it's being imposed on people who are not you, whose work you're interfacing with.

If we all have the attitude "it's good/fast because it's rust" this is not going to lead to a lot of cruft making its way into the ecosystem.

Obviously, in the real world, this basically never happens. Performance is one thing that's basically always going to 'leak', so you still get people rewriting stuff in assembly, or making custom asics, because the abstractions that higher level languages offer are not perfect.

In a strongly typed language, with strong safety guarantees, I think there are less ways an abstraction can leak (for instance, by corrupting memory or whatever), so there's a correspondingly lower cost to pulling in a dependency than there would be if you were working in an unsafe language, or a dynamic one.

I also think if performance is the only way in which your dependency leaks implementation details, then it still makes sense to pull in a dependency first, profile, then swap out if necessary.

We absolutely have an NPM/leftpad culture in Rust.

Is that better or worse than C and C++ where dependencies are so painful that you end up reinventing the wheel most of the time? I honestly don't know.

On UNIX systems just using pkg-config and similar tools, or just adopt either conan or vcpkg, which contrary to cargo also support binary libraries out of the box.

Plus vendoring C and C++ libraries is not a dark science, only known by old druids.

Also with c/c++ style system dependencies, I feel like there are a lot of issues with things like version conflicts which are solved much more simply by a package manager like cargo.

I agree that it's functional, but to say relatively easy I think is a bit of a stretch.

Usually when compiling from source many libraries provide pkg-config configuration files on "make install".

Right now, I'm using the latest version of "reqwest" known to "crates.io." It's pulling in Tokio v0.2.23, not the new tokio v1.0.0. No surprise there, the new version only came out yesterday. So we'll see how the new version works at some time in the future.

It's good to get to version 1. The semantic versioning rules allow breaking changes without changing the first digit when the first digit is 0. Typical complaint on forums: "bignum happens to use rand internally, and it happens to only use version 0.5.0, with restrictions against using a higher version due to breaking changes." Rust still has many low-level crates at version 0.x.x, from "bytes" to "uuid". Reaching 1 indicates greater stability.

Rust makes version pinning feasible, e.g. by allowing multiple versions of the same package in a build (not all module systems have this feature!) but doesn't encourage it. You've identified a problem with using 0.x.y-versioned packages as dependencies (which means de-facto opting out of semver), but that's not a problem with Rust specifically; it could occur in any language.

It is up to the library authors to uphold its semantics.

Isn't it the case with pkg-config that everything is stored in a central location?

In any case, I think you can't seriously argue that the c/c++ dependency management solution is anywhere close to running `cargo build`/`cargo run` in terms of simplicity.

Specially the C++ community has seen lack of something like cargo as weaknesses and moved to sort it out.

I think this is the aspect of modern approaches async which I am most ambivalent about. One of the things I have learned about programming over the past 10 years is that I, as the programmer, really want to own the flow of control of my program. Once I hand that over to some other system, usually in the name of convenience, I will start to have issues which are difficult to understand and solve.

For instance, a while ago, I was working on a project which was using making heavy use of RXJava. One of my colleagues pushed a commit, and suddenly CI was failing on a unit test which passed when run locally. It turns out the problem was because the CI server was running tests with a different scheduler, so GC was happening at a different time, creating an NPE which didn't happen locally. Imo when you start to see unit tests behaving inconsistently based on a factor which is completely outside the actual code you yourself have written, this is a sign you are going down the wrong path.

I also wonder how much a lot of the buzz around async actually has to do with the fact that it's a bit brain-bending to wrap your head around at first, as compared to its actual utility. I think for a lot of us as programmers, we enjoy that feeling of understanding something difficult - like when you really get recursion for the first time - and we're attracted to the idea of really fundamentally new concepts being introduced to programming.

But it seems to me that async is one of those concepts which brings us farther away from actually programming the hardware, and puts a kind of middle-man between us and the CPU, and I'm not sure that has ever been a good thing.

More to your point, as I think you were using this as an analogy for the perils of giving up control: Rust's explicitness should entail all the semantics of your program, and hence async Rust makes you model out all the potentially-racy async interactions with Arc, Mutex, etc. The same middle-man (the borrow checker) who watches over your regular ol' sync code's memory-correctness, now expects extra constraints to be upheld for values passing through async-boundaries. And for me the whole point of Rust is that this correctness proof will do a better job than any person could, for any moderately sized program. So this is middle man you'd want between you and the CPU.

That said, your async runtime could definitely do shenanigans that screw up your nicely modeled program, but that would be a bug in that specific runtime. I haven't deeply read Tokio's source and even if I did, making a qualified judgment about it is beyond me.

An async runtime is a totally different animal. If you hand me a block of async rust code and ask me how it will execute, the answer I have to give is "it depends on the runtime". This is the disconnect I am talking about.

I'd be curious to hear about examples where the runtime did or would surprise you!

- So what if I am implementing a high-throughput, performance critical system which makes heavy use of async, and under certain circumstances the runtime I'm using falls off a performance cliff. It's going to be difficult to diagnose and solve this problem, because the critical path of my program actually winds through a library which is essentially a black box to me.

- What if I have two dependencies, and each one internally depends on a separate async runtime. And what if each of these runtimes is designed with the assumption that it is the main owner of system resources, like threads. There may be conflicts which are very difficult to understand but have effects on the performance of my program.

I think fundamentally, an issue with this type of "middleware" is that by its nature, an async runtime, like Tokio for example, has to be implemented with a lot of assumptions about how "the generic program" should optimally handle async. It may work great for the vast majority of use-cases, but fundamentally whenever you design a super general, abstract system like this you have to make tradeoffs.

In some ways Rust has taken probably the best possible approach to this, by making it modular and allowing you to bring your own runtime, but I think in practice, if the use of async continues to become pervasive in Rust and certain libraries get locked into certain ecosystems, it will not be so easy in practice to take advantage of that modularity.

I didn't know that cross "async" communication was cheaper, that does seem like a good selling point, but what exactly makes it cheaper? After all threads share the same address space, so you can just pass pointers around the same way you would within the same thread. I expected the overhead to be roughly similar.

Things can get cache-expensive if the code is running on different cores, but then again using all the hardware resources available is generally something you want to do if you care about performance.

And of course with threads it's harder to actually run single-core, you need to dedicated a specific core which brings operational complexity.

Maybe one can enforce this convention in the particular project, but there's no ecosystem-wide consensus on this, and in fact I don't want this to be consensus. I write blocking non-async functions every day. Why am I wrong to do so?

I feel like the "what color is your function" thing is incomplete. There are arguably 3 types of functions:

- Functions which do all their work synchronously and return without blocking

- Async functions which contain an internal state machine

- Functions which block on expensive IO or long computations

Mixing blocking functions and async functions in the same kernel thread leads to various performance disasters. Javascript is so meticulous about not having blocking IO in part because its basically impossible to tell from a function's signature whether it will block the thread. Lua has this problem - callback oriented lua feels like a natural fit for the language, but lots of 3rd party libraries are packed with blocking calls. Writing asyncronous lua feels like fighting a river. You have to constantly guard against calling blocking code, and most API docs won't tell you where they block.

This (in my opinion) not "wrong". At least not in general. There are instances where it might be more or less probematic though.

It's probably problematic if you already have a bunch of async code in the codebase, because other readers of the code are likley to expect blocking functions to be async.

It's maybe problematic for high performance or high scale code. Synchronous blocking functions are more likely to hit OS limits (file handles, network sockets, etc) than async code. If the code is obviously written from the ground up for high scale/performance, this is less likely to be a problem, but if it's proof of concept code that's likely to get pushed into production by over eager PMs as soon as it passes tests, it'll be worse.

It's possibly problematic if done in a language/frameworks where async is idiomatic - it'd be wrong to write using blocking functions in a nodejs codebase, because you'd be breaking other people expectations when reading/understanding the code.

Maybe a useful rule of thumb might be "if more than some number (perhaps 30 or 50%) of other people working on the code might think 'hang on, I'm gonna refactor this to use async', then maybe using a non-asyn blocking function was the wrong choice. That means it's _never_ the wrong choice for one person codebases. It means it's probably almost always a wrong choice in a javascript codebase. For everything else? "It depends". I'd always choose to go with "the principle of least astonishment" - do whatever other people who might be affected would expect you to wherever possible.

An async system which poisons the rest of my code to force async usage doesn't seem like it will scale to code leveraging multiple libraries and will likely fail at the first lib where the author decided not to bother. The beauty of coroutines in go and Java(soon) is that the async functionality remains local to the code that can make use of it - everyone else just sees a thread-like API.

In a codebase where concurrency is carefully controlled and constrained, an async system that gives you visibility into where the yield points are is very valuable: https://glyph.twistedmatrix.com/2014/02/unyielding.html .

You are quite correct. This happens.

That's the bit you'd use an async runtime for, in a (mostly) dedicated thread.

The heavy computations would be in other threads that aren't doing so.

What does 'blocking' mean? I would expect the definition of synchronous to be the exact opposite; i.e., a synchronous function must block the caller until the function has finished executing. For that matter, what is "regular logic"? The name implies there is some sort of "irregular logic" to contrast it with.

I get the feeling that the writing may be unclear because the concepts are themselves not well-defined.

With blocking I mean waiting. For I/O to complete, for time to pass, or for another task to complete something. In event-based programming, functions must not block. Async functions may seem to block, but they don't rely because a state machine is involved.

That said, if you can get your job done without this stuff, that's fine too, but the reasons it was pursued specifically involve the above.

That being said I can't shake the feeling that going for something like Tokio in such a case is a bit like healing a paper cut by amputating the arm. Sure, technically you don't have the original problem anymore...

So, I can relate to not wanting to pull in the dependency. But otherwise it seems pretty straightforward to me. You just macro-decorate the main function, sprinkle some async/await around, maybe add a join or a mutex somewhere, and then pretty much forget all about event loops, messaging, threads and whatnot. I feel like I must be missing something important here.

that is ... not how I have experienced it. I work on building highly concurrent systems every day but async drives me insane. to me the fundamental issue is that although the code now reads linearly, it no longer executes linearly (or reasonably close to linearly), which is 1000x more confusing.

the other thing, when I'm using rust to build something high performance, part of the reason is it provides greater control. I just can't square that with macro-decorating my main function, and handing over the core control-flow to someone else's runtime.

With async/await the apparent complexity gets reduced at the cost of vastly increased actual complexity. E.g., now instead of everything being your code that you can look at and reason about, all your concurrent workloads disappear in this void that promises to do the right thing with them. If it works the way you intended, great. If it doesn't, the rabbit hole can now be really deep.

And then, it's also a trust issue. Now you have to trust other people to have done a good job.

Ok, yes, this makes sense.

And the loss of control is also an issue for me. I write code for memory-constrained environments, with blocking code and OS threads I can usually bound my memory consumption fairly easily. If I surrender the control to a scheduler runtime I feel like it becomes a lot harder, although here I'm willing to concede that it might have more to do with my lack of experience with Tokio than an objective issue.

    fn read(&mut self, buf: &mut [u8]) -> Result<usize, io::Error>

    fn read<T: AsMut<[u8]>>(self, buf: T) ->
        impl Future<Item = (Self, T, usize), Error = (Self, T, io::Error)>

I recently tried writing a small program that would manually poll a future to get a feel for it - utter disaster. conflicting versions of tokio, compiles but crashes because something is called outside of the tokio runtime context, etc. all the examples have #[tokio::main]-decorated main - it's like, I'm not giving you my #$(&#@(&ing main function! the programs I write have tons of stuff going on! I can't just give some library my entire control flow!

sorry for the rant! felt good to write it though.

Maybe it is undesirable because that often times plain mono-thread synchronous is fast enough, easier to read, easier to debug and safe to handle to a junior ? Not everybody in a team has the same level of expertise.

And Rust is not an interpreted language. IMHO, interpreted languages should just drop to a compiled one to keep it KISS. Instead of going the async road, just to discover in production, it is unstable because back-pressure was not taken into account. And, in Rust, it is probably not worth the effort and ultimately bloat most of the time.

Shared-memory concurrency is pretty much always buggy, IME, even if your team thinks they're experts.

> And Rust is not an interpreted language. IMHO, interpreted languages should just drop to a compiled one to keep it KISS. Instead of going the async road, just to discover in production, it is unstable because back-pressure was not taken into account.

WTF? Switching to a compiled language doesn't magically make your threads nonblocking. Maybe you can serve 10x more users with a compiled language, but if we're talking about slow network requests then async can make your throughput thousands of times higher.

You are reading something I did not write. I am not obsessed with the nonblocking mantra. Blocking is not inherently bad. Multi-processus is also a perfectly valid concurrency model.

Nor am I obsessing about being ultra performant to achieve the revered C10K when I don't need to or can get around it. That was my point.

Not everybody is Facebook or Netflix. For the vast majority of small and medium enterprises, it faster, simpler, safer (and possibly cheaper) to quickly develop a blocking program without thread and spawn multiple processes.

Because WTF does using a compiled language have to do with anything?

> Nor am I obsessing about being ultra performant to achieve the revered C10K when I don't need to or can get around it. That was my point.

Then what is it that you imagine using a compiled language would help with?

Writing bug-free shared memory concurrency programs with Go is trivial. Your opinion is based on outdated information.

I vaguely remember expecting a reference and getting a copy or vice versa...

I mean, I agree that Go is miles ahead of most other languages when it comes to helping prevent concurrency bugs, but it's sill tricky. Same with Rust.

Isn't that exactly what Go people push? "Don't communicate by sharing memory; share memory by communicating" and all that. If you start pushing pointers to shared memory around then I'd expect all of the problems of traditional multithreading to reappear.

Passing pointers to shared memory is the foundation of a huge number of idiomatic, performant, and productive design patterns and architectures. There exist a number of conventions and tools, like the race detector, which reduce the risks of data races to entirely reasonable levels.

https://blog.rust-lang.org/2015/04/10/Fearless-Concurrency.h...

Like any static analysis, it’s a give and take between making sure your analysis is sound, while still allowing useful programs.

I'm not saying locks are better than async/await (although they are[1]). You're saying the borrow checker itself can't handle them in real world use?

[1] https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...

(The borrow checker does not understand locks as a special construct, to be extra clear.)

Did you read the post I linked? It lays out the details. I am happy to clarify if you don’t get the specifics.

Sorry, my mistake!

Edit to add: futures work badly in every language, so there's no shame in the borrow checker not working with them.

Edit 2: But in that case we're back to "why would Rust want async/await over (potentially green) threads with its first-class support for locks?"

1. Rust's Journey to Async/Await - https://www.youtube.com/watch?v=lJ3NC-R3gSI

2. The Talk You've been Await-ing for - https://www.youtube.com/watch?v=NNwK5ZPAJCk

The first video goes into all the bits you're concerned about and all the things that Rust has tried before arriving where they are now

Tokio basically is green threads as a library.

This is partly why I prefer threading over async in Rust. Look, we went through some enormous effort to make threading good and fun again. Why wouldn't you use threading?

Because the C10^nK problem where n increases periodically is still a thing?

Or rather -- async tasks are as close as you can get to green threads in rust without a runtime that would impose overhead on every program

When you are writing high-performance server code, async is the common scenario.

Especially harmful is that common libraries like reqwest (for HTTP requests) pulls async.

But for the rest of us, simple, blocking code will do just fine and save us a few headaches.

I understand your point, but if performance isn't a concern, why use Rust at all? If the intricacies of async is that much of a burden then Rust is probably not the right tool of the job.

K8s is brings way more complexity and headache, so it's kind of funny that you suggest that before using async/await.

https://lucumr.pocoo.org/2016/10/30/i-dont-understand-asynci...

If your planning to scale up, k8s is likely something you'll want later. Additional complexity in your apps is not.

My point was that it really isn't that much additional complexity. 99.9% of the time, the main difference is that you'll have to write "await". You don't really need to know that there's a state machine hiding beneath.

On the other hand, there has been from day one a sort of hype around Rust's safety features, and an eagerness to promote any new library or framework written in Rust as a savior of programming. This library, as you note, will be used inappropriately (i.e. in contexts where it's not really necessary or reasonable to do) and lead to the worst kinds of bugs--those that lurk in complicated, difficult to understand code, and that are generally worse than any memory-related security vulnerability.

async is not mandatory for reqwest. It provides a blocking API as well.

> reqwest is the default HTTP request library in Rust ecosystem

There is no "default" libraries. There are popular HTTP clients other than reqwest that also provide blocking APIs such as isahc and ureq

I am using isahc, but that also doesn't help when (say) Rusoto AWS library pulls reqwest pulls Tokio pulls async.

Basically, it isn't entirely opaque to you how it is handled.

Web servers are all about I/O and handling small tasks (requests), and are a perfect use case for asynchronous programming.

> That sounds like premature optimization in many situations IMO

Maybe in some cases... but then just don't use futures. Rust does not have a runtime, so it gives you the choice. std is all blocking, so you can spawn threads and do event-driven programming, which might be just fine for a lot of people.

async is more ergonomic for Rust specific reasons, makes sense for a lot of use cases, and was a highly requested language feature, so it was added to the language. OS threads can work just fine for many people. If that includes you, you don't have to use async.

In principle yes, but in practice I disagree. Due to keep-alive you want to be able to handle many idle connections at the same time, but you rarely want to handle many active connections at the same time.

Example: Whenever you talk to another services (e.g. a database) you need to limit the number of connections you have open. You can't just blindly open a new connection per incoming request. This means that your practical level by parallelism is often bounded by your database. If every request talks to Postgres and you have a connection pool with a limit of 50, then there is nothing to gain by having support for 1000s of "active" connections. You'd rather want Postgres to focus on finishing existing requests than opening new ones.

And once you look into Postgres you'll observe the same thing: There's only limited amount of CPU/IO so there's no point in having 1000s of "active" requests going on at the same time.

It's worth mentioning that even if you're IO-bound at your DB, running an async application server now means you don't need to tie up a thread waiting on it. Memory usage aside, you more or less don't need to think about threads much, whereas a threadpool (one waiting on IO) is something you have to actively manage.

ctrl-f "sanxiyn" yields 28 instances, most of them restating in every subtree the same point about how you think threads > async.

Since I think most of us tend to read the comments section top to bottom, it seems ideal to limit your opinion to a couple comments and then put your effort into making those comments a good rundown of your position. It would certainly be more interesting to read and consider.

1. If you use async in single thread mode, you can save thread synchronization.

2. async works better for idle connections and slow connections, even when the absolute number of connections is not large.

3. async task is easier to cancel than thread.

I still won't use async since thread synchronization hasn't been slow for me, thread cancellation hasn't been problematic for me, and I use nginx to handle idle connections and slow connections, but it's useful to know in case I need.

I also routinely interview candidates that want to work at my company. We typically downlevel or turn away candidates who do not have experience solving C10K problems (unless they can appropriately fake that experience).

Even though you may not need to solve C10K problems, (like in any education) it is typically very useful for engineers to think about and attempt solving artificial C10K problems to better educate themselves for when they need to solve those problems.

Meanwhile, if you're the CTO of a company and truly know your business will not require C10K ever in its life, and you know this is the wrong time to educate yourself and your employees, then yes you're correct that async is the wrong abstraction for you right now. Frankly in that case I'd argue Rust may also be the wrong abstraction for right now.

The async concept has been used for decades in pretty much every product I've worked on professionally, from enterprise raid controllers to network protocol implementations and telephony software. An engineer I respect once told me that really it's the only way to write services at scale, and anything else is just a step on the road until you reinvent it. He was probably exaggerating, but it is very important, and nearly ubiquitous.

Having used custom frameworks for async code in C and C++, it's really refreshing to have it baked into the language and well supported. It's yet another arrow in Rust's fantastic quiver.

If you can't handle thousands of requests per second with thread per request, that's more about your software stack, not about threading.

I guess it was different in the past when computers were slow. I can believee that.

You used to be able to easily DOS apache servers this way, because you just needed enough concurrent connections to exhaust its thread pool and then it wouldn't be able to handle any more requests. And then you need a bit rate on each connection just high enough not to trip apache's connection timeout. (So like, 20 TCP connections each sending 1 byte every 20 seconds would do it. Not sure about today but Apache used to be brought to its knees with 1 bps of bandwidth.)

You could probably mitigate this by putting nginx in front of your server, but this works because nginx uses async internally to handle requests. And that won't work if you ever do proxy passthrough (for SSE, websockets, etc).

It's a totally legitimate answer, though, it just made me smile.

It probably doesn't. Spawning threads per request is a lazy pattern.

We use Apache to handle billions of requests per day, and even its thread pool can be an issue.

This mechanism is implemented by Apache httpd, Tomcat and pretty much every classic application server.

True, but there's a reason that Apache usage is declining at the rate that it is.

One is they make your functions colored; async functions world best with other async functions while normal blocking functions work best with other blocking functions.

They also introduce a lot of noise; putting async/await everywhere doesn't tell you anything interesting.

Considering a normal-sized Linux server can handle a million threads without much trouble, it really seems like misplaced effort.

I haven't used it yet, so I can only repeat the advertising copy, but nevertheless wanted to give some perspective from other ecosystems.

[1]: https://wiki.openjdk.java.net/display/loom/Main

This is definitely a good thing. All computations should me "marked" as total or effectful with various possible effects (blocking, async, nondeterministic, possibly non-terminating etc etc).

Reasoning about your program is hard when each computation is a blackbox possibly containing any side-effects which could cause unpredictable changes in the control flow and result in a completely incomprehensible way.

Async/await thing is indeed least sound and ergonomic way of doing this. Monads with monad transformers are a bit better. Algebraic effects are the best in terms of composability, ergonomics and mental overhead, but not here yet (though OCaml may be soon become the first industrial-grade language incorporating them [1])

[1] https://www.youtube.com/watch?v=z8SI7WBtlcA

Marking functions for side-effects would be a good thing but it isn't what function coloring means in this context.

An async function and a normal function are semantically the same, they just have different syntaxes and you can't easily call one from another.

They can be both be either blocking or non-blocking, especially if they take other functions as arguments.

That's precisely the coloring that makes async useful. Without it you need to explicitly protect all shared data with mutexes or other synchronization primitives. 40+ years of threaded programming has shown that programmers cannot generally be trusted to get this right, and this in an area ripe with bugs.

But they are not, AsyncIO and BlockingIO are different side effects, thus you have different types of computation, that's exactly what I'm talking about. In languages with monads or algebraic effects these would have different types.

You don't say that Lists and Arrays are semantically the same, despite being similar sequential collections, they still have separate types for a reason. Though it's good to be able to abstract over them.

And in languages with monads we can parametrize over various effect types by using tagless final approach, which allows us to write computations which could be interpreted in contexts of various effects (in this case, Async and Sync), just as we parametrize containers with types of content (in [1] there is an example of how we can parametrize computation over various async implementations), but still these are different effects.

[1] https://kubuszok.com/2019/io-monad-which-why-and-how/#typed-...

I do agree that parametrizing over the blocking strategy is a great idea, but languages that simply provide an async syntactic marker don't necessarily allow that, and if your language is powerful enough you do not need the annotation in the first place.

- async (or explicitly Future/Stream): external effects (I/O, interacts with synchronization, whatever)

- &mut: local effects

- otherwise: pure

Not in Zig: https://kristoff.it/blog/zig-colorblind-async-await/

> during compile-time, it’s possible to inspect if the overall program is in evented mode or not, and properly designed code might decide to move to a threaded model when in blocking mode, for example.

Seems like that would use a lot of memory for all the stacks

So that would mean a lot of memory for 1 million threads, 2TB of RAM. But you can change the default. With a 64k stack you'd use up ~68GB of RAM, which doesn't seem like a lot for 1 million threads and 1 million requests happening at the same time.

It might not be worth doing it in practice, but it is something to keep on mind.

Or would you want to do that with a Raspberry Pi? :-)

> What "normal-sized Linux server" has 70GB of RAM?

Also, why are you "quoting" what I did not say?

I don't get what's so bad about "colored" function. That color is just about the return type of the function.

How do you return an error from a function that does not return a Result? You must call unwrap (panic) or change the colour of your function by changing the return type to Result and fix all the caller.

Similarly, if you want to use a future from a non-async function, you either call `block_on(...)`, or you change the return type to a future by marking the function async.

I don't think it is that bad. That's just the way explicitly typed programming languages work.

Basically if you just ignore the async/await keyword, the code should read mostly the same as synchronous code (which is the point of the async/await effort).

Maybe you have a concrete example of convoluted async code?

Contrast this with the thread model, where the runtime needs to create and destroy threads where the code asks for it. In other words, your program logic is mixed up with runtime considerations.

For a practical example, let's say you want to use rust to extend some C code to create a network client that calls back into the C code. What if the C code is not thread-safe? With async, no problem, just use the CurrentThread runtime. This is my use case, anyway.

Why are there so many libraries that depend on Tokio then? If only the edges need to actually use an async runtime, the middle-tier libraries can just be plain futures.

In other words with async you have to use code that knows how to yield, but the benefit is that you have a lot of flexibility in the runtime. That means you can scale up, scale down, and fit it into weird environments (like in other programs that aren't expecting concurrency).

Threading (or process model) has a lot of upside though, too. For one, it's pre-emptive, so scheduling can be more "fair". For another, it's a lot easier to debug (erlang does a great job here).

I'd generally default to using threads, unless I have a specific reason for async and/or the code is fairly low-level and might be used in a variety of environments.

Disclaimer: don't take my claims as authoritative. I know a few things from seeing what works and not, but I could be wrong on some of the finer points.

I agree async has usability problems. Hopefully they'll get better over time; I'm looking forward to eventually having generators rather than dealing with futures::stream::StreamExt and the like.

I don't think everyone needs to use async all the time. Take a web app, for example. If the webserver is directly Internet-facing, it has to deal with lots of keepalive connections, so the core webserver logic (hyper or equivalent) should be async. But if you're not dealing with too many active requests and don't care about the performance difference, I don't think there's any reason you shouldn't have all your request handling just use threads, sending replies to hyper with a channel and blocking when necessary.

Last I checked I couldn't find an ergonomic and efficient "half-async" bounded channel implementation. By which I mean one that allows you to treat the sender as blocking and the receiver as async, or vice versa. That'd be really useful for writing synchronous programs that use async libraries. I certainly don't see any reason one couldn't exist. Maybe it already does and I missed it.

Today I'd maybe take a look into crossbeam and their queue implementations for this kind of synchronization.

https://docs.rs/crossbeam/0.8.0/crossbeam/

See https://en.wikipedia.org/wiki/Green_threads

There's a reason that most OS native UI frameworks use some sort of event polling structure(WNDPROC[1], Looper/Handler[2], etc) because they're a nice structure for efficient handling of events. Tokio lets you do that across a diverse of events and get the same type of savings.

[1] https://en.wikipedia.org/wiki/Message_loop_in_Microsoft_Wind...

[2] https://developer.android.com/reference/android/os/Looper

Furthermore, some classes of major macro-optimizations can't be implemented effectively if you do everything with kernel threading. This is the reason most modern server software architectures tend be thread-per-core pure async with no real multithreading per se -- it is for the performance.

At a more practical engineering level, these architectures are not more complicated, just different. Some things are much simpler to design because most multithread coordination problems go away. It is nice to be able to write virtually all of your code in single-threaded style where you don't have to worry about locking and consistency, especially as concurrency increases. On the other hand, you have to learn how to design schedules because the OS will no longer be doing that (poorly) for you. It isn't free in that you have to develop expertise in things you may not know but it is often worth it.

Still a fan of async programming, but that's a false benefit in my books :)

No you’re not. There is a similar situation in Kotlin, which supports coroutines. Makes things more complicated and is often used for very questionable reasons.

This is why I’m excited about project Loom, which will use the same old thread abstraction but can be configured to use fibers under the hood instead. Java devs don’t even care that its not traditional threading, its the same API! This simultaneously solves the „coloredness“ problem of functions.

Why is it complicated? Because you didn't bother to learn new API and it is sufficiently different from older one?

I.e. parallelism means executing several tasks on different compute units (cores) at the same time (multiple threads abstraction). Concurrency simply means that you can execute several tasks over a period of time, but it can happen on the same compute unit (so even within one thread). Tasks could be interleaved and still all progress over time.

I guess some ideal usage is a combination of parallelism and concurrency, but using a separate thread for each task isn't necessarily the most optimal method, because threads have their gotchas like context switching and etc.

I've seen that a lot on the internet and I guess it must depend where you come from, because I find async/await orders of magnitude easier to reason about than threads+channel (and I'm not even talking about using epoll manually, which is just inscrutable as soon as there is a little bit of complexity involved)

My understanding from reading along with many of these conversations is that the ultimate goal of the async IO frameworks is to amortize system call costs across multiple IO operations, instead of having >1 per.

I've heard some speculation about system call overhead going up in order to guarantee correctness (for Spectre and Meltdown class scenarios, but also for some other classes of concurrency issues). In which case io_uring is the carrot and system call slowdown the stick.

For example, futures in Rust can be used with both OS threads and lightweight tasks. Tokio is mostly agnostic about the choice of the executor.

It definitely takes some time to grok async Rust (even if you come from C#/JS), but I think it really shines once you get to know it, similar to the benefits you get from learning about iterators & higher level functions as opposed to plain loops.

For instance, I've recently implemented the Raft protocol as part of a distributed algorithms course. Using Tokio and a single threaded executor made the implementation fairly readable, mostly relying on few async constructs(futures, tasks, channels and select loops) to model fairly complex behavior (bidirectional messaging, multiple states, timeouts, etc..) Doing so in a more traditional callback oriented style would've required maintaining a very complex state machine(in addition to the state machine of the algorithm itself)

Also, Async/Await originated in C#, a language which already supports threads(and many other concurrency models), not JavaScript which historically relied on callbacks

Isn't async by design more efficient even when you have multiple threads you can spawn? My understanding is the event loop would do async tasks in the thread's quiet times, and put those tasks to sleep while it's waiting for io and other things, meaning the thread isn't blocked. Comparing that to (my understanding of) threads, while you're not blocking the main thread, you're still blocking the spawned threads while waiting for io.

Isn't this thread blocking something you would want to avoid if you can regardless of whether or not you have additional threads to play with?

You gain some distinct advantages too by doing this, because you can then just run a single thread in your worker pool and if you ever need to make the program multi-threaded - god forbid - you can add locks to shared resources (or in Rust's case the compiler will help you with this) and then bump up the number of threads.

CPUs are really really fast today, I think engineers generally underestimate the amount of performance you can get with a single-thread and non-blocking I/O.

Async language constructs make this process a bit easier. The only issue IMO is that it is awkward to call async functions synchronously, or that it can have hidden costs to do so. I think languages will improve on this.

I believe on Linux, using non-blocking system calls on threads can help reduce expensive context switches too... whereas spawning multiple threads and having them use blocking system calls can cause more context switches.

I will say though.. I've seen developers just async-ify everything in codebases without thinking about why or if it is beneficial.

I don't know about the end of this sentence, but yeah, I'm the kind of person who enjoys very much writing async code and finds that it (sometimes) models the problem much better than sync code and much, much, much better than event-driven/polling programming.

But then, I'm also the kind of person who enjoys coding with CML channels (aka Rust mpsc channels aka Go channels aka Erlang mailboxes aka pi-calculus channels etc.), so I guess I might be a mutant.

The model doesn't work for all forms of concurrency, but I think it works for a lot of things that people at the "top of the stack" (application developers) do.

I don't know what kind of code you're looking at, in general, but you should be able to massage most async stuff into a list of things if you're not in callback soup. Granted, lots of people don't try to stay out of the callback soup, but.... I feel like even that is better than just like "try to validate concurrency invariants", which is a much harder problem for arbitrary code IMO?