The problem is that the library author must then code for the worst case and the compiler won’t help them; injecting preemption into synchronous code can expose things about it that used to be unobservable, but now can make the difference between success and deadlock. (Similarly to how injecting mutation into pure code can expose—and bar the optimizer from changing—things that were previously unobservable, like “in which order does map() visit elements?”.)

There still will be async-compatible and async-incompatible libraries, except now you can’t tell which one is which except by looking at the documentation—look at the lengths R5RS goes to in order to define things like returning multiple times from inside a letrec while R6RS just flat out prohibits that.

Yes, this seems to be a pro-colouring comment while elsewhere in this discussion I make what seem to be anti-colouring comments. I really don’t know what the answer is.

Based on what I've seen in Rust's async, you don't actually want this in a low level language. Those calling scenarios are fundamentally different. If you magically make something async, other code can now suddenly mutate state that your function is referencing on its own stack and really mess things up.

In Rust this isn't an issue because you're not allowed to pass references into an async function that outlive the future. If you take a synchronous function that holds references and add the async keyword, you probably will get compile errors about this. I don't think Zig has anything like that.

>Zig's view on this, which I endorse, is that suspending to your event loop is not substantially different from suspending to the kernel.

No, that doesn't make sense, not even in the context of Zig. Suspending to the event loop allows other coroutines to run. Blocking in a syscall stops the whole program and all the coroutines.

But to address your comment directly, no it's not, not when you're actually building a different state machine with different constraints each time. I actually wish we could put this protocol code in one C library in a way that only handles the protocol logic and not the I/O part, and then we'd be done with it forever. But you know that wouldn't be good enough for some, those libraries exist and people still don't use them and instead they go and write their own in the new fancy language with their cool new concept for a state machine.

You can have one Python library at least[1], h11 and h2 in particular make a pretty convincing case as the ultimate in Python HTTP. (You will always have wrappers, though, because not all levels of detail are appropriate for all uses—similar to how imagemagick/libgd/etc. wrap libtiff which wraps libjpeg and wrap libpng which wraps zlib and so on.)

The problem, as far as I can see, is that to avoid calling out to I/O you need inversion of control, and either you convert your code into CPS and state machines manually, which almost impossible, almost unreadable, and makes layering libraries a distant dream (not that people don’t do it anyway), or you use some sort of generic inversion-of-control mechanism like async/await or call/cc, which really wants dynamic memory allocation.

It seems like being able to accurately track stack usage (per module, not per program) could solve this, but that means (we must abandon arbitrary recursion and) the compiler must be able to track arithmetic equations, which is liable to make Gödel and Turing rise from their graves just to shout NOPE! at you. There’s also ABI: what do you do when a new version of a protocol library requires a dozen more bytes on “the” stack, and you were putting these on your stack or in static storage?

(As far as I can tell, Rust’s no_std answer to ABI of this kind is “no.”)

[1] https://sans-io.readthedocs.io/

To that end, Zig does have another kind of function coloring that's useful there: functions which take an allocator parameter. I like that about Zig, I wish C++ and Rust were better about doing it. Sadly you can still hide this by referencing a static allocator in Zig so it's not totally there yet on the language level.

Yes, this is a great approach. Unfortunately people in e.g. the Rust world tend to write crates that assume they can call malloc, recv, and send, rather than one crate just implementing the protocol logic and another that does all that other stuff.

Zig's way of doing things causes the library people naively write and use to call malloc, recv, and send to also be a library that just implements the protocol logic, reads from and writes to static buffers, and doesn't call malloc.

Edit to reply: Yes, Rust has no_std, but most libraries that implement a protocol are not divided into a no_std library that implements just the protocol logic and a separate std-using library that wraps it, nor are most libraries that implement a protocol asynchronously wrappers around a no_std library that is not async and only implements the protocol and makes 0 syscalls. A consequence of the fundamental non-composability of naively written Rust libraries is that users who want the wrong combination of asyncness and malloc-calling very frequently need to reimplement existing Rust libraries. A consequence of the fundamental composability of naively written Zig libraries is that they never need to do that.