> For eg. it's a near identical scenario in python async.
I'm at no point claiming that Python is colored (the "What Color is Your Function?" post does). It seems that by my interpretation Python may indeed be as uncolored as Rust (I've only used it in a very limited fashion). Can't compare Scala as I haven't used it.
From the original post:
> This is where the “red functions can only be called by red functions” rule comes from. You have to closurify the entire callstack all the way back to main() or the event handler.
I think that very much supports my point, and I don't see why async runtimes should not be allowed as a tool to call a async from a sync function. In some languages (like JS) that have their own runtimes, you don't really have that choice though.
Rust assists you by enforcing the distinction with compile time type checking. But all that is doing is yelling at you that you are trying to mix the two fundamentally non-composable things together. You still have to do the conversion yourself. You still have to do extra work to accomplish the conversion.
Also it's just ugly to have layer after layer of async/sync wrapping going on. So what ends up happening is that you end up preferring one color over the other. Either you try to keep everything asynchronous or synchronous to avoid the whole thing.
I disagree with this take; I understand the point of the original “What color is your function?” as colored functions creating an impassable divide between Promise-wrapped values and sync code.
Indeed, in the end, the author mentions e.g. C# as not being colored, because even if sync/async functions are different, it features escape gates to unwrap promises and bring them back into the sync world.
> trying to mix the two fundamentally non-composable things together. You still have to do the conversion yourself.
I think the fact that you can do the conversion shows that they are composable. As I've written in the blog post, I don't mind verbosity verbosity much, as long as it gets me some control.
> The point of the colored function is that the color is a leaky abstraction.
And I think that's where the difference lies. In JS, an async function will leak all up your callstack, while in contrast in Rust, you can clearly control where the leaking stops.
> Also it's just ugly to have layer after layer of async/sync wrapping going on. So what ends up happening is that you end up preferring one color over the other. Either you try to keep everything asynchronous or synchronous to avoid the whole thing.
I think that's fair, but I haven't really had much problems with that in practice.
Go. Go eliminated the distinction. Everyone of his points about JS/C#/Dart all tick the same boxes he calls out as problematic in Rust as well. The size of the problem may be simpler but you haven't eliminated it.
Everything is composable if you wrap it appropriately. Saying that since you can convert something into a newer more composable form doesn't imply that the original is composable. It just means that since we can write code everything is composable with the right amount of work. What people actually mean when they say two things are composable is that you don't need a shim to compose them together.
Same problem with Go's `Context` or error as a return value.
Same problem with pure/impure functions, or functions returning optional, or platform-specific functions, etc.
Sync/async is not special, it is just one instance of a problem that comes up pretty much everywhere (Algebraic effects may be a solution, but I'm not sure).
Zig has the same codebase with regards to allocation as well; there is no "allocator_std" or some nonsense like that.
Where is the paradox, exactly?
As I said in the original comment, sync/async and no-alloc/alloc or no-error/error are in a parallel situation. And in a language design you can either expose “colors” to your users, or hide it through some magic and a global state. I personally prefer the former, and I liked Zig's idea to expose the allocation explicitly, instead of what Rust did initially (there are slowly rolling out the option to use a specific allocator for collections, which is good even if it took almost 6 years after 1.0). With error Handling, Zig also took the same route with error sets instead of exceptions.
That's why I don't think the colorblind async mechanism is a good fit in the language (and it adds some cognitive burden, because newcomers need to learn that looking at code full of async/await doesn't necessarily mean they are looking at non-blocking code).
(Also, async-std is a third party library with little in common with std and whose name has been chosen for “marketing” reason, so I' don't really think the comparison is relevant.)
It depends on what you mean by "separate codebases." asyc_std is a third party package someone made, so it's a separate codebase because of that.
We could also include all of it (or something like it) and it would be a "single codebase," but I think that's more about your wording than your point.
If you make code that requires allocation and does not have an allocator passed into it, there is literally nothing stopping you from allocating anyway. Zig does not stop you from, for example, heap allocating without the usage of an allocator (Zig does not know what heap allocation even is). You can directly use the std page_allocator (and others) wherever you want, too.
There are very few good reasons to do this (e.g. spawning a thread requires particular allocation; using an allocator might not be sound depending on the OS).
All you have to do is document that it allocates memory and that's fine; there is no split.
But nonetheless I think there is a split (at least for now). Zig async function frames are different than regular functions, so even if syntactically, `foo()` could magically be switched between sync or async based on io_mode, in practice if you have a recursive call tree doing fs operations, it'll work in one mode (up until a stack overflow, that is) and throw a compile error upfront in the other. Presumably this would not be an issue anymore once [0] is implemented.
This is not entirely correct analogy. You can embed your allocator inside one of the parameters, too, as a strategy (perhaps a certain type of "object" should carry it's allocator with it).
Then uncolored generic functions could operate on allocator-bound and allocator-unbound "objects" alike, even if say the generic function calls a "object method" that performs an allocation in one case but not the other.... You just have to be careful about how to handle errors.
There is no equivalent for async.
If you were going to be serious about function coloring in zig, the place where there is function coloring is functions with error return values but that's actually a reasonable place to have function coloring, and still, it's easy to build seams in both directions, so a red function can call a blue function, and a blue function can call a red function pretty easily.
Sure there is, a symmetrical one: you can embed the Future/Promise inside one of the return value of you sync function. Both work, but both are equally hackish.
> If you were going to be serious about function coloring in zig, the place where there is function coloring is functions with error return values
I actually address it a bit below.
I'm most familiar with Go, and there I've gotten very used to the idea that my functions will probably need an error return, so e.g. I write my main func to call into a run(...) error func first thing. I've had to go back and replumb things with context, and it's no fun. Similar strategies apply in JavaScript, where you just know that any kind of button press will probably need to be a Promise because it might have to wait on a network call or whatever. It's a pervasive problem.
E.g. let f call g which calls h. And suppose you replace h with an async function. Then you'd have to manually annotate g and f as async functions too. You don't have to annotate a function as being allocating or not, so the allocation "color" isn't nearly as bad. Languages like Go or Julia or Zig that have "colorless" coroutines are much more robust.
And suppose you replace h with an allocating function, then in Zig you have to manually add a allocator as a parameter in g and f and update their call site accordingly, because now they are also allocating.
And suppose you replace h with a function that returns an error, then in Go you have to manually add an error as a return value in g and f.
h become impure? Then g and f are not pure anymore either.
h becomes Windows specific, then g and f are Windows specific[1].
The situation is highly similar.
[1]: or you need to write a non-windows version of the work done by the h function, which fundamentally equivalent to calling block_on on an async function to make it a blocking one.
Did JS Solve(tm) "colored" functions?
No, Promises can only be interacted with within the async framework. There is no ways to
1. Call an async function/get a Promise in a sync function
and
2. “Extract” the value in the promise and use it as naked value.
JS would need e.g. a “stop there, wait for the promise to result, unwrap its content and give it to me” instruction to be like Rust in this case.
For instance, this will essentially halt your runtime:
let y;
Promise.resolve(123).then(x => { y = x; });
while (!y) {}
console.log(y)The await/async feature seems like a sensible addition to JavaScript, but I'm open to the idea that there are alternative solutions that would be even better. As an example, I don't know much about what Java is up to but I understand it's not going with await/async exactly.
It might keep things simple, but I don't know if this is really a sensible design constraint going into the future. Parallelism is where performance is going to come from in the future, and having the web - the most common way most people consume software - be single-threaded seems like a massive waste of hardware.
If I have a 16 core processor just so the add trackers in each of my chrome tabs can run in parallel, this is a world I don't want to live in.
Others have already pointed out I wasn't strictly correct when I put there's exactly one thread, on account of the Web Workers API. You can write parallel code for the web if you really need to, but it's somewhat walled off from the main JavaScript environment, so we still get to keep much of the benefit of the single-threaded model. For instance, all UI events are handled in the main thread, and that's not something you're empowered to break or override. That's a good thing.
Really though, if your page can't get by on one core of a modern CPU, it probably shouldn't be based on web technologies at all. Ordinary websites have no business writing parallel code just to prepare the DOM. I'd rather the average site not have the option at all. The web will never be the ideal platform for writing parallelised number-crunching code like modern high-performance game-engines, neither should it aim to be so.
> If I have a 16 core processor just so the add trackers in each of my chrome tabs can run in parallel, this is a world I don't want to live in.
I don't follow. What can't you currently do with that enormous computational horsepower? The browser already has plenty of power. Spam and bloat aren't going to go away from the web any time soon, and anyway they're an argument against adding even more features into the browser.
The downsides of the web as a platform wouldn't go away by improving JavaScript's ability to execute code in parallel. Web-based UIs would still seem clunky and half-baked compared to native apps, and they'd still use far more computational resources than native apps.
JS parsers, compilers, and even some garbage collectors are threaded.
IO (arguably the most common case for threads) is threaded behind the scenes.
Web Workers allow using multiple cores/processes for typical parallel programming. Message passing exists as do shared memory and even atomics (note: node, firefox, and chromium-based browsers all have it available after Spectre/meltdown mitigations were added, but it's still disabled in Safari).
What JS really needs is a CSP or Actor model on top to abstract away threads/OS processes/cores/hardware and let the VM handle them seamlessly like in Erlang/BEAM.
What kind of website would benefit from this?
If it's concurrency, is it something that could be done in a library?
As for parallelism, I don't think many SPAs and PWAs really need to be able to make full use of multicore for acceptable performance, do they? If you're writing heavyweight number-crunching code, that belongs either server-side or in a native application.
If you try to port Red Dead Redemption 2 to JavaScript, you're going to have a bad time. That's not the fault of the web, it's just not an appropriate choice of platform.
Like I said in another comment in this thread, the downsides of the web as a platform wouldn't go away by improving its support for parallelism or for concurrency. Web-based UIs would still seem clunky and half-baked compared to native apps, and they'd still use far more computational resources than native apps.
I also do prefer native, yet as mentioned, web browsers or web widgets as GUI toolkits are a fact and aren't going away.
Specially if everyone keeps pushing packaged ChromeOS apps, aka Electron.
JS being single threaded means that while some tracker is parsing a big json blob your website becomes unable to handle input events until the tracker is done.
Let’s say there is an option to do this in some form of threaded worker or coroutines then the main UI thread does not become blocked.
I agree that it's a neat idea to use asynchrony to move specific heavyweight browser-native computations out of the JavaScript thread, but is JSON parsing really a significant performance issue?
> JS being single threaded means that while some tracker is parsing a big json blob your website becomes unable to handle input events until the tracker is done.
I'd rather parallelism not be a viable option for accelerating trackers. Users are known to hate unresponsive UIs. I'd rather that web trackers bring an unavoidable performance penalty.
> Let’s say there is an option to do this in some form of threaded worker or coroutines then the main UI thread does not become blocked.
You've just described the Web Workers API.
I don't know whether today's tracker scripts use Web Workers. I imagine probably not though, I think the Web Workers API is only appropriate when you're doing substantial computation.
``` async function main() { await stuff(); }
if (module === require.main) main(); ```
> One the one hand it's a great article that hits the nail on the head why using async/await in JS (and other languages) is very painful
> Rust's async/await even though - you might have guessed it - it doesn't apply!
So the author starts off by declaring that the colored functions problem only applies to other languages but not to rust.
Soon after, he takes exception with the claim "You can only call a red function from within another red function".
His primary argument in support of this is that in rust you can execute an async function blockingly in an executor. And to do vice versa, i.e. call a sync function in an async loop, that you can either block the event loop or hand it off to a different machinery.
The fact is, both of these things can be done in other async runtimes as well. For eg. it's a near identical scenario in python async. Like others have pointed out in this comment thread, this is true in Scala too. Perhaps the author is unaware of this?
So yes, rust does indeed have the colored function dichotomy, as any async runtime would, the author's angry assertions to the contrary notwithstanding. (I say this as a fan of both rust as well as async runtimes, which I feel are a nice fit for certain use cases.)