News for hackers. Stuff that matters.
Re:Red and blue functions are a good thing by diragon on Monday April 26, 2021 @07:35AM (#26941430)
First it was "Rust's async isn't f#@king colored!" (https://www.hobofan.com/blog/2021-03-10-rust-async-colored/)

Then "Rust async is colored, and that’s not a big deal" https://morestina.net/blog/1686/rust-async-is-colored

And now it's a good thing.

Honestly, this sounds like somebody struck Achilles' heel of Rust and it hurts. They're neatly covering 3 stages of grief over these 3 articles, and I wouldn't be surprised if we could uncover the other 2 in the blogosphere somewhere.

Re:Red and blue functions are a good thing by mumblemumble on Monday April 26, 2021 @07:53AM (#26941535)
> They're neatly covering 3 stages of grief over these 3 articles

No. It could only be something like three stages of grief if a single person were cycling among these positions. When it's just three different people writing three different blog posts, it's either three completely unrelated things, or a debate.

Re:Red and blue functions are a good thing by pwdisswordfish8 on Monday April 26, 2021 @08:48AM (#26941950)
Or maybe, and just maybe, the whole ‘stages of grief’ concept is not actually predictive of which opinion is actually right, and it’s just a cheap rhetorical trick providing an excuse to patronizingly dismiss someone else’s scepticism towards an idea.
Re:Red and blue functions are a good thing by adonovan on Monday April 26, 2021 @09:13AM (#26942160)
Exactly. Kubler-Ross, the author of the original "stages of grief" paper, has since tried to clarify her horribly misunderstood idea: the "stages" are neither essential nor sequential, but are merely some of the many ways in which people react to grief.
Re:Red and blue functions are a good thing by ghosty141 on Monday April 26, 2021 @09:27AM (#26942271)
I doubt the commentor was 100% serious when applying the "stages of grief" to blogposts.
Re:Red and blue functions are a good thing by mumblemumble on Monday April 26, 2021 @11:14AM (#26943368)
True, but whether or not something is a joke and whether or not it is rhetorically misleading are orthogonal characteristics.
Re:Red and blue functions are a good thing by brokenkebab on Tuesday April 27, 2021 @07:25AM (#26954490)
I don't think it's misleading. I read the OP comment as a humorous warning that we should be cautious around arguments which very probably came out of wish to defend beloved PL. These opinions often bring interesting thoughts, but more often then not they are built as "I need to proof X, therefore I select facts for it"
Re:Red and blue functions are a good thing by clarge1120 on Monday April 26, 2021 @10:33AM (#26942904)
Thank you. The Gamma radiation was starting to glow.
Re:Red and blue functions are a good thing by Sacho on Monday April 26, 2021 @07:58AM (#26941581)
The two articles you list actually agree with each other - the second one is basically a more thorough argumentation of the points raised in the first one.

For example, the first article agrees that "async is colored"(despite the title), but that the big issue of "colored" functions, "You can only call a red function from within another red function", doesn't exist in Rust. This is also a major point in the second article.

I think a more accurate description would be that Rust async is informed by painful async implementations(Javascript) and has tried to avoid most of their shortcomings.

Re:Red and blue functions are a good thing by skohan on Monday April 26, 2021 @07:48AM (#26941498)
I actually have found all the angst and consternation around the "coloring" of Rust's async approach to be rather funny, because there are heaps of concepts in Rust which introduce something analogous to function coloring.

A good example would be lifetimes. Once you introduce lifetime parameters into a data type, it starts to introduce ballooning complexity into everything that touches it.

Re:Red and blue functions are a good thing by orra on Monday April 26, 2021 @08:18AM (#26941720)
Is that problematic in practice? Write functions that are pure or take references (not ownership), and for the rest, use a judicious .copy()?
Re:Red and blue functions are a good thing by skohan on Monday April 26, 2021 @09:10AM (#26942132)
Yeah sure it's problematic. For instance, if you have a deeply nested struct, and you have to add a lifetime somewhere in the bottom, now you have to propagate it all the way up to the top level. Also if you run into a case where you need an explicit lifetime in a function somewhere, this ends up propagating outward.

I've run into cases with Rust where I refactored a significant portion of a program to avoid nested references and explicit lifetimes in a way I haven't had to in other languages

Re:Red and blue functions are a good thing by fauigerzigerk on Monday April 26, 2021 @10:21AM (#26942785)
Yes, but you only have three choices basically:

a) Accept the performance hit that comes with GC or defensive copying.

b) Convince yourself that developers can keep all those potentially dangling references in their heads at all times.

c) Use something like Rust lifetimes.

My preference would be (a) whenever possible, (c) when absolutely necessary, and never (b)

Re:Red and blue functions are a good thing by zozbot234 on Monday April 26, 2021 @10:33AM (#26942909)
You don't need GC or copying. Rust supports reference-counted handles that will never dangle, and will only leak in very rare cases involving self-referential data structures.
Re:Red and blue functions are a good thing by fauigerzigerk on Monday April 26, 2021 @10:34AM (#26942924)
I consider reference counting a form of GC - a rather slow one with lots of potential pitfalls.
Re:Red and blue functions are a good thing by pkolaczk on Monday April 26, 2021 @11:37AM (#26943694)
A "rather slow" is quite debatable. Tracing GC gets speed advantage at the cost of much higher memory and energy usage, unpredictable pauses, cache thrashing, etc. If you try to make these things equal, tracing can become embarrassingly slow, or even just crash with OOM.
Re:Red and blue functions are a good thing by nickez on Monday April 26, 2021 @10:54AM (#26943134)
It might be slow, but it is deterministic which is more important in systems programs.
Re:Red and blue functions are a good thing by Joker_vD on Monday April 26, 2021 @11:16AM (#26943406)
In what sense it is deterministic? Any refcount decrement may or may not be the last one and that may or may not cause a huge graph of objects being free()d which (even without destructors) takes unpredictable amount of time.

Then again, malloc() itself is usually not deterministic in a sense that it may take an arbitrary amount of time to make an allocation: many implementations shift work from free() to malloc() to get better asymptotic time complexity.

Re:Red and blue functions are a good thing by cesarb on Monday April 26, 2021 @11:51AM (#26943873)
> In what sense it is deterministic?

In at least two senses:

1. Memory usage: with reference counting, at any given moment, the amount of memory used is the minimum necessary for all live references.

2. Timing: with reference counting, the only moment when memory will be released (and their corresponding destructors called) is when a reference count is decremented.

> Any refcount decrement may or may not be the last one and that may or may not cause a huge graph of objects being free()d which (even without destructors) takes unpredictable amount of time.

That's actually predictable: it will always happen precisely at the last refcount decrement for that object, and will release only that object and other objects owned exclusively by that object, and nothing else. The size of that "huge graph of objects" is bounded, which makes the time taken to release it predictable.

Re:Red and blue functions are a good thing by Joker_vD on Monday April 26, 2021 @01:07PM (#26945006)
> The size of that "huge graph of objects" is bounded, which makes the time taken to release it predictable.

By the same line of reasoning, since the total size of the heap is bounded, it makes the time taken to sweep through it predictable.

Re:Red and blue functions are a good thing by cesarb on Monday April 26, 2021 @01:23PM (#26945220)
Unfortunately, that's not the case, since you can't easily predict the number of passes over that heap, nor can you predict when they will happen.
Re:Red and blue functions are a good thing by pkolaczk on Monday April 26, 2021 @11:43AM (#26943765)
The problem of destructor chains has been solved a long time ago. You can deallocate on another thread or deallocate in small batches during allocations.

Also pauses from malloc/free are orders of magnitude smaller than pauses caused by tracing GC. Low pause tracing GC creators make it a big news when they manage to get pauses below 1 ms, while malloc/free operate in range of tens or maybe hundreds nanoseconds.

Tracing GC can pause a fragment of code that does not make any allocations. Refcounting can't. Refcounting has only local effects - you pay for it where you use it. But when you add tracing GC, its problems spread everywhere.

Re:Red and blue functions are a good thing by ghusbands on Tuesday April 27, 2021 @05:08AM (#26953675)
Delayed destruction (either on another thread or batches) removes the possibility of safely using RAII via finalizers, which removes one touted advantage of refcounting.

Most modern implementations of refcounting end up having a cycle collector, anyway, which also means you can end up unexpectedly paused, removing another touted advantage.

There are always trade-offs, so you can't really call any of it solved or make absolute claims. In general, GC implementations (including initially-refcounting) tend to converge over time in both performance and features.

Re:Red and blue functions are a good thing by pkolaczk on Wednesday April 28, 2021 @07:09AM (#26968016)
> Delayed destruction (either on another thread or batches) removes the possibility of safely using RAII via finalizers

No, it doesn't in general. Delayed destruction can be used selectively only for these object graphs that could cause large batches of deallocations, and only once you find by profiling that you really have a problem. Most object graphs are not like that, so don't penalize the average case for the edge case you might even not have at all. You can still use synchronous deallocation on the objects where immediate deallocation is needed. Secondly, I bet delayed deallocation on a background thread happens a lot sooner than waiting for the next GC to happen and to call the finalizers (which may never happen). With delayed deallocation the system knows how much there is to deallocate. With tracing GC there is no such knowledge until tracing is over.

> Most modern implementations of refcounting end up having a cycle collector, anyway, which also means you can end up unexpectedly paused, removing another touted advantage.

That's why Rust doesn't have a default cycle collector. Reference cycles are IMHO a non-issue in practice. There is no point in paying for cycle-collection by unpredictability of the whole program, when it helps only with edge-cases.

> In general, GC implementations (including initially-refcounting) tend to converge over time in both performance and features.

I'd frame it differently: all-purpose one-size-fits-all memory management solutions are almost universally suboptimal and easily beaten by solutions which give more control in programmer's hands. Rust is about providing that control about fine details and giving you a set of handy tools. The toolset might be slightly harder to learn initially due to borrow checker and lifetime rules, but over time I didn't notice any productivity hit from that (actually quite contrary, but this is a different topic).

And somehow, even without me doing any heavy optimisations, my Rust programs tend to use 100x less memory and perform way faster than my Java programs. How can it be explained, when I have much more Java experience than Rust experience?

Re:Red and blue functions are a good thing by Joker_vD on Wednesday April 28, 2021 @09:08AM (#26968878)
> Reference cycles are IMHO a non-issue in practice

Any double-linked structure makes it very much an issue. And before you reply with "who uses linked lists anyway", consider the publisher/listener pattern: a publisher has the list with its listeners, a listener usually knows what publisher(s) it's subscribed to.

Re:Red and blue functions are a good thing by pkolaczk on Wednesday April 28, 2021 @10:12AM (#26969572)
1. You can solve these issues with weak references.

2. A lot of stuff that need doubly-linked lists is enclosed in libraries anyways, so you don't need to worry about it.

3. Reference cycles are ugly even in GCed environments and generally should be avoided. They make programs hard to understand and follow.

Re:Red and blue functions are a good thing by Joker_vD on Wednesday April 28, 2021 @10:51AM (#26970031)
> They make programs hard to understand and follow.

Really? Using a lookup table to annotate objects externally instead of stuffing annotations inside the objects themselves makes programs harder to understand and follow? No, it's having objects with hundreds of fields that get gradually filled by different places in code too makes programs harder to understand, because it's unclear what data is available where and when. Having children in tree-like structures have pointers to their parents makes programs harder to understand and follow? Again, no: when traversing a tree one can do without to-parent pointer available in the children by stuffing it in the traversal context, but that too makes the program harder to understand.

Re:Red and blue functions are a good thing by pkolaczk on Wednesday April 28, 2021 @12:58PM (#26971588)
If you stuff annotations inside objects in order to locate other objects, you're technicality still making references. Whether you use an integer index, string key, pointer or a thing called "reference" in your language doesn't really matter that much - it is just an implementation detail.

What I meant was you should structure the program in a way to avoid circular dependencies, instead of pretending to not having them by using indexes/keys.

> Having children in tree-like structures have pointers to their parents makes programs harder to understand and follow

Yes. It is not a tree anymore. It is a graph, and even not a simple DAG. More state = harder to follow and more ways it can go wrong. For example pointers could not properly match (the parent pointer doesn't point to the correct parent). Circular dependencies also make it impossible to initialize objects without temporarily inconsistent states.

BTW I've seen real memory leaks in GCed (traced) programs because of improper use of such back-references to parent. It is really easy to screw up such structures e.g. when copying, by forgetting to update some of these references and letting them point to the old structure, keeping it in memory.

And what's even worse are circular dependencies between different components in a program. I worked on big codebases where figuring out the proper initialization order was a huge PITA because of reference cycles. And failures to get it right manifested with subtle null pointer exceptions. I'd really love the language forbid them and forced developers to strive for simpler designs.

Re:Red and blue functions are a good thing by pornel on Monday April 26, 2021 @12:01PM (#26944035)
This is because the original article talked about a couple different things under the same title of a "coloring" problem:

1. JS has no way to wait for results of async functions in a sync context. This creates hard split between sync and async "colors". This is what most people agree is the major problem.

2. C# can avoid the split, but the syntax for using sync and async is different. The original article wasn't very clear about why this is bad, other than it exists.

The second case gets interpreted as either:

2a. Programming languages should be able to generically abstract over sync/async syntax.

2b. Programming languages shouldn't have a different sync/async syntax in the first place.

Rust's case is consistently:

- It generally doesn't have the problem #1. Sync code can wait for async. Async code can spawn sync on a threadpool. There are some gotchas and boilerplate, but this kind of "color" isn't an unsolvable dead-end like it is in JS.

- It has distinct syntax and execution model for async, which still counts as "color" (#2) per the original article. However, Rust can abstract over sync and async, so it doesn't have the problem #2a. The syntactical distinction is very intentional, so it doesn't consider #2b to be a problem at all.

Re:Red and blue functions are a good thing by merb on Monday April 26, 2021 @05:49PM (#26948665)
the problem is that if you introduce async, your could should preferably async all the way to the bottom. of course you can trick yourself by using thread pools and move long running functions to them, but not everybody knows that.

of course sync functions have the same problem (not the same problem, but it's the same concept, just in the other way), except that most frameworks just deal with it at the very front, so developers do not need to deal with it.

Re:Red and blue functions are a good thing by pornel on Tuesday April 27, 2021 @07:51AM (#26954682)
The whole point is that use of async doesn't need to be infectious. You may want to make everything async for elegance and consistency, but it's not a requirement strong enough to cause problems. If you have a library that is sync-only, it's fine. If you have a large project and can't be bothered to rewrite everything to be async, it's fine.

"Not everybody knows (how to do) that" is a weak argument, because it's a relatively straightforward thing to learn: if it can block, wrap it in `spawn_blocking()`.

Re:Red and blue functions are a good thing by ibraheemdev on Monday April 26, 2021 @07:25PM (#26949613)
> of course you can trick yourself by using thread pools and move long running functions to them

How is that tricking yourself? Blocking code exists, and you will almost always run into it one way or the other.

Re:Red and blue functions are a good thing by tadfisher on Monday April 26, 2021 @07:51AM (#26941521)
These are three different opinions from three separate people. Please consider that the Rust community does not share one consciousness, as your post doesn't add anything useful to the conversation.
Re:Red and blue functions are a good thing by wffurr on Monday April 26, 2021 @07:55AM (#26941557)
The other links are interesting context for the debate over async in the Rust community. Too bad GPs post didn’t frame it that way.
Re:Red and blue functions are a good thing by darthrupert on Monday April 26, 2021 @08:18AM (#26941721)
Is that why 5 different people answered parent's comment dismissingly over a 15 minute period?
Re:Red and blue functions are a good thing by lukebitts on Monday April 26, 2021 @09:39AM (#26942364)
So are you arguing that the rust community shares one consciousness? Cause otherwise I don’t see your point
Re:Red and blue functions are a good thing by fshbbdssbbgdd on Monday April 26, 2021 @09:02PM (#26950441)
If any community were equipped to navigate the challenges of sharing one consciousness, surely it would be Rust.
Re:Red and blue functions are a good thing by jackcviers3 on Monday April 26, 2021 @12:26PM (#26944424)
But they are a good thing, from a type perspective, they encode the idea that a side-effect is going to happen at the type level, and then force you as the caller to handle that fact or pass the buck to your caller.

Effect types are a signal for all programmers that "Here be dragons". They outline parts of your code which may not be safe to reorder during refactoring. And they can cover so much more than just asynchronous I/O. [1]

1. https://en.wikipedia.org/wiki/Effect_system