News for hackers. Stuff that matters.
Re:Rust Is Hard, Or: The Misery of Mainstream Programming by ekidd on Thursday June 02, 2022 @08:53PM (#31602761)
> Most of the pain here come from the unholy trifactor: combining async, lifetimes and dynamic dispatch with trait object closures; which is indeed very awkward in practice.

Even in regular Rust, trying to get too clever with lifetimes can cause serious pain. The usual culprit is complex code that tries to never allocate memory. "Oh, well this closure borrows this parameter from the parent function, and then stores a reference to it in this stack-based structure inside the closure, and then we pass everything by reference to this higher order function..." Just say no.

If you add async to the mix, then you need to keep your ownership simple. If you have a long-running async function, then pass parameters by value! If you have a polymorphic async function, then return your result in a Box. (This also breaks up the generated state machine used to implement async functions, and can reduce binary size.)

So much of this pain is caused by premature optimization.

In C++, if you get to clever, you eventually make a mistake and segfault. In Rust, if you get too clever, it eventually becomes impossible to satisfy the borrow checker. Most of the time, the solution is to be less clever.

Async Rust was a fascinating experiment, but in practice, I think it turned out to be a tool that's best used conservatively. I've written some very stable and high-performance production code using async Rust. But I keep it simple when I can.

Re:Rust Is Hard, Or: The Misery of Mainstream Programming by mwcampbell on Thursday June 02, 2022 @11:14PM (#31603830)
> So much of this pain is caused by premature optimization.

Part of the problem is that Rust itself has such lofty aspirations to do it all (particularly "abstraction without overhead"), and is largely successful at that. So if I compromise on efficiency, it feels like my code is unworthy of the language it's written in.

Also, it's easy to feel that the slightest inefficiency puts us on a slippery slope to the bloat and slowness of something like Electron. I'd like to fight back against that bloat, the upgrade treadmill, and the rapid obsolescence that serves hardware makers but hurts poor people. On the one hand, there was good software in the 80s and 90s that made liberal use of heap-allocated, reference-counted objects with dynamic dispatch. On the other hand, there were slow, bloated desktop applications before the rise of Java, C#, Electron, etc. So I don't know where the right balance is at.

Re:Rust Is Hard, Or: The Misery of Mainstream Programming by Hirrolot on Thursday June 02, 2022 @11:43PM (#31604023)
I agree; if you use dynamic `Arc`s almost everywhere in your code, what's the point in a systems PL whose essence is in managing static lifetimes? Sometimes people use Rust just because many other languages suck; they are choosing between two evils: tedious programming in Rust or inadequacies of another language. It should not be like this. This is why I think we need a high-level, no-BS version of Rust.
Re:Rust Is Hard, Or: The Misery of Mainstream Programming by therockhead on Friday June 03, 2022 @04:11AM (#31605582)
>if you use dynamic `Arc`s almost everywhere in your code,

Right but you dont have todo this everywhere, just some sporadic use where it makes life a lot easier. It doesn't have to be all or nothing.

Re:Rust Is Hard, Or: The Misery of Mainstream Programming by AndrewGaspar on Friday June 03, 2022 @01:09AM (#31604521)
> This is why I think we need a high-level, no-BS version of Rust.

F# is probably what you're looking for if you want functional-lite programming with a strong type system, but don't want to deal with the pains caused by static resource management.

Re:Rust Is Hard, Or: The Misery of Mainstream Programming by chalst on Tuesday June 07, 2022 @06:12PM (#31660672)
F# has some nice things in it: I especially like how it handles multiple return values. But compared to the language it is most clearly decended from, OCaml, it ties you to .NET Core and it lacks an ML-like module system and macros.

If you want to use .NET, F# is a fine choice. But otherwise, and especially in the context of an alternative to Rust, I recommend OCaml.

Re:Rust Is Hard, Or: The Misery of Mainstream Programming by olalonde on Friday June 03, 2022 @03:51AM (#31605446)
It might sound superficial but the simple fact that there is seemingly a hard dependency[0] on Visual Studio (Code) is a major turnoff for me. I am quite attached to my Vim/command line workflow.

[0] https://docs.microsoft.com/en-us/dotnet/fsharp/get-started/i...

Re:Rust Is Hard, Or: The Misery of Mainstream Programming by Paradigma11 on Friday June 03, 2022 @05:15AM (#31605972)
Re:Rust Is Hard, Or: The Misery of Mainstream Programming by nickduggets on Friday June 03, 2022 @07:31AM (#31606933)
That link includes documentation for building and running F# projects using only the .net core framework. Seems like the main set of headings is misleading, making it sound like you must install VS or (shudder) VSCode to get F# installed.
Re:Rust Is Hard, Or: The Misery of Mainstream Programming by spopejoy on Saturday June 04, 2022 @10:58AM (#31621051)
Vote here for Haskell, which receives great hate on HN because it allows essentially unlimited abstraction.

Meanwhile it has one of the fastest concurrent+GC runtimes, obliterating Ocaml in this regard, and competing handily with JVM or CLR langs.

Finally, it has standard abstractions for concurrency and parallelism that are basically flawless.

Clearly, the hate comes from the association of Haskell with category theory, which is frankly useless/borderline harmful to the working Haskell programmer.

CT was critical to core library devs being able to deliver the main workaday abstractions that make Haskell such an unreasonably productive app languages. But since 2012 you can simply use these main abstractions and enjoy a level of safety and maintainability that rust, c++, ocaml, kotlin, java can't touch.

It's not perfect, but there's been huge improvements of late to tooling (cabal) so there's never been a better time. I really wish the tide could turn, because if you're willing to take on rust for low-level, you're missing out if you don't try haskell for everything else server-side.

Re:Rust Is Hard, Or: The Misery of Mainstream Programming by adwn on Friday June 03, 2022 @05:09AM (#31605932)
> I agree; if you use dynamic `Arc`s almost everywhere in your code, what's the point in a systems PL whose essence is in managing static lifetimes?

You can have the best of both worlds: In many cases, I simply put an object into an Arc, clone that a bunch of times, and pass/store it to wherever it's needed. Then, within loops, I pass a reference to local function calls (instead of cloning the Arc for every call). For any given v: Arc<T>, doing &*v is zero-cost – unlike in many other languages, where you'd have to pass the Arc itself, which would involve atomic increments/decrements without escape analysis.

Re:Rust Is Hard, Or: The Misery of Mainstream Programming by jen20 on Friday June 03, 2022 @12:07AM (#31604177)
Which parts of Rust would you give up to make this happen? And how would this be implemented?
Re:Rust Is Hard, Or: The Misery of Mainstream Programming by Hirrolot on Friday June 03, 2022 @01:10AM (#31604524)
Hide memory management under a language runtime. Make a single Fn trait/string type instead of multiple ones. Add effect polymorphism to deal with function colours. Remove async/.await and express it using algebraic effects, do the same for streams and iterators.

How it'd be designed and implemented is probably a theme for a separate blog post, not a HN comment.

Re:Rust Is Hard, Or: The Misery of Mainstream Programming by pjmlp on Friday June 03, 2022 @01:23AM (#31604593)
So basically OCaml.
Re:Rust Is Hard, Or: The Misery of Mainstream Programming by jules on Friday June 03, 2022 @08:38AM (#31607404)
OCaml is great, but it lacks traits / type classes. That plus some syntactic tweaks would make OCaml amazing for most general purpose programming.
Re:Rust Is Hard, Or: The Misery of Mainstream Programming by pjmlp on Friday June 03, 2022 @01:54PM (#31611565)
That is what functors and modules are for.

And then the O in OCaml is all about OOP.

Re:Rust Is Hard, Or: The Misery of Mainstream Programming by jules on Friday June 03, 2022 @06:22PM (#31614416)
Functors and modules are nice but do not have the convenience of traits and type classes. The problem with functors and modules is that you have to manually instantiate the right functors and modules when you want to use some operation. Type classes and traits build the right instantiations for you based on the types.

I want to write `toString (1,true)` not `ToStringPair(ToStringInt)(ToStringBool).toString (1,true)`.

Re:Rust Is Hard, Or: The Misery of Mainstream Programming by lelanthran on Friday June 03, 2022 @01:20AM (#31604577)
> Which parts of Rust would you give up to make this happen? And how would this be implemented?

Traits, maybe? Switch to an OO model more closely aligned to what the majority of developers understand. Dump all of the line-noise-type syntax.

My Own Toy Language (ComingRealSoonNow)^tm that I started designing had exactly one goal - prevent the majority of memory errors, not prevent ALL memory errors. All I wanted was to indicate when/where an object will be destroyed/mutated. That's enough for me.

Re:Rust Is Hard, Or: The Misery of Mainstream Programming by jules on Friday June 03, 2022 @08:34AM (#31607371)
Traits are the one part of Rust that should definitely not be given up. It is way better than the traditional OO model.
Re:Rust Is Hard, Or: The Misery of Mainstream Programming by pjmlp on Friday June 03, 2022 @04:19AM (#31605642)
Actually something like traits (as concept) is what powers COM, WinRT, the basis of Objective-C protocols that influenced interfaces in more common OOP languages.

Also the basis for one programming language that used to be widespread in the enterprise for quick and dirty solutions, VB and its ecosystem of OCX libraries.

It is more widespread that people think, because when many argue about OOP, they miss the full spectrum of how OOP is approached.

Re:Rust Is Hard, Or: The Misery of Mainstream Programming by NoGravitas on Friday June 03, 2022 @12:27PM (#31610491)
Traits seem, to me, a lot like generic functions and methods in CLOS (Common Lisp). It's just that CLOS doesn't bundle a group of generic functions into a trait, they're a la carte.
Re:Rust Is Hard, Or: The Misery of Mainstream Programming by pjmlp on Friday June 03, 2022 @12:48PM (#31610710)
Yes they do, they are called protocols.
Re:Rust Is Hard, Or: The Misery of Mainstream Programming by lelanthran on Friday June 03, 2022 @05:55AM (#31606287)
> Actually something like traits (as concept) is what powers COM, WinRT, the basis of Objective-C protocols that influenced interfaces in more common OOP languages.

So? Giving up Rust-Traits doesn't mean that I will give up on interfaces as a whole.

I don't like the way C++ does interfaces, for example, but that doesn't mean that my ideal language won't have interfaces.

Re:Rust Is Hard, Or: The Misery of Mainstream Programming by danielheath on Friday June 03, 2022 @04:58AM (#31605871)
Being able to build on libraries that are efficient means that your inefficient rust can still be lightweight.
Re:Rust Is Hard, Or: The Misery of Mainstream Programming by DeathArrow on Friday June 03, 2022 @01:17AM (#31604560)
>This is why I think we need a high-level, no-BS version of Rust.

F# to the rescue. :)

Re:Rust Is Hard, Or: The Misery of Mainstream Programming by xyzzy_plugh on Thursday June 02, 2022 @11:27PM (#31603927)
You hit the nail on the head, at least for me. I've found Go to be the right sweet spot between efficiency and productivity. I can't get anything done in Rust. My worst enemy is myself.
Re:Rust Is Hard, Or: The Misery of Mainstream Programming by galangalalgol on Friday June 03, 2022 @12:28AM (#31604301)
I like the thick stdlib in go, but it feels a lot slower even than java. The thin stdlib and giant dependency trees is my primary complaint with rust. I like most of the syntax, but async is not my threading style, I mostly use fine grained threads like rayon provides. I'm currently more excited about julia though. Same dependency hell, but super fast and flexible, and the code looks good.
Re:Rust Is Hard, Or: The Misery of Mainstream Programming by brabel on Friday June 03, 2022 @02:35AM (#31604969)
You think Java and Go are slow but like Julia? It takes 2 seconds just to fire up a Julia interpreter. Its runtime is extremely heavy.
Re:Rust Is Hard, Or: The Misery of Mainstream Programming by indeyets on Friday June 03, 2022 @06:59AM (#31606712)
well… you start it once and just use forever after that
Re:Rust Is Hard, Or: The Misery of Mainstream Programming by galangalalgol on Friday June 03, 2022 @08:00AM (#31607125)
That is what I usually do. And you can supposedly precompile everything if you just want to deploy a service so restarts are short. I have not done that though.
Re:Rust Is Hard, Or: The Misery of Mainstream Programming by kenhwang on Friday June 03, 2022 @01:23AM (#31604592)
I just tell everyone, just write Rust like you'd write Scala. Don't try to optimize the crap out of anything until you need it, especially if no one else is going to use your code.

Just make the allocations.

Even if you write it that way it's still significantly more performant and lightweight than the alternatives without much loss in productivity.

Re:Rust Is Hard, Or: The Misery of Mainstream Programming by secondcoming on Friday June 03, 2022 @04:45AM (#31605795)
This doesn’t bode well for Rust in Linux!
Re:Rust Is Hard, Or: The Misery of Mainstream Programming by crabbygrabby on Thursday June 02, 2022 @09:15PM (#31602999)
This is good advice. Write simple code first. Another generic thing I've come across is, "if the borrow checker is screaming at me when I hit a patch of code, I probably have a design flaw not a programming problem".

I've seen a lot of new people also prematurely over generalizing code. I know it sounds terrible to say, but if you probably aren't going to reuse it, you really don't have to prepare your code base just in case you might later. Decide that later and move along...

Re:Rust Is Hard, Or: The Misery of Mainstream Programming by FpUser on Thursday June 02, 2022 @09:33PM (#31603147)
>"if the borrow checker is screaming at me when I hit a patch of code, I probably have a design flaw not a programming problem".

This reminds me: "you are holding it wrong"

Re:Rust Is Hard, Or: The Misery of Mainstream Programming by estebank on Thursday June 02, 2022 @09:44PM (#31603221)
In industrial machines, there's such a thing as holding it wrong. In some cases, that means getting maimed (that old-school table saw doesn't care whether it chews on wood or flesh) or precluded through a clunkier mode of operation (you need to press these two buttons separated at roughly arm length to make sure that they can't be actioned while you have an arm in the way of the heavy arm-eating chunk of metal). The former is C and unsafe Rust, the later is safe Rust and GC languages.

You're trying to draw a comparison with a consumer product with a design flaw.

Re:Rust Is Hard, Or: The Misery of Mainstream Programming by FpUser on Friday June 03, 2022 @05:22AM (#31606036)
>"You're trying to draw a comparison with a consumer product with a design flaw."

Not at all. I just do not like to dance around bonfire with tam-tam and being told that this is the one and only way

Re:Rust Is Hard, Or: The Misery of Mainstream Programming by kaba0 on Friday June 03, 2022 @06:38AM (#31606578)
It's not the only way, but if you do the tradeoff of going with manual memory management, then you should accept what the tradeoff entails. Essential complexity is non-reducible -- you either manage it automatically through a GC or choose a model that while let you ignore it quite often, will make you think about it in the end.
Re:Rust Is Hard, Or: The Misery of Mainstream Programming by bsder on Thursday June 02, 2022 @10:58PM (#31603736)
> If you have a polymorphic async function, then return your result in a Box.

Agreed. Copy, Clone, Box, Arc, RwLock, etc. are your friends. Don't be afraid to use them.

You don't need as much performance as you think you do. Passing things around on the heap is fine for most applications. The Rust compiler is often smarter than you think.

And, when the Rust compiler is dumb or your code is getting stuck, you have code that you can understand and find the hotspot of. Now you can be clever.

Re:Rust Is Hard, Or: The Misery of Mainstream Programming by teaearlgraycold on Thursday June 02, 2022 @11:40PM (#31604003)
Step one is just clone() everywhere all the time. Then you should cut back AFTER analyzing where all of your memory is being lost to.
Re:Rust Is Hard, Or: The Misery of Mainstream Programming by chombier on Friday June 03, 2022 @04:21AM (#31605652)
Can one really escape clone hell afterwards, though?
Re:Rust Is Hard, Or: The Misery of Mainstream Programming by wongarsu on Friday June 03, 2022 @05:22AM (#31606030)
I generally go by the principle to borrow if I can without effort, otherwise I think a second of the code is in the critical path or the object is megabytes in size. If neither is the case, I clone without shame. The remaining 0.1% gets my attention (Arc, lifetime annotations, refactoring, or whatever is appropriate).

The important part is perspective. Nobody cares if you clone your command line arguments ten times during startup. As Dijkstra said, optimize the 2% of your program that matter, keep the rest simple

Re:Rust Is Hard, Or: The Misery of Mainstream Programming by kaba0 on Friday June 03, 2022 @06:41AM (#31606597)
Exactly. Even very experienced people will be easily misled to believe something is performance-sensitive -- computers are ridiculously fast. Like honestly, if it is not a tight loop chances are it seriously doesn't matter what you do. Remember, we write plenty of shell scripts that literally spawn a new process for almost every line, and yet they feel instantaneous.
Re:Rust Is Hard, Or: The Misery of Mainstream Programming by zepearl on Friday June 03, 2022 @11:19AM (#31609492)
This fully hits what I experienced.

I'm using Rust since ~2 months. At the beginning I was trying to write my usual C/C++ code and I got completely entangled with lifetimes relationships and the code ended up becoming a mess.

Nowadays as soon as the compiler starts mentioning lifetimes I see that as a warning => I then take a step back and change approach/design => no problems.

I use a little bit of async (I create dedicated variables that are moved to the async functions) and classic multithreading (I use "mpsc" to exchange data between the main & subthreads) and so far I never had a single segfault nor any kind of weird behaviour, which is incredible if compared to some other languages at least for my programming skills :)

Re:Rust Is Hard, Or: The Misery of Mainstream Programming by josephg on Thursday June 02, 2022 @09:50PM (#31603267)
> If you have a long-running async function, then pass parameters by value! If you have a polymorphic async function, then return your result in a Box.

I've taken to making heavy use of the smallvec and smartstring crates for this. Most lists and strings are small in practice. Using smallvec / smartstring lets you keep most clone() calls allocation-free. This in turn lets you use owned objects, which are easier to reason about - for you and the borrow checker. And you keep a lot of the performance of just passing around references.

I tried to use async rust a couple of years ago, and fell on my face in the process. Most of my rust at the moment is designed to compile to wasm - and then I'm leaning on nodejs for networking and IO. Writing async networked code is oh so much easier to reason about in javascript. When GAT, TAIT and some other language features to fix async land I'll muster up the courage to make another attempt. But rust's progress at fixing these problems feels painfully slow.

https://crates.io/crates/smallvec / https://crates.io/crates/smartstring

Re:Rust Is Hard, Or: The Misery of Mainstream Programming by rapsey on Thursday June 02, 2022 @10:42PM (#31603630)
However the constant checking if something is on the stack or heap is a big performance problem for smallvec. It results in terrible cpu branch prediction.
Re:Rust Is Hard, Or: The Misery of Mainstream Programming by kaba0 on Friday June 03, 2022 @06:44AM (#31606613)
I don't see how is it bad for branch prediction. Like, if you use the same smallvec object in a loop and do not insert any elements to it (so that it doesn't allocate on the heap) the branch will always turn in the same direction making it easy and cheap to predict. And I would think that in most use-cases your smallvec object will either remain in one state, not changing in-between.
Re:Rust Is Hard, Or: The Misery of Mainstream Programming by nmca on Friday June 03, 2022 @07:09AM (#31606776)
+1, I would love to see a trace of smallvec killing performance due to branch prediction.
Re:Rust Is Hard, Or: The Misery of Mainstream Programming by brabel on Friday June 03, 2022 @02:32AM (#31604958)
> So much of this pain is caused by premature optimization.

Rust is normally used only when high performance is of uttermost importance, so it will always attract people who want to optmise everything.

> Most of the time, the solution is to be less clever

I've done it myself and saw performance degrade, as was expected. That's fine, but by that point you might as well just use another language that has none of the hard problems and end up having very similar performance, which is what I did.

Re:Rust Is Hard, Or: The Misery of Mainstream Programming by Animats on Friday June 03, 2022 @03:47AM (#31605425)
> Rust is normally used only when high performance is of uttermost importance, so it will always attract people who want to optimise everything.

Which is not the way to do things. Profile, then optimize.

I'm writing a metaverse client that's heavily multithreaded and can keep a GPU, a dozen CPUs, and a network connection busy. Only some parts have to go fast. The critical parts are:

* The render loop, which is in its own higher-priority thread.

* Blocking the render loop with locks set during GPU content updating, which is supposed to be done in parallel with rendering.

* JPEG 2000 decoding, which eats up too much time and for which 10x faster decoders are available.

* Strategies for deciding which content to load first.

* Strategies for deciding what doesn't have to be drawn.

Those really matter. The rest is either minor, infrequent, or not on the critical path.

I use Tracy to let me watch and zoom in on where the time goes in each rendered frame. Unless Tracy says performance is a problem, it doesn't need to be optimized.

Re:Rust Is Hard, Or: The Misery of Mainstream Programming by SleepyMyroslav on Friday June 03, 2022 @06:47AM (#31606631)
Coming from gaming industry i think you might want to measure how far you can go with a single threaded rendering. There is limit of content and code that can be a brick wall later. Here is an example from SIGGRAPH 2021 where Activision presents how multithreaded rendering looks like: https://youtu.be/9ublsQNbv6I ps I don't work with Activision its just a public example that illustrates industry practice.
Re:Rust Is Hard, Or: The Misery of Mainstream Programming by Animats on Friday June 03, 2022 @01:42PM (#31611403)
I'm using Rend3/WGPU, where multithreaded rendering is coming, but isn't here yet. Work is underway.[1]

The Rust game dev ecosystem is far enough along for simple games, but not there yet when you need all the performance of which the hardware is capable.

[1] https://www.youtube.com/watch?v=DDG4bcGs7zM

Re:Rust Is Hard, Or: The Misery of Mainstream Programming by SleepyMyroslav on Friday June 03, 2022 @05:33PM (#31613939)
Cool video that matches best practices. Reducing memory footprint is always good and laying out things in memory is also good way to speed things up without changing amount of work.

I have trouble with concept of WGPU. GPUs are complex by themselves to bolt on top any abstraction that is coming from Web. But its just me, its not important since I am not a 3d programmer myself. I am more Engine / CPU optimization guy.

My interest in Rust and this topic is that i would like to see fine grained task parallel systems written in Rust. Instead of systems with separate thread for render that became a bottleneck years ago. I wish you good luck and hope to see a success story about Rust.

Re:Rust Is Hard, Or: The Misery of Mainstream Programming by Animats on Saturday June 04, 2022 @02:20AM (#31617561)
We'll see how it works out.

WGPU's API is basically Vulkan. It exists mostly to deal with Apple's Metal. Metal has roughly the same feature set as Vulkan, but Apple just had to Think Different and be incompatible. I'm not supporting the Wasm or Android targets. Android and browsers have a different threading model, and I don't want to deal with that at this stage. Linux/Windows/Mac is enough for now.

Thought for the near future - will VR and AR headgear have threads or something more like the processes with some shared shared memory model from Javascript land?

(That video isn't me, it's the Rend3 dev, who also works on WGPU.)

Re:Rust Is Hard, Or: The Misery of Mainstream Programming by SleepyMyroslav on Saturday June 04, 2022 @09:21AM (#31620017)
Gaming solved same problem by opting out of Apple rendering API support. ( ahaha ). Those who have to support mobile can't skip it ofc.

I dont work with VR/AR myself so I dont know.

Re:Rust Is Hard, Or: The Misery of Mainstream Programming by anonymoushn on Thursday June 02, 2022 @11:43PM (#31604021)
never allocating memory can be and should be simple.
Re:Rust Is Hard, Or: The Misery of Mainstream Programming by bpye on Friday June 03, 2022 @12:36AM (#31604344)
What language actually makes it simple?
Re:Rust Is Hard, Or: The Misery of Mainstream Programming by anonymoushn on Friday June 03, 2022 @02:07AM (#31604805)
In Zig, we are writing a moderately complex application that calls mmap many times in the first few seconds and never thereafter. We can use every part of the stdlib and most libraries because we can pass in allocators backed by the memory we reserved at startup.
Re:Rust Is Hard, Or: The Misery of Mainstream Programming by ncmncm on Friday June 03, 2022 @04:18AM (#31605636)
This is very commonly done in C++. My programs do a flurry of mmaps in the first second, then run for, sometimes, years without allocating again.
Re:Rust Is Hard, Or: The Misery of Mainstream Programming by bpye on Friday June 03, 2022 @01:14PM (#31611020)
I was thinking about C and C++ and whilst they allow you to be very careful about allocations, they also give you every opportunity to get it wrong.
Re:Rust Is Hard, Or: The Misery of Mainstream Programming by ncmncm on Friday June 03, 2022 @07:33PM (#31615132)
We need not act on the opportunity.

All that is actually needed is for it to be more work to get it wrong than to get it right.

Re:Rust Is Hard, Or: The Misery of Mainstream Programming by jedisct1 on Friday June 03, 2022 @02:17AM (#31604879)
Zig, definitely.