I can sort of understand where you're coming from — manual memory management can be difficult, and doing it improperly can cause bugs. However, in my experience, we're very far from having a magical solution for memory management. C++ definitely isn't it, and while Rust does bring significant advances in this field, it's a very large and complicated language. Unfortunately, the memory management strategy of every other language I've tried introduces performance penalties that make it unsuitable for e.g. video games. Trust me, I really wish this weren't the case! :(

Until we have some kind of significantly better solution that solves all memory management problems, I would rather work in a simple language that lets me carefully do everything myself, and if that language is also an improvement over C, I'm happy. However, that's just me, and I can fully appreciate that others are free to choose the tools that are good for them!

I hear you, but I think the problem is that you're framing this as "I, the developer, don't want to accept these costs". And that's fine when the software doesn't leave your system.

The problem is that you're them pushing other costs onto your users ie: exploitable software. So from the developer perspective, great, it works for you, but the cost is there.

I'm sympathetic to not wanting to use the other languages available, I'm not saying that any other language is doing things the "right" way, there's room for a lot of improvement. But I personally think that setting out to build new systems software in a memory unsafe language is setting users up for very serious harm.

I think I understand your view better now. Are you aware of any current memory management strategies (implemented as part of a language or otherwise) that perform well in situations with high performance requirements? For example, as someone who works on video games and real-time audio, most options seem non-starters to me aside from Rust, even if I decided to make sacrifices for the sake of security, and I at least have the impression I've explored this space quite a bit. Anyway, I would be happy to learn more about minimal memory safety strategies that don't require massive scaffolding and also allow for high-performance situations.

Not in mainstream languages. There's a lot of ongoing research in the space. Otherwise, Rust is probably the most mainstream language that achieves your goals.

Games are a bit different imo. While they're often networked they tend to not get attacked the same way as other software for a variety of reasons (though some games become so popular that it becomes worthwhile, like Minecraft). If a language set out to be "safer" (ie: improve temporal safety) but still prioritized performance, and emphasized its use case as being gaming, or explicitly for non-security-sensitive use cases, I'd be a lot more onboard with that. Jai seems to be driving towards that.

My issue with Hare is that it's presented (both on its page and in this HN thread) as being a language for general systems work.

Thank you for all of your feedback, I hope you end up at least trying Hare for the use cases that feel right to you! :)

I appreciate that. I really do hate to be critical of open source work. I just feel this is an important issue.

Not staticassertion, but I'm a hobbyist in real-time audio. I like Rust as a vocabulary for describing/teaching safe programming (&/&mut/Send/Sync). I find that multithreaded programs written in Rust are usually correct while multithreaded programs written in C++ are usually wrong, because Rust encodes the rules of shared-memory threading in its type system (&T: Sync objects are thread-shared, but are either immutable or atomic or requires locking to acquire a &mut T). I also appreciate guiding users towards exclusive references (&mut) to make it easier to reason about code. However I find it makes it too difficult to mutate through shared references or write unsafe code (passing Stacked Borrows while lending out &mut is more like solving puzzles than writing imperative code, and writing code that never touches & and &mut is a quagmire of addr_of_mut!() and unsafe blocks on every pointer dereference), and the Rust community appears uninterested in making unsafe programming more ergonomic.

Personally I'm a fan of C++'s unique_ptr/& as an unsafe escape hatch from Rust's single ownership or runtime refcounting overhead. It's at least as safe as Rust's unsafe pointers, and far more pleasant to use. Qt's QObject ownership system is reasonably ergonomic and QPointer is fun (though dangerous since it can turn into null unexpectedly), but Qt uses it pervasively (rather than only when safe memory management fails), relies on prose documentation to describe which pointers transfer ownership or not (resulting in memory management bugs), and QObject child destruction and nulling-out QPointers relies on runtime overhead. I haven't tried ECS or generational indexes yet, but those are popular in games, and Vale has its own ideas in this field (https://verdagon.dev/blog/generational-references).

On an aesthetic/principled level, I'd rather punt alias analysis to the programmer (pointer/restrict or &/&mut) rather than compiler complexity/magic (TBAA and provenance checking). Glancing at https://harelang.org/specification/, it seems Hare lacks an equivalent of restrict/&mut, and I wonder if that prevents the compiler from ever adding support for removing redundant loads/stores through pointers.

> On an aesthetic/principled level, I'd rather punt alias analysis to the programmer (pointer/restrict or &/&mut) rather than compiler complexity/magic (TBAA and provenance checking).

That would certainly be nice, but the state of the art on what problems even are is far ahead in optimizing compilers than anyone else - "having a restrict keyword" doesn't solve every aliasing problem afaik, and nobody respects the performance people when they tell you undefined behavior in C is actually useful. So nobody has come up with a simple solution for a better language that solves problems like pointer provenance and yet is "faster than C".

Actually most people's ideas of how to make programs faster are complicated things like autovectorization that don't work and would make it slower.

> However I find it makes it too difficult to mutate through shared references

It's not that difficult, you just need to use UnsafeCell<…> or one of its safe derivatives (each of which has some potential runtime overhead) to keep the semantics tractable.

One of the strange things about Rust is the &UnsafeCell<T>/*mut T dichotomy. &UnsafeCell<T> is easier to work with, and you can soundly acquire &mut T as long as they never overlap, but you can't turn a Box<UnsafeCell<T>> into a &UnsafeCell<T> and back to a Box<UnsafeCell<T>> to delete it, because provenance or something.

*mut T is harder to work with, this is UB according to miri since you didn't specify `&mut x as *mut i32 as *const i32`:

    let mut x = 1;
    let px = &mut x as *const i32;
    unsafe {
        *(px as *mut i32) = 2;
    }

Problem is, most APIs won't give you a &UnsafeCell<T> but rather a &mut T. Not sure if you can convert a &mut T to a &UnsafeCell<T> (you definitely can't using `as`). If you want to create multiple aliasing pointers into a non-UnsafeCell type or struct field, one approach (basically a placeholder since &raw isn't stable, https://gankra.github.io/blah/fix-rust-pointers/#offsets-and...) is:

    let mut x = 1;
    let px = addr_of_mut!(x);
    unsafe {
        *px = 2;
    }

Cell::from_mut converts from `&mut T` to `&Cell<T>` and that's a newtype around UnsafeCell - it has the same `as_ptr` method.

You cannot turn a &T into a Box<T>, because &T borrows T, while Box<T> owns T, and moreover it holds it in a separate allocation, so even &mut T cannot be transformed into Box<T> --- it already lives in some allocated space and whatever there is a reference to, cannot be moved to a new allocation. For moving T you need T, not a reference to T. The case with UnsafeCell<T> substituted in place of T is just a special case.

UnsafeCell<T> also owns T, so transforming &mut T into UnsafeCell<T> also doesn't make sense. The unsafe equivalent of references is pointers.

C++ lets you easily delete a T* or T const*, Rust has https://doc.rust-lang.org/std/primitive.pointer.html#2-consu... I guess?

> UnsafeCell<T> also owns T, so transforming &mut T into UnsafeCell<T> also doesn't make sense.

I wanted to transform a &mut T into &UnsafeCell<T> (note the &) and copy the reference, to allow shared mutation scoped within the lifetime of the source &mut T. How can this be accomplished?

> C++ lets you easily delete a T* or T const*, Rust has https://doc.rust-lang.org/std/primitive.pointer.html#2-consu... I guess?

Box deletes the owned object when it goes out of scope without being moved (like unique_ptr in C++). So if anything, you want to go the other way: https://doc.rust-lang.org/std/boxed/struct.Box.html#method.f...

However, you can delete a *T by using <https://doc.rust-lang.org/std/alloc/trait.Allocator.html#tym...> with the Global allocator (since this is the one you're most likely using).

> I wanted to transform a &mut T into &UnsafeCell<T> (note the &) and copy the reference, to allow shared mutation scoped within the lifetime of the source &mut T. How can this be accomplished?

If you want to have two instances of one &mut T, you don't go through &UnsafeCell<T>. Instead you may cast &mut T into *mut T and then use this: <https://doc.rust-lang.org/std/primitive.pointer.html#method....>. This however will cast into any lifetime, so if you want to bind the two lifetimes together, then you need to have the lifetime of the original &mut T explicitly specified, and then you assign the result of the method I linked to a variable with explicitly specified type where you specify the lifetime annotation. Alternatively, you may write a separate function which accepts both references as arguments and binds the two lifetimes together the usual way.

I admit it's a bit unergonomic. The best way currently would be to have the data stored as UnsafeCell in the first place and then call get_mut() on it to get all the references. However, if this reference comes from outside, you're left with the little mess I outlined above.

These are different things. UnsafeCell<T> is for shared mutable data. *mut T is for data that you assert will never be mutated while it's being shared with some other reference, but you can't rely on the compiler to prove this fact for you.

If I have a &mut T, what pointer type do I convert it into (and create multiple copies of), to allow shared mutation scoped within the lifetime of the source &mut T?

You can have multiple *mut T aliasing to the same data, but if you mutate through them you're on your own wrt. Rust's safety rules.

Thank you for the resources, I'll look into these details in more depth!

I’m sure it is not the answer you want to hear, but partial use of GCs seems to be exactly that. Modern GCs have insanely good throughput OR latency.

Quite a few languages have value types now, with that you can restrict your usage to stack allocations for the critical hot loops, while low-latency GCs promise less pauses than the OS itself, which should be plenty good for even the most demanding games.

Hey, I'm open to any answer that helps me write better programs. :) Which languages do you have experience working with in high-performance situations? I, for one, had high hopes for using Go for video game development, but it turns out that even in highly-tuned Go code with the latest GC optimisations, there are still significant GC pauses that cannot be overcome [0]. However, perhaps you're referring to other types of GCs I'm not aware of?

[0]: https://ebiten.org/blog/native_compiling_for_nintendo_switch...

I don’t have much experience with C#, but currently that seems to have the best balance of control over allocations and a performant GC due to having value types (and pointers as well if I’m not mistaken?)

But regarding GC, Java is unquestionably the king in that aspect, throughput-wise G1 is unbeatable and its relatively new ZGC might be of interest to use. It is the one I thought about previously, it currently promises sub-millisecond max pause times and this pause time doesn’t grow with heap size. Unfortunately Java doesn’t have value types yet, so you either write your hot loops with only primitives and allocations you make sure gets optimized by the escape-analyser, or do some manual memory management with the new Panama APIs, which are quite friendly in my opinion.

EDIT: Just read your link, while Java can be AOT-compiled with GraalVM, only the enterprise version supports some of the more exotic GC-variants (though not sure about ZGC). It should be free for personal use, but do have a look at it. Though what I wrote concern mostly running code with the JVM.

Yep, worth noting that there are a number of actual games that use MonoGame / FNA, including low-latency platformers like Celeste. I've actually found games written in these engines to be among the best performing games all around on old hardware.

Java's ZGC as of jdk 17 has very low pause times (e.g. a p99 of roughly 0.1 ms in this benchmark[0]). Their stated goal is to remain sub 1 ms, but in practice it stays well below that.

The JVM isn't the most common game dev platform, but I have been enjoying using LibGDX with Scala on jdk 17 with ZGC.

[0] https://kstefanj.github.io/2021/11/24/gc-progress-8-17.html

I use pony https://ponylang.io/ as a language - it's an Actor based language with GC where every actor has its own memory and is responsible for its own GC.

The main feature is its ability to safely share or move data around between actors in a way that is data-race and deadlock free. Pony doesn't use locks anyways :-)

A high level as to how it achieves this:

i. All variables have a "reference capability" - which is a description of what you can and cannot do with the variable alias (pointer) you have.

ii. The references to the data (think pointer) are passed in messages.

iii. As references are made and removed, actors send messages to the originating actor updating the count of references. When the count reaches zero, that data can be GC'd.

It's nice, because unlike some other language runtimes, it doesn't have to stop the world to work out what can and can't be GC'd. It's all done on-the-fly as it were.

Hi, this article says that I could overcome GC pauses. Thanks.

> while Rust does bring significant advances in this field, it's a very large and complicated language.

The gist of Rust on this fairly easy, is the heritage/chasing of C++ that makes Rust complicated.

A "Rust simple like pascal/c" have potential and I bet 1 billon the borrow checker will not make it hard to use (check https://vale.dev)

> A "Rust simple like pascal/c"

This has been tried, see Cyclone. It was a lot less practically usable than modern Rust.

> C++ definitely isn't it, and while Rust does bring significant advances in this field, it's a very large and complicated language.

It really isn't. Not compared to C++, at least. Or to managed language runtimes, which are just as "large and complicated", only beneath the hood.

Putting things "below the hood" with as few leaks as possible is one of the key ways of managing complexity. So if a language can do this for a certain set of use cases then it's worth using for those use cases. Everything becomes quicker and more productive. There's a reason people that few people nowadays write the server side of web applications in C++. Rust isn't a huge improvement for that use case compared to a managed language.

Rust is definitely as complicated as C++. However its complexity isn't as big of a deal because it's so much safer. If you forget one of the extremely complicated Rust rules you'll get a compile error. If you forget one of the extremely complicated C++ rules you hopefully will get a compile error. Maybe a warning with `-Wall` or maybe you'll silently get memory errors at runtime!

Rust is a complicated language, but I don’t it reaches C++ levels of complexity. One of the pernicious aspects of “mastering” C++ is understanding all of its leaky abstractions; there’s nothing like SIOF or SFINAE in Rust.

It definitely does. I think a lot of people think it doesn't because a) most people who know both have far more experience with C++ and are yet to experience its really complicated bits yet, and b) C++ has an actual specification so you can read about all its complexity.

Rust may not have SFINAE but C++ doesn't have for<'a>, Phantom data or Pin.

> Rust may not have SFINAE but C++ doesn't have for<'a>, Phantom data or Pin.

I'll grant you PhantomData, but I'd argue with the other two. C++ does have lifetimes and pinning semantics, they're just implicit and (largely) taken for granted. That is, until you cause memory unsafety with either.

IMO, the overall pattern between C++ and Rust is that "advanced" use requires many of the same skills between the two, but that (1) Rust is much better about avoiding "advanced" use, and (2) Rust forces the user to be much more explicit about what they actually mean (cf. lifetimes and pinning). These are arguably more complex than what C++ does, but only in the sense that C++ amortizes that complexity in blood.

> C++ does have lifetimes and pinning semantics, they're just implicit and (largely) taken for granted.

C++ does have pinning, but it is much rarer than in Rust because of user-defined move constructors.

It's certainly much more complicated than Go, and not just on the surface

But they are nowhere near the same niche. Go is much much closer to JS than to Rust by design, it just mimics being lower-level.

System level programming almost by definition requires quite a bit of complexity, and you can’t hide it no matter how elegant your abstraction is. Essential complexity is non-reduceable:

I would assert that writing compilers, linkers, hypervisors, unikernels, GPU debuggers, cloud infrastructure, is system level programming.

Well, that's because Go, as per design, is quite limited. Its type system is lacking things Rust has. The languages have (had?) different goals. You can see one past example in generics. How long it took to finally drag the Go developers to implement them, coming to recognize their usefulness, instead of sticking to the rather limiting "No we want the language to be very simple so that everyone can understand and use it." attitude. Rust has been designed with that safety aspect as one of its primary goals and that will incur some cost in being less simple.

Maybe be less zealous and aware of your assumptions?

Your assumption is that memory safety has to be baked in the language. It could be baked into proof assistants that are part of (optional or add-on) tooling, like what sel4 does. A simpler language makes this more possible, and the things that a proof assistants can do go far beyond what rust is able to provide, without sacrificing compilation speed or other forms of optimality (e.g. avoiding the heap) when you don't want or need such a high level of security guarantee (e.g. writing a cli tool that never sees the internet)

As for me, I'm terrified that rusts complex macro system will hide/obfuscate discovery of other forms of security regression, like timing or energy side channels.

My assumptions are based on study and experience. My "Zealotry" is just a desire to reduce harm to users in an area that I personally believe we should strive not to regress on.

I'm not interested in discussing Rust. Frankly, I'm sure there will be plenty of other people already doing so.

What's clear from this thread is that Hare does attempt to move the needle, relative to C, with regards to safety. My opinion is that that's not enough for the use cases they're targeting, but I suppose it's really up to whoever's writing the software to decide that.

> A simpler language makes this more possible

This is false. Rust’s borrow checker is nothing else but an included proof assistant for rust code. The reason it can catch so many memory issues and data races is specifically due to a more restricted language. Also, sel4 is a relatively tiny program which was written for an unusually long time by domain experts. Formal verification simply doesn’t scale to global properties, that’s why some restrictions are useful as they allow local reasonings instead.

For a more hands-on example look at the quality of auto-complete in case of Intellij’s Java vs a dynamically typed language. This night and day difference in quality is yet again possible due to what the language can’t denote.

Re rust macros: I don’t get your point, AFAIK they simply expand to regular old Rust code and then gets compiled, so the exact same safety guarantees apply.

> Formal verification simply doesn’t scale to global properties

This is an assertion you are making with absolutely no evidence, and also totally self-contradictory with your statement "Rust’s borrow checker is nothing else but an included proof assistant for rust code".

While we're at it, also Ada does this, which has long been used for large scale mission critical applications where formal assurances are necessary (with even more available optional safeties than rust provides).

I meant to write every global property due to Rice’s theorem.

And I don’t believe my claim is unsupported, the largest formally verified program is the mentioned sel4, which is still tiny compared to even the smallest of business apps and was written by domain experts over multiple years.

Restricting a problem to a subset is like the numero uno step to solve any hard problem - and this is what rust basically mandate. It won’t provide bug-free programs, but it can reliably prove the absence of memory bugs and data races due to the borrow checker, which can do its work on function-scope, since all the relevant information is encoded in the function’s generic lifetime arguments.

These claims aren’t contradictory when you understand the domain in question: formal verification of C abstract semantics doesn’t scale particularly well. Rust (and Ada) both have restricted memory semantics (particularly around aliasing) that effectively eliminate some of the hardest problems in whole-program static analysis.

Formal verification doesn't scale to global properties in the general case. Global properties that are simple and type-like (in that they match the syntactic structure of the program in a fully "compositional" way, like Rust lifetimes) can be checked with comparable ease. Complex properties can often be checked within a single, self-contained, small-enough program module. Trying to do both at the same time - check complex properties globally - is highly problematic. That's why the Rust borrow checker has to make simplifying assumptions, and use 'unsafe' as an escape hatch.

I'm not really convinced that this is true. I think you're brushing up against Rice's theorem, which is that proving arbitrary properties about arbitrary programs is equivalent to the halting problem. That's why we constrain languages with type systems, which limits any typed language from expressing arbitrary turing complete programs.

Proof assistance is sort of irrelevant. Types and proofs are related, as denoted by the curry howard correspondance.

The real issue with "throwawaymaths"' point is that they're saying "use proof assistants" to people who are using proof assistants. SEL4 is a terrible example of a success story, as it took ages to complete, and then there was immediately a bug found in a class they weren't looking for - because rice's theorem.

They're clearly advocating for the use of specific and explicit proof assistants, which is fine and a totally reasonable thing to advocate for, but in no way is related to rust or the discussion, which is why I chose not to engage.

> > discovery of other forms of security regression, like timing or energy side channels.

> so the exact same safety guarantees apply.

I see this a lot with Rust, and with encryption. There are no magic bullets. There is no "you are safe because you used this" tool.

In this case, Rust's safety gaurantees apply only to memory safety.

GP wasn't talking about memory safety above.

Sure, and I do share your concern regarding over ambitious claims on rust’s safety benefits, but I just don’t get why would a macro hide these any more than let’s say another function call would.

I don’t mean to say that a macro can’t get needlessly complex, but the same is true of functions that are inherent in basically any language. In the worst case macros can be expanded and looked at in their full forms. They are as always abstractions, which are pretty much necessary, but they can be abused as well.

> It could be baked into proof assistants that are part of (optional or add-on) tooling, like what sel4 does. A simpler language makes this more possible, and the things that a proof assistants can do go far beyond what rust is able to provide

I’m actually doing my PhD on the verification of Rust programs and wanted to add that the opposite is actually true. The type system of Rust helps to create a much simpler model of the language which allows us to do proofs in much larger scale than with C (for the same effort). This is specifically because of how ownership typing helps us simplify reasoning.

"Optional" means unused or misused until proved otherwise. No checking and no guarantees can be assumed, and if someone tries to deploy an add-on proof assistant I'd expect managers to see programmers who waste time pursuing some warnings instead of making progress on new features.

This is wholly unimaginative, there is a wide window of usage patterns that are not "unused or misused", for example, on by default but off with a project flag, or off when building a dev release but on when building a prod release, and also don't forget my point that many projects simply don't need the level of memory safety that rust provides. For example if you are single threaded and never free, or if you have an arena strategy.

Reducing concurrency and/or dynamic memory management makes a program easier to reason about for the compiler (and more likely to be correct in practice), not less in need of correct memory management.

I'm "wholly unimaginative" about what variables can be acceptably corrupted; I can only think of deliberately reading uninitialized memory as a randomness source, a case that is easier to prevent (by clearing allocated memory by default on the OS side) than to enable.

There are plans to research an optional borrow checker for Hare. Hare also does offer many "safety" advantages over C: checked slice and array access, exhaustive switch and match, nullable pointer types, less undefined behavior, no strict pointer aliasing, fewer aggressive optimizations, and so on. Hare code is much less likely to have these errors when compared to C code.

I would ultimately just come out and say that we have to agree to disagree. I think that there is still plenty of room for a language that trusts the programmer. If you feel differently, I encourage you to invest in "safer" languages like Rust - but the argument that we're morally in the wrong to prefer another approach is not really appreciated.

I think a section on safety might be worthwhile. For example, Zig pretty clearly states that it wants to focus on spatial memory safety, which it sounds like Hare is going for as well.

That's certainly an improvement and worth noting, although it obviously leaves temporal safety on the table.

> but the argument that we're morally in the wrong to prefer another approach is not really appreciated.

Well, sorry to hear it's not appreciated, but... I think developers should feel a lot more responsibility in this area. So many people have been harmed by these issues.

> I think developers should feel a lot more responsibility in this area.

I think most programmers would agree with that sentiment. Getting everyone to agree on what is "responsible" and what isn't however...

Hare is a manifestation of the belief that in order to develop responsibly, one has to keep their software, and their code, simple.

An example of what I mean by this: An important feature of Rust is the use of complex compiler features in order to facilitate development of multithreaded programs and ensure temporal safety. In Hare programmers are encouraged to keep their software single threaded, because despite features like Rust's, concurrent programs turn out much more complex to write and maintain than sequential ones.

Keeping software single-threaded also eliminates many ways in which a program could fail due to lack of compiler enforced temporal safety.

Single threaded development seems a noteworthy goal, and I partially agree that it often leads to much simpler code and works well in systems like Erlang. But it is also a questionable focus in the days of barely increasing single core performance, especially in a systems language.

I believe one of the reasons Rust got so popular is that it made concurrency much easier and safer right at a time where the need for it increased significantly.

If that is the recommendation, maybe the standard library could focus on easily spawning and coordinating multiple processes instead, with very easy to use process communication.

Unfortunately you can't make things faster by making them concurrent, at least not in the way computers are currently designed. (And they're probably designed near-optimally unless we get memristors.) In my experience it's the opposite; you can make concurrent programs faster by removing it, because it adds overhead and they tend to wait on nothing or have unnecessary thread hops. And it makes them more power efficient too, which is often more important than being "faster".

Instead you want to respect the CPU under you and the way its caching and OoO instruction decoding work.

It sounds like Hare has a philosophy around safety here and it's just not documented - at least not where I could find it, scrolling around a bit.

I did find this page, though: https://harelang.org/blog/2021-02-09-hare-advances-on-c/

Which I found very interesting.

That’s just not realistic. We live in a complex world, and we need complex software. That’s why I vehemently hate these stupid lists: http://harmful.cat-v.org/software/

Don’t get me wrong, it is absolutely not directed at you, and I absolutely agree that we should strive for the simplest solution that covers the given problem, but don’t forget that essential complexity can’t be reduced. The only “weapon” we have against it is good abstractions. Sure, some very safety critical part can and perhaps should be written in a single-threaded way, but it would be wrong to not use the user’s hardware to the best of its capability in most cases, imo.

In a world where 99% of software is still written to run on 4-16 core machines and does tasks that any machine from the last 10 years can easily run on a single thread if it was just designed more simple instead of wasting tons of resources...

I'd wager that most of the applications that need to be "complex" in fact only haven't sorted out how to process their payloads in an organized way. If most of your code has to think about concurrent memory accesses, something is likely wrong. (There may be exceptions, like traditional OS kernels).

As hardware gets more multithreaded beyond those 16 core machines, you'll have to be more careful than ever to avoid being "complex": when you appreciate what's happening at the hardware level, you'll start seeing that concurrent memory access (across cores or NUMA zones) is something to be avoided except at very central locations.

> essential complexity can’t be reduced

I suggest looking at accidental complexity first. We should make it explicit instead of using languages and tools that increasingly hide it. While languages have evolved ever more complicated type systems to be (supposedly) safer, the perceived complexity in code written in these languages isn't necessarily connected to hardware-level complexity. Many language constructs (e.g. RAII, async ...) strongly favour less organized, worse code just because they make it quicker to write. Possibly that includes checkers (like Rust's?) because even though they can be used as a measure of "real" complexity, they can also be used to guardrail a safe path to the worst possible complex solution.

> I suggest looking at accidental complexity first. We should make it explicit instead of using languages and tools that increasingly hide it.

The languages that hide accidental complexity to the greatest extent are very high level, dynamic, "managed" languages, often with a scripting-like, REPL-focused workflow. Rust is not really one of those. It's perfectly possible to write Rust code that's just as simple as old-school FORTRAN or C, if the problem being solved is conducive to that approach.

I don't see it as an improvement, because they are taking what Modula-2 already offered in 1978 in terms of systems programing safety.

I guess not having to type keywords in caps (because who uses IDE/smart editors) and having a C like syntax is the selling point.

I don't really want to engage with the RESF. We have the level of safety that we feel is appropriate. Believe me, we do feel responsible for quality, working code: but we take responsibility for it personally, as programmers, and culturally, as a community, and let the language help us: not mandate us.

Give us some time to see how Hare actually performs in the wild before making your judgements, okay?

I'm a security professional, and I'm speaking as a security professional, not as an evangelist for any language's approach.

> Give us some time to see how Hare actually performs in the wild before making your judgements, okay?

I'm certainly very curious to see how the approach plays out, but only intellectually so. As a security professional I already strongly suspect that improvements in spatial safety won't be sufficient to change the types of threats a user faces. I could justify this point, but I'd rather hand wave from an authority position since I suspect there's no desire for that.

But we obviously disagree and I'm not expecting to change your mind. I just wanted to comment publicly that I hope we developers will form a culture where we think about the safety of users first and foremost and, as a community, prioritize that over our own preferences with regards to our programming experience.

I am not a security maximalist: I will not pursue it at the expense of everything else. There is a trend among security professionals, as it were, to place anything on the chopping block in the name of security. I find this is often counter-productive, since the #1 way to improve security is to reduce complexity, which many approaches (e.g. Rust) fail at. Security is one factor which Hare balances with the rest, and I refuse to accept a doom-and-gloom the-cancer-which-is-killing-software perspective on this approach.

You can paint me as an overdramatic security person all you like, but it's really quite the opposite. I'd just like developers to think more about reducing harm to users.

> to place anything on the chopping block in the name of security.

Straw man argument. I absolutely am not a "security maximalist", nor am I unwilling to make tradeoffs - any competent security professional makes them all the time.

> the #1 way to improve security is to reduce complexity

Not really, no. Even if "complexity" were a defined term I don't think you'd be able to support this. Python's pickle makes things really simple - you just dump an object out, and you can load it up again later. Would you call that secure? It's a rhetorical question, to be clear, I'm not interested in debate on this.

> I refuse to accept a doom-and-gloom the-cancer-which-is-killing-software perspective on this approach

OK. I commented publicly that I believe developers should care more about harm to users. You can do with that what you like.

Let's end it here? I don't think we're going to agree on much.

> There is a trend among security professionals, as it were, to place anything on the chopping block in the name of security.

I really have to disagree on this, in spite of not being a security professional, because the history has proven that even a single byte of unexpected write---either via buffer overflow or dangling pointer---can be disastrous. Honestly I'm not very interested in other aspects of memory safety, it would be even okay that such unexpected write reliably crashes the process or equivalent. But that single aspect of memory safety is very much crucial and disavowing it is not a good response.

> [...] the #1 way to improve security is to reduce complexity, [...]

I should also note that many seemingly simple approaches are complex in other ways. Reducing apparent complexity may or may not reduce latent complexity.

History has also proven that every little oversight in a Wordpress module can lead to an exploit. Or in a Java Logger. Or in a shell script.

And while maybe a Wordpress bug could "only" lead to a user password database leaked but not the complete system compromised, there is a valid question which is actually worse from case to case.

Point is just that from a different angle, things are maybe not so clear.

Software written in memory unsafe languages is among the most used on the planet, and could in many cases not realistically replaced by safer languages today. It could also be the case that while bug-per-line might be higher with unsafe languages, the bang-for-buck (useful functionality per line) is often higher as well (I seriously think it could be true).

Two out of your three examples are independent to programming languages. Wordpress vulnerability is dominated by XSS and SQL injection both of which are natural issues arising from the boundary of multiple systems. Java logger vulnerability is mostly about the unjustified flexibility. These bugs can occur in virtually any other language. Solutions to them generally increases the complexity and Hare doesn't seem to significantly improve on them over C probably for that cause.

By comparison memory safety bugs and shell script bugs mostly occur in specific classes of languages. It is therefore natural to ask for new languages in these classes to pay more attention to eliminate those sort of bugs. And it is, while not satisfactory, okay to answer in negative while acknowledging those concerns---Hare is not my language after all. Drew didn't, and I took a great care to say just "not a good response" instead of something stronger for the reason.

> #1 way to improve security is to reduce complexity

If managing memory lifetime is an inherently complex problem (which it is), the complexity has to live somewhere.

That somewhere is either in the facilities the language provides, or in user code and manual validation.

Imho as everywhere in this field there are tradeoffs to choose for improving this problem: Complexity (rust, formal proofs), runtime overhead (GC), etc.

Hare tries to be simple, so that it's easier to reason about the code and hence maybe find/avoid such bugs more easily.

For the last 50 years, from personal computer devices to nuclear power plants, the world is sitting on a pile of C code.

Why are you afraid of manual memory management?

The world is just doing fine with it, because, believe me or not, the world's worst problems like violence, war, poverty, famine, just to name a few, are not caused by C bugs.

How would I trust the programers who implement automatic memory management that they are not going to make any mistake?

"How would I trust the programmers who implement compilers that they are not goin to make any mistakes? I'll write the assembly myself, thank you very much."

It's much better to push logic from being manually re-written over and over by tens of thousands of different programmers, to being built into a language/tool/library by a team of a few hundred experts and then robustly tested.

> "How would I trust the programmers who implement compilers that they are not goin to make any mistakes? I'll write the assembly myself, thank you very much."

Nope, doesn't work. Then you have to trust that the programmer who wrote the assembler didn't make any mistakes. Real programmers program in octal.

> Then you have to trust that the programmer who wrote the assembler didn't make any mistakes.

This is true, but the actual solution is safer assembly. ARM is heading in this direction with PAC/BTI/CHERI. Intel, being Intel, tried adding some and it was so broken they removed it again.

It could go much further.

While any modern operating system is the living counter point, so far it's manageable.

They aren't a counterpoint at all. They're confirmation. Security-wise legacy operating systems (Linux, NT, ...) suck. New security vulnerabilities are discovered every week and month in them to the point that nobody actually considers these "multi-user systems" any more and obviously every box hooked up to the internet better be getting patches really frequently.

For the record I said manageable not great or perfect.

Every (popular) modern operating system sits on decades old foundations written in C that can't just be replaced, so that's not a particularly strong argument.

It's noteworthy that Google is financing the effort to bring Rust to the Linux kernel, that Microsoft is also investing in the language and that there are newer, production usage focused operating systems written in Rust. (eg Hubris [1])

[1] https://github.com/oxidecomputer/hubris

Redox is probably a better example than Hubris:

https://www.redox-os.org/

Hubris is intended to run on microcontrollers in a very low-level context (e.g. no display), so is very unlikely become a desktop / user-facing OS.

(I work at Oxide, mostly writing Hubris code)

I agree. It is quite clear that it is impossible to write large code bases safely with manual memory management. Even very small programs often have massive problems. I think many programmers are simply in denial about this.

I see Rust as a counterexample, serving as a formalization of provably safe patterns of manual memory management. I do wish it made it easier to write human-checked code the compiler cannot verify; unsafe code is so painful with unnecessary pitfalls, that many people write either wrong-but-ergonomic unsafe code (https://github.com/mcoblenz/Bronze/, https://github.com/emu-rs/snes-apu) or add runtime overhead (Rc and RefCell).

While Drew is the designer of Hare, a lot of us worked on Hare, and we tried really hard to create something useful and valuable. I think it would be a shame if you disregard it because of something that Drew said at whatever point in time. If you have the time, please try Hare and let us know what you think!

Aside from that, the question of memory safety is more complex than you make it out to be, and Drew, myself and others have discussed it in a lot of detail in this thread, Drew mentioned there are many memory safety features, potential plans for an optional borrow checker and so on — please draw your conclusions based on the full information.

I am sympathetic, but I agree with others that any new systems language in 2022 must have memory and thread safety with minimal escape hatches as its utmost priority and a core component of the language design.

Otherwise, what's the point? Yet another language that is a bit more convenient than the alternatives but doesn't do much to help with all the vulnerabilities and bugs in our software? We already have quite a few languages like that: Zig, Nim and D to name a few. (for both Nim and D GC is optional)

Rust is by no means the ultima ratio. It's complex and places a lot of immediate burden on the developer. I'm sure there are better solutions out there that don't sacrifice the safety guarantess. Particularly because Rust not only has the borrow checker but also a general focus on very type-system heavy designs.

But it has proven to be a significant step up from the likes of C and C++, and the additional burden pays off immensely in maintainability and correctness. I can count the memory/thread unsafety issues I encountered in 5 years of Rust on one hand, and in each case the culprit was a C library. (either because of a bad wrapper, or inherent incorrectness in the library)

Memory and thread safety can't be retrofitted into a language without producing a mess, as can be seen by the discussions and efforts in D, C++ and to some extent Swift.

They need to be central to the core language and standard library.

> Rust is by no means the ultima ratio. It's complex and places a lot of immediate burden on the developer. I'm sure there are better solutions out there that don't sacrifice the safety guarantess.

There are undoubtedly better solutions than Rust, but they tend to allow for more developer-managed complexity rather than less! For example, one might imagine a cleaned-up variety of the GhostCell pattern, to decouple ownership- and borrow-checking from the handling of references. The principled extreme would be a language that just implements full-blown separation logic for you and lets you build up the most common patterns (including "ownership" itself) starting from that remarkably "simple" base.

I don't think that's really the case. I can't think of any new safe GC-free languages that are more "manual" than Rust, but there are at least a couple that use reference counting with compile time optimisations to make it not really slow. Koka looks the most interesting to me at the moment.

Making the Rust ownership model central to the core language and standard library has meant that after 11 years of Rust, your program still can't have two objects of the same type owned by different allocators. As a result, I am interested in other approaches to these problems.

Local allocators are being standardized. This has nothing to do with the Rust ownership model, C++ took a long time to standardize an allocator abstraction too.

> Otherwise, what's the point?

Hare seems to solve spatial memory safety and null safety, which I think is a big deal.

Better than C is certainly a step up, but in my opinion also solving temporal safety must be a goal for a new language in this space.

This is a very unsympathetic take. If you expand to the full context, you'll note that I weigh memory safety against other trade-offs, and come away with a different answer than Rust does.

Hare does have safety features. Checked array and slice access, mandatory error handling, switch/match exhaustivity, nullable pointer types, mandatory initializers, no undefined behavior, and so on. It is much safer than C. It just doesn't have a borrow checker.

> This summarizes everything one needs to know

Your comment comes off as very dishonest. How does this quote summarize "everything I need to know". The quote you show appears to be out of context, I followed your link and that it appears to be a conclusion following a number of points explaining why, for the author, Rust's safety feature is apparently not enough to counterbalance it's flaws.

I don't fully agree with the points there (I enjoy Rust), but what does criticizing Rust have to do with "not understanding memory safety"?

Can you elaborate on why somebody has to appreciate Rust? Because I don't see it. Rust is not a religion and we shouldn't treat it as such.

I don't think it's fair to boil the language down to one person, or a person's views down to one statement.

Is appreciating Rust some kind of high ideal in the world of programming?

Yes, Rust is quite posh nowadays.

Just like Scala and F# a decade ago.

Or like ruby 15 years ago, or like Java in the 90s.

How is that statement an indictment against him? That’s his opinion. Does it make you a bad person to eschew Rust?

> Programmers can not be trusted with manual memory management. We have decades of proof, billions and billions of dollars of bug fixes and mitigation investments, real world damages, etc.

We have sanitizers if you are a bad programmer, use that if you don't trust yourself

It’s just ridiculous to call someone a bad programmer over memory bugs.

By your own words you are definitely a bad programmer if you have ever written more than 1000 lines of code in low level programming language, because there is simply no way you haven’t made an error. You just don’t necessarily know about it, which in my opinion makes you a worse developer, especially with this ancient and destructive mindset.

There is nothing wrong with being a bad developer, we are all bad developers if we don't understand what we are doing

We have the tools to eliminate memory bugs already

Forcing people to use a language with a buitin sanitizer/babysitter that runs everytime you compile your code and as a result makes you wait 10+ minutes between each line of change is dumb

Expecting your code to be bug free because the code written by someone else told you so is also dumb

https://github.com/rust-lang/rust/issues/38899

who to trust in that case?

--

educate and trust developers, use tools to audit your code when needed

Sanitizers can only show you some memory problems that happen during a given runtime with given data.

And your 10+ minutes baseless assumption is just demonstratively false. In the default debug mode it is literally instant with human perception on my current, not small project. Longer compiles only happen in release mode or when you introduce new dependencies.

Shall I link you to Valgrind bugs as well?

> And your 10+ minutes baseless assumption is just demonstratively false

Ok, not 10+ minutes, more like 6 minutes on a beefy machine:

https://www.youtube.com/watch?v=nR2WDBMjkh8&t=976s

> Shall I link you to Valgrind bugs as well?

See, you fail to understand the point

As was already stated, sanitizers catch a subset of issues and rely on test coverage. Sanitizers have been in use by projects for years, with lots of money spent on fuzzing and test coverage, and there are still numerous issues.