Does using Zig over C++ lead to "less memory, and represents a modest performance"? Or was the C++ implementation a bit sloppy? (lacking data oriented design for instance)

Also, what specifically are you most excited using Zig for?

The new Zig implementation is certainly more well designed than the C++ implementation, for several reasons:

* It's the second implementation of the language

* It did not have to survive as much evolution and language churn

* I leveled up as a programmer over the last 7 years

* The Zig language safety and debugging features make it possible to do things I would never dream of attempting in C++ for fear of footguns. A lot of the data-oriented design stuff, for example, makes use of untagged unions, which are nightmarish to debug in C++ but trivial in Zig. In C++, accessing the wrong union field means you later end up trying to figure out where your memory became corrupted; in Zig it immediately crashes with a stack trace. This is just one example.

* Zig makes certain data structures comfortable and ergonomic such as MultiArrayList. In C++ it's too hard, you end up making performance compromises to keep your sanity.

Generally, I would say that C++ and Zig are in the same performance ballbark, but my (obviously biased) position is that the Zig language guides you away from bad habits where as C++ encourages them (such as smart pointers and reference counting).

As for less memory, I think this is simply a clear win for Zig. No other modern languages compete with the tiny memory footprints of Zig software.

Some of the projects I am exited to use Zig for:

* rewriting Groove Basin (a music player server) in zig and adding more features

* a local multiplayer arcade game that runs bare metal on a raspberry pi

* a Digital Audio Workstation

Woah.

A little bit ago I went to write a little tool that mucked with the FL Studio FLP format. It was easy enough to guess out the bits that I cared about, so I pretty much just did that using a couple quick projects with specific things in them. However, I did check to see if anyone else had mucked around with the FLP format, and couldn’t help but notice your name. Was pretty surprised as a very curious onlooker to the Zig programming language. You certainly seem to get around :)

That’s a little tangential, but I guess I mention it because I was actually wondering if this was ever something you planned on doing given the fact that it was clear you had dabbled with DAW stuff (forgive me for not knowing if you have a more rich connection to music production than just that; I never bothered to check.)

A DAW in Zig sounds like a kick-ass idea. I tried to write a sort-of DAW toy with friends in Rust and it was a lot of fun even if it mostly made me realize how challenging it could be. (And also how bad at math I am. It took me so much effort to feel like I could understand FFTs enough to actually implement them.) It makes me wonder if your Zig DAW would be open source? It would be a fun project to attempt to contribute to, if or when it ever came to fruition.

Exciting stuff. Congrats on the Zig milestone.

Do you understand how little DSP is involved in writing a DAW? It has almost nothing to do with DSP and everything to do with application architecture, data management, threading and more.

That said, Rust seemed pretty promising for writing the audio engine of a DAW due to the memory ownership model. It was relatively easy to come up with a way to architect a very basic lock-free audio thread and feel sure that it was at least memory correct. I have no idea how a real DAW avoids certain pitfalls in the audio thread; lock-free isn’t too hard, but avoiding allocations in all circumstances seems tricky.

Take a listen to my interview with Justin Frankel of Reaper - somehow in our 3hr+ epic chat, he acknowledges that they don't avoid locks entirely. And Ardour, despite trying, also fails to do so. Anecdotally, from my conversations with other DAW developers, they also do no manage it 100%. As Justin put it, it's more about contention avoidance than lock avoidance.

Avoiding actual stack-based allocation is pretty easy in C++. You just have to want to do it, and remember to do it.

Memory correctness is really easy in the RT threads of a DAW precisely because they do (should) not allocate. You're dealing with pre-allocated memory blocks that do not change over time. It's almost the simplest case of memory mgmt that I know of within a DAW. I have seen several new-to-DAW developers adopting ill-advised schemes for memory mgmt in an RT context. The "list of blocks" is one pattern that I consider something to avoid (and unnecessady). Single-reader/single-writer lock-free (circular) FIFOs get you almost all the way there, for everything.

The main reason I felt Rust was nice was simply the re-assurance that I could not, even if I wanted to, accidentally cause a data race within the confines of safe Rust. It’s not that the actual code was hard, but it did force me to ensure that what I was doing with the data model was actually safe. It wound up guiding how data flow worked in the playback engine.

Anyway, thanks for the pointer to the interview. I don’t expect myself to be writing the next Reaper or Ardour so it is probably OK if I don’t quite grok what you’re getting at. But, I do find this stuff rather interesting, so I would love to have a listen when I get a chance.

If I understood it correctly, you think smart pointers and reference counting are bad habits. Why? Especially the smart pointers bit.

Why does Zig use less memory than other languages? Is it inherent to Zig, or can it be reproduced in other languages?

These are things that a "systems language" programmer should put in the work to do correctly/near-optimally, and not ask the compiler to just do something "good enough", like Python would.

I don't really feel this is the case in Rust.

You can easily add a manual “drop” call in Rust at any point if you want to force an allocation to be freed sooner, but I speak from years of experience using Rust for work when I say that Rust’s RAII model is not problematic in practice. I’m not simply speculating or theorizing, and I have professional experience with a variety of languages at all levels of the stack. I personally don’t mind garbage collectors most of the time, but Rust is great when you need more control.

In C++, RAII can absolutely be problematic because you are able to easily do things that cause undefined behavior by accident, which is arguably worse than either leaking by default or the mere act of holding onto memory for the duration of a scope.

If you can propose a system which cannot be contrived to have any downside, that would be fantastic! In the real world, Rust’s approach to memory management is extremely pragmatic and beneficial. I’m sure someone will eventually improve on Rust’s approach, but “leak by default” isn’t it.

I honestly do enjoy following Zig… it is a fascinating language taking a really interesting approach to solving many problems, but its memory safety story is not where I want it to be yet. Leaking memory by default is technically safe, but it's not exactly endearing.

I explained in my previous comment that you can explicitly "drop" any value at any time in Rust, if you choose.[0] But if you don't, it will still be dropped at the end of the scope. The developer has control, but the language will watch your back.

[0]: https://doc.rust-lang.org/std/mem/fn.drop.html

I don't even think most mainstream uses of PHP have done the process-per-request model for decades, but I could be wrong.

Here is a series of articles that describes it in detail

https://www.php.net/manual/en/features.gc.php

Especially look at this example where the same script is executed with and without GC

https://www.php.net/manual/en/features.gc.performance-consid...

Don't get me wrong. I do think that rust and c++ RAII is much more convenient and safe than the C or Zig way.

(I'd even prefer if you could annotate given struct in rust so the compiler could drop them as soon as it's no longer used, but that s not that simple)

It would probably be a breaking change to automatically call an explicit Drop implementation anywhere other than the end of the current scope, so I think that would have to be left as-is. String doesn't implement Drop, so it could easily be dropped eagerly within the scope as soon as it won't be referenced again. Such a change would be roughly equivalent to any of the compiler optimizations that reorder statements in ways that should be unobservable.

    fn foo(xx : &Mutex<String>) {
        let lock = xx.lock();
        let ptr = &mut *lock as *mut String;
        unsafe { use_from_c(ptr) }
    }
  

Outside of uses of unsafe, are there any other serious problems with eagerly dropping values that manually implement Drop? Maybe this fallback mechanism would also have to be invoked simply for casting to a raw pointer anywhere in the function. Either way, a list of exceptions to eager drop would arguably be better than not having eager drop, as long as the list was sound.

It would still be a breaking change, and would definitely require at least being restricted to a new edition. Some people currently use Drop as a kind of scoped “defer”, especially for things like instrumentation, so maybe it would be time for the language to introduce a proper “defer” statement that exists for that purpose instead of making Drop so lazy for everyone.

I’m assuming by string you mean a stack allocated array of char and not a std::string.

For locks it is seriously the best way to manage them.

So how do you solve SIMD with C, without language extensions?

C is not a special snowflake.

Writing generic SIMD code is more portable. C has libraries, and Zig also has vector primitives available through built-in functions.

In fact, a few GC enabled languages have explicit SIMD support.

https://ispc.github.io/

https://github.com/halide/Halide

For certain problems you would want a TLA+ specification for safety and especially liveness either way. It's not like Rust absolutely guarantees correctness in all cases.

Rust sits in a sweet spot between C/Zig and languages like Java, but it's not an appropriate replacement for either of them.

I believe that Rust can be as low level as you want. If you have to fight the language to accomplish something, it's because you're probably trying to do something unsafe, and Rust will let you do that if you pay the price.

I was looking for examples of real cases where Zig is a better option than Rust. And while Zig is a lovely language, its main points against Rust are that it lets you do unsafe things more easily, and it's easier to learn.

There is a scalable solution for UAF that doesn't involve introducing a lifetime/region system: garbage collection. Of course, that comes with its own set of tradeoffs.

There are many cool possibilities that C++ has never explored and frankly I find your argument unimaginative.

From the man page (which I believe quotes ISO C): "The free() function shall cause the space pointed to by ptr to be deallocated; that is, made available for further allocation." If free doesn't do that because another allocation is pointing into that page, that's a violation of the contract. Standard quarantine doesn't violate the spec because the space will eventually become available no matter what, but perpetual quarantine does.

One way to fix the problem would be to round up all allocations to 4kB, but that's very wasteful and slow due to cache and page table traffic; you'd be better off from a performance point of view with a garbage collector. More promising is ARM MTE, though the tag is currently too small to really be called a solution to UAF as opposed to a mitigation. A 64-bit tag would be enough, but I'm not sure what the performance costs of that would be--I wouldn't be surprised if the increased memory traffic makes it slower than a GC.

Quoting and bolding. I think this is the thing that I need to change most about my own style of programming.

pub enum List { Empty, Elem(i32, Box<List>), }

instead of :

pub struct List { head: Link, }

enum Link { Empty, Some(Box<Node>), }

struct Node { elem: i32, next: Link, }

I thought Aria's "Entirely Too Many Lists" tutorial actually tries to build this, but it actually doesn't, she draws you the resulting "list" and then is like, OK, that's clearly not what we want, let's build an actual (bad, slow, wasteful) linked list as our first attempt.

The difference really lies in the fact that i now have some data stored on the stack (The element) and some data stored on the heap because it's recursive. Was just a random example of where it's poor practice. As others have noted, linkedlists are a terrible data structure to begin with.

I enjoy synthesizers (including Eurorack) and looking forward playing with a Zig DAW!

But in terms of the implementation, this means working with untagged unions, because the tags are maintained externally.

ORLY?

Maybe you can put together a better, faster team (like Presonus managed to do for Studio One), but most current DAWs have been in existence for 20 years or more. Catching up with that is a challenge, and if you don't catch up then it's an interesting toy or half-a-DAW.

So why?

ps. obviously I am biased.

> As for less memory, I think this is simply a clear win for Zig. No other modern languages compete with the tiny memory footprints of Zig software.

Is not true at all. There are several other modern languages that compete with Zig for small memory footprints.

However, I could believe Zig's stdlib and culture encourages low memory footprints, but there isn't anything novel in the language that makes it inherently lower memory footprint. Though to name a few languages I'd say that can be directly comparable are Rust, D (esp. BetterC mode), and Nim. Even Julia & Go in the right context. Though honestly I often prefer wasting a few hundred bytes of RAM here and there, even on a microcontroller, for pure convenience.

edit: forgot Odin. Another comment mentioned it, though I've never used it. It looks like it's used in production.

* MultiArrayList

* arbitrary size-integers that make it easy to pack data very tightly

* easy access to several special purpose allocators which reduce the need for strategies like reference counting

* MultiArrayList is convenient and easy to do in Rust, D, Nim. Odin seems built around it. * Arbitrary bit-size integers seem more gimmicky than anything and similar to struct bitfields in other languages (except Rust oddly). Most compilers don't even pack uint8/uint16's for performance reasons. * Special purpose allocators are a bit more interesting. Still they don't provide actual memory safety AFAICT and can have fragmentation and performance penalties.

It'll be interesting to see larger Zig code bases emerge in different fields and see how the memory footprint compares in practice.

http://www.catb.org/esr/structure-packing/

https://github.com/Twon/Alignment/blob/master/docs/alignment...

Also ARM for example doesn't have 8/16 bit registers so int8 or int16 will use a 32bit register:

https://stackoverflow.com/a/23716920

Curiosity got to me, perhaps Zig had improved significantly. So I compared the first benchmark I found (kostya/benchmarks/bf) with Zig with Nim. For the smaller input (bench.b) Zig did run with ~22% less RAM (about 20kB less).

However, for the larger input (mandel.b) Nim+ARC used ~33% less RAM in safe mode: Nim 2.163mb -d:release; Zig 2.884mb -O ReleaseSafe; Zig 2.687mb -O ReleaseFast. The Nim requires 0.5mb less ram and the code is ~40% shorter. I don't have time to try out the Rust or Go versions though.

edit: grammar

Note I ran the benchmarks locally (MacBook Air M1) because the reported benchmark uses the older (default) Nim GC while I only use Nim+ARC. I also had to fix the Zig code and it took a few tries to get the signed/unsigned int conversions working. I tried tweaking flags for both a bit as well to see how stable they were. Zig's memory usage was pretty constant. Nim had a small speed vs memory tradeoff that could be tweak-able, but the defaults used the least memory.

Overall I'd expect exact memory usage by language(s) to vary some by benchmark and one random benchmark isn't conclusive. Still I didn't find anything to indicate Zig is clearly better than other new generation languages. Manual memory management might actually be worse than letting a compiler manage it in some cases.

1: https://github.com/kostya/benchmarks/tree/master/brainfuck

That's particularly interesting considering the rust compiler in rust has never been as fast as the original OCaml one

I was there and had to suffer through this more than virtually anyone else :)

> 7 femto-bunnies - rustboot building Rust prior to being retired

> 49 kilo-hamsters - rustc building Rust immediately after rustboot's retirement

> 188 giga-sloths - rustc building Rust in 2020

The reduced memory has significant value. Being able to do the same build on less expensive hardware or do more with the same hardware is a significant financial performance improvement

https://github.com/WebAssembly/wabt/tree/main/wasm2c

The simple example in the README.md seems straightforward enough, but I wonder if there are any pathological explosions in practice.

For a pathological explosion, you mean something like a Zip Bomb? Wasm and C are pretty close together in their semantics, Wasm hides the stack and prevents jumping into the middle of a function (CFI, Control Flow Integrity). I think the code bloat should be on the order of some multiple of the smallest interpreter.

I just did a quick scan of Wasm interpreters (3 in Rust, 1 in C)

    yblein/rust-wasm ~4kloc
    rhysd/wain ~16kloc
    paritytech/wasmi ~25kloc
    wasm3/wasm3 ~22kloc (C)

    4.1M wasidoom.o
    1.8M wasidoom.wasm

* Is that a list of compilation targets?

* If not all behavior tests pass, does that not mean that the compiler fails to compile programs correctly?

Please indulge those of us who are not familiar with self-hosting compiler engineering.

Snippets of zig code that use language features and then make sure those features did the right thing. You can find them here: https://github.com/ziglang/zig/tree/master/test/behavior

> Is that a list of compilation targets?

Mostly. Pedantically, it's a list of code generation backends, each of which may have multiple compilation targets. So for example the LLVM backend can target many architectures. The ones that are architecture specific are currently debug-only and cannot do optimization.

> If not all behavior tests pass, does that not mean that the compiler fails to compile programs correctly?

Some tests are not passing because they cause an incorrect compile error, others compile but have incorrect behavior (miscompilation). Don't use Zig in production yet ;)

(edit: fix formatting)