Personally, I find the "bring your own allocator" philosophy to be pretty radical. Yeah, other systems programming languages can facilitate additional allocators beyond "the" allocator for the language's runtime, but Zig seems to optimize for that case, which makes it a lot more intuitive from a learning perspective (no more guessing about where the memory lives). Even Rust (last I checked) defaults to assuming some global one-size-fits-all allocator.
There's also Zig's flavor of compile-time code / metaprogramming. It's probably less powerful than Rust's macros, but I feel like it's a lot cleaner and intuitive, and I'd argue that being able to run ordinary Zig code at compile time is powerful enough of a feature for Zig's use cases. Ultimately, it's a nice happy medium between full-blown metaprogramming (like in Lisp and - from what I understand - Rust) v. preprocessing (like in C/C++).
And yeah, I'm sure there's plenty of prior art for everything that Zig does, but I don't know of any other languages that combine these things in such a simple and intuitive and principle-of-least-astonishment-friendly way Zig does.
You can substitute whatever allocator you want: https://doc.rust-lang.org/std/alloc/trait.GlobalAlloc.html
Unless you're talking about some other restriction I'm not aware of.
This is vastly more powerful, I have used this to tailor an allocator to specific data structures for better performance.
There is of course nothing stopping a function from ignoring this and using the C global allocator if need be (or, as I've done in some experiments, using a C library's custom allocator - in my case that of SQLite).
(EDIT: from what I understand, there's technically nothing stopping C from using this sort of strategy, either; a struct of function pointers ain't exactly exotic. It's just a matter of libraries being written with that convention in mind, which doesn't seem to be very common.)
Edit: I guess the difference is that Zig doesn't have years of not supporting it, making the ecosystem more prone to support it.
[0] https://twitter.com/SebAaltonen/status/1080235671883841541
- As you mentioned, if I'm editing a document, it's useful to have an allocator on a chunk of memory dedicated to that document. When I close the document, I can then simply free that chunk of memory - and everything allocated against it.
- If I'm implementing an operating system, I'm probably going to want to give each application, driver, etc. a limited amount of memory to use and an allocator against that memory, both so that I can free a whole process at once when it terminates and so that a single process can't gobble up memory unless my operating system specifically grants it more to use (i.e. by itself allocating more memory and making the process' allocator aware of it).
You probably don't want to do this directly. Instead you want to walk the object graph and run cleanup code for everything in that graph, because in general there will be resources that aren't just memory that need to be released, and for consistency with normal operation, it should deallocate memory as it goes.
You probably don't want to allocate an actual "chunk of memory" either. That just creates unnecessary fragmentation. All you really need is accounting and the ability to report when you're consuming too much memory.
Your driver example is not an example where you would allocate memory per software component. You would actually want to allocate per device, not per driver module; it's just confusing because in many cases there is only one device. But if you can plug in many devices that use the same driver, you'd want independent allocation accounting per device.
Zig already handles this with its "defer" feature; as a resource goes out of scope, it can be released automatically. In the document example, that document's existence would likely be a running function, and as that function terminates, it would likely have "defer" statements kick in that free the document's chunk of memory and release any file descriptors and such.
> You probably don't want to allocate an actual "chunk of memory" either. That just creates unnecessary fragmentation.
If anything that should help reduce fragmentation, or at least help reduce its impacts, since you have better control over whether that allocation exists as a contiguous block.
> All you really need is accounting and the ability to report when you're consuming too much memory.
Which is trivial to do when you know for sure that a given component can only work with a given chunk of memory.
But yeah, nothing stopping anyone from implementing an allocator that cares nothing about where its bytes actually live, and just keeping a running tab of how much memory it's used. That is: using custom allocators is an elegant and simple way to implement that accounting, since that's basically what an allocator already is.
> But if you can plug in many devices that use the same driver, you'd want independent allocation accounting per device.
We're probably talking about the same thing here, then, but with slightly different terminology (and perhaps different structure); I'd be pushing for each device to be controlled by an instance of a driver (much like how an ordinary process is an instance of a program), and it would be those per-device instances that would each have their own allocator. Those instances are what I'm calling "drivers" in this context; they might share the same code, but they run independently (or at least they should run independently; a single malfunctioning disk shouldn't bring down all the other disks).
No, that would mean an application managing multiple documents would need one thread per document, which is not normal practice for GUIs. In fact it would then need one event loop per document thread which is not even possible on many platforms.
"defer" simply doesn't serve as a wholesale replacement for destructors, but that's a tangent to this discussion.
> If anything that should help reduce fragmentation
No, there would be fragmentation at document granularity. For example, if you create a document, add a lot of content to it, then delete some of that content, then do that again for several documents, the memory used would be the sum of the maximum sizes of the documents.
I agree with the rest of your comment.
Unless those functions are async, which Zig also supports (even on freestanding targets!). Single OS thread, single event loop, many concurrent cooperatively-scheduled functions. Or you can get fancy and implement a VM that in turn runs preemptively-scheduled userspace processes, in essence basically reinventing Erlang's abstract machine (and this is exactly a pet project I'm working on, on that note).
And even keeping each document in its own (OS) thread ain't really that unprecedented; browsers already do this, last I checked (each open tab being a "document" in this context) - in some cases (like Chrome) even doing one "document" per process.
> For example, if you create a document, add a lot of content to it, then delete some of that content, then do that again for several documents, the memory used would be the sum of the maximum sizes of the documents.
Would that not also be the case if all those documents used a single shared block of memory? Again, splitting things up helps avoid fragmentation here, especially if you know that most documents won't exceed a certain size (in which case fragmentation is only an issue for data beyond that boundary) - or, better yet, if you ain't storing the whole document in memory, in which case the buffer of actively-in-use data can be fixed. Further, if each allocation is a whole page of memory, then that's about as much control over fragmentation as an application can hope for beyond itself being the OS (and probably won't make much of a difference if those pages are scattered across RAM anyway; swapping would definitely suffer on spinning rust, but that's already bad news performance-wise anyway).
That is not correct. (Source: I am a former Mozilla Distinguished Engineer.)
Chrome (and Firefox, with Fission enabled) do one process "per site", e.g. one process for all documents at google.com. (In some cases they may use finer granularity for various reasons, but that's the default.) In each process, there is one "main thread" that all documents share.
> Would that not also be the case if all those documents used a single shared block of memory?
No. Memory freed when you delete content from one document would be reused when you add content to another document.
And really, what you're talking about here is "the standard library data structures," which aren't super likely to be used in firmware anyway. It's a lot easier for ecosystem data structures to add support, after all, they already would choose to call the global allocator, so now they can do either. And it is much easier for them to cut backwards-incompatible changes, if they have to.
Why not? Zig's std library is specifically designed to be usable for freestanding/baremetal targets (e.g. firmware), and the compiler is smart enough to only include the parts of a library (including std) that are actually used. If you do need to reimplement a part of std, you can just... reimplement that part, and import your own implementation instead of the one from std.
Unless you're talking purely about Rust?
BufWrite would do it the same as any data structure would, an additional parameter, all the same.
The C++ STL has custom allocators (though the initial design was sort of botched; there is a new polymorphic allocator mechanism that aims to fix it IIRC)
- Arena allocators, where your code allocates a chunk of memory and then creates its own allocator just for that chunk; when that chunk gets freed, so does everything in it. Handy for short-lived data.
- Using different allocators for different regions of memory makes it trivial to compartmentalize things; you could use the OS allocator (or a straight pointer if you're implementing your own OS) to preallocate chunks of memory, slap allocators on those chunks, and hand those to different components, preventing any given component from bringing down the whole program due to a memory leak.
- It's possible to use allocators for verifying code correctness (e.g. detecting memory leaks, testing code under memory exhaustion conditions, etc.).
Other languages also have more-or-less general partial evaluation constructs, but they're not revolutionary because they didn't realise they can express traditional constructs in terms of partial evaluation. Zig is revolutionary in that its simple partial evaluation construct replaces generics/templates, typeclasses/concepts/traits, macros and conditional compilation. The result is something that is consistent, extremely powerful, and yet exceptionally simple.
What people miss is that a simple a language allows you to apply your smarts to solving the actual problems in front of you instead of puzzling over language features. You can only handle so much cognitive load at once, and ideally the vast majority of that should be devoted to whatever problem you’re solving, not to the language itself.
Ironically, this fact is the same fact that makes complex languages more fun for hobbyists. There’s simply more to explore, and to try, and to solve, when your object isn’t building a product but instead playing with a language. It’s a different purpose, but people very rarely acknowledge this fact, likely because they’d rather pretend their purposes are clearly mechanical and business oriented. It’s ok to just want to have fun sometimes.
It definitely has merit. I've run into this exact same thing in type-system heavy languages like Haskell/Scala/Rust, where I spend more time juggling abstractions than implementing features.
But there is an additional dimension: abstractions often make the first implementation much more cumbersome. But most code is maintained and read much more than it is written, and abstractions can make extending and maintaining a code base much easier.
It's also good to remember that the existence of certain language abstractions doesn't mean you have to use them.
You have to find the right tradeoff.
It forces you to implement a lot of logic in "userspace" that other languages do for you automatically.
Complexity for certain abstractions moves from the language to user code, at the expense of consistency, cohesion, totality and (auto-generated) documentation.
It will be interesting to see how things play out for Zig once the ecosystem grows a little and libraries appear, but there are very significant downsides to this approach.
[1] http://www.winestockwebdesign.com/Essays/Lisp_Curse.html
So I think it's a fair statement. Rust is safety obsessed. Zig is doing its best while paying top tribute to other gods.
Assumption 2: the more we do with a language, the bigger the language has to be.
I am sceptical about (1), and the only way (2) can possibly be true is if the standard library is part of the language (which it really is not: it's user space stuff, curated approved by whoever's in charge). Don't be excessively pessimistic. It's just as irrational as misguided optimism.
I think the person you were replying to implies that after a certain point (1.0 release?), Zig's userbase will increase to such levels that it can be considered "used" by (many) people
We'll see how it goes. I won't bet my hat on it, but Zig does seem to be on a good path to stay simple even as it matures.
This is what C++98 did, except they called their one true comptime evaluation construct "templates", and they did it by accident. There's a reason why Rust introduced generics and typeclasses separately: C++98 templates as bespoke comptime evaluation was a disaster, and this was clear already in the C++ community.
Right, except not at all, because templates' syntactic elements are distinct from the "object" part of the language, so it is not a partial evaluation construct for C++, but rather a separate (and rather complex) meta-language for C++. In Zig there is just Zig (with its superb error reporting mechanism), and comptime partially evaluates it. Zig distances itself from C++'s problematic design much more than Rust, which, when all is said and done, is pretty darn similar to C++.
But that's the problem with revolutionary design. Your ability to compare it to what came before it is limited because it isn't really similar to anything. Luckily, Zig can be fully learned in a day or two, so there's no need to rely on comparisons for long. You can quickly learn it and decide if it's your cup of tea or not; even if it isn't, you'd have learned something quite refreshing and inspirational, and without spending too much time.
I do agree that there is something more mysterious about Zig. Nobody knows how "good" Rust is yet, either, but it's probably no worse than C++ when we factor all elements that matter to C++/Rust developers, and we're willing to accept that it's also probably not drastically better, except maybe when it comes to undefined behaviour. Zig is more of an unknown because it is so different. It has the potential to be worse than C++, but it can also be much better. At the very least, it is very interesting in that it offers a completely new vision for how low-level programming could be done.
For me, memory and data-race safety and absence of undefined behavior are critical features, but it would be misleading if I were to go around saying "C, C++ and Zig are all pretty darn similar".
The similarity is that they both espouse the very same design philosophy for low-level programming. It's a pretty big similarity.
> For me, memory and data-race safety and absence of undefined behavior are critical features, but it would be misleading if I were to go around saying "C, C++ and Zig are all pretty darn similar".
It would be misleading, because memory safety and undefined behaviour in Zig is much closer to Rust than to C++. Even where it's not the same as Rust, it's still very different from C/C++. Safety and correctness are as emphasised in Zig as in Rust; they just go about achieving them differently. It is not clear at all which of them achieves correctness better.
This is so far from true I cannot take you seriously.
Zig doesn't have any kind of lifetime analysis, so it's as vulnerable to use-after-free/dangling pointers as C and C++ are. That alone rules out Zig from ever being considered "memory safe" in any meaningful sense.
[Yes, I'm aware of GeneralPurposeAllocator, but that is not something you want to ship in production. "Never reuse any virtual address space" is a disaster for the OS (VMA fragmentation, TLB shootdown IPIs) and the hardware (TLBs, caches). That's why no-one ships such a thing in production for C/C++. GeneralPurposeAllocator will no doubt be useful for debugging (though less effective than ASAN or Valgrind) but safety in production is the game here; ASAN doesn't make C/C++ "memory safe".]
Zig also allows data races so Zig programs can have undefined behaviour via data races on non-atomic values. Again, this cannot be fixed.
Even smart pointers (e.g. reference counting) are nasty in Zig. Zig doesn't have destructors so cleaning up an owning pointer or a refcounting pointer requires developers to write manual "defer" statements. Worse, these only work at function scope so you also have to write manual cleanup code for every data structure containing a smart pointer. Without idiomatic smart pointers Zig will likely be more prone to UAF bugs (and leaks) than C++.
Actually you do, because memory safety bugs are more likely to be exploitable than some arbitrary correctness bug, because they can be weaponized to take full control of the program.
The reality is that UAF/dangling pointers are a major source of CVEs in mature software. Rust prevents those in practice, Zig doesn't. You think Zig is going to be much better than Rust at preventing other kinds of bugs. I see zero evidence of that.
[1]: I.e. if x and y have the same meaning ("reference"), in Lisp -- and in any other language with macros -- you could write a parameterised expression e, such that e(x) does not have the same meaning as e(y); you can't do that in Zig.
I like Zig, but can't see anything particularly revolutionary about it's design.
The features mostly an evolution of concepts found in languages like D or Nim.
Between the two, Rust is much more revolutionary. Although Rust, of course, was also heavily inspired by research languages that came before.
There's very little happening in CS that hasn't already been conceptualized in the 50-70ies.