News for hackers. Stuff that matters.
Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by the_duke on Sunday March 07, 2021 @05:32AM (#26374761)
What do you find radical about Zig?

I like Zig, but can't see anything particularly revolutionary about it's design.

The features mostly an evolution of concepts found in languages like D or Nim.

Between the two, Rust is much more revolutionary. Although Rust, of course, was also heavily inspired by research languages that came before.

There's very little happening in CS that hasn't already been conceptualized in the 50-70ies.

Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by yellowapple on Sunday March 07, 2021 @11:08AM (#26376961)
> What do you find radical about Zig?

Personally, I find the "bring your own allocator" philosophy to be pretty radical. Yeah, other systems programming languages can facilitate additional allocators beyond "the" allocator for the language's runtime, but Zig seems to optimize for that case, which makes it a lot more intuitive from a learning perspective (no more guessing about where the memory lives). Even Rust (last I checked) defaults to assuming some global one-size-fits-all allocator.

There's also Zig's flavor of compile-time code / metaprogramming. It's probably less powerful than Rust's macros, but I feel like it's a lot cleaner and intuitive, and I'd argue that being able to run ordinary Zig code at compile time is powerful enough of a feature for Zig's use cases. Ultimately, it's a nice happy medium between full-blown metaprogramming (like in Lisp and - from what I understand - Rust) v. preprocessing (like in C/C++).

And yeah, I'm sure there's plenty of prior art for everything that Zig does, but I don't know of any other languages that combine these things in such a simple and intuitive and principle-of-least-astonishment-friendly way Zig does.

Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by leshow on Sunday March 07, 2021 @04:58PM (#26380177)
> Even Rust (last I checked) defaults to assuming some global one-size-fits-all allocator.

You can substitute whatever allocator you want: https://doc.rust-lang.org/std/alloc/trait.GlobalAlloc.html

Unless you're talking about some other restriction I'm not aware of.

Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by slimsag on Sunday March 07, 2021 @05:14PM (#26380308)
In Zig, you can provide a different allocator for a single data structure (or even distinct instances of the same data structure). There is no "global" allocator (what Rust lets you swap out.)

This is vastly more powerful, I have used this to tailor an allocator to specific data structures for better performance.

Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by steveklabnik on Sunday March 07, 2021 @05:18PM (#26380331)
That is what they're referring to, it is a single global allocator, rather than a per-data structure or per-instance one. You can do this in Rust, there's just no abstraction for it. One is coming.
Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by glandium on Sunday March 07, 2021 @05:31PM (#26380432)
I don't know Zig, but it sounds like Zig allows to use arbitrary allocators for anything. The abstraction Rust is getting will only work for things that do account for using arbitrary allocators. Anything that doesn't will end up using the global allocator. That's a significant difference.
Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by yellowapple on Sunday March 07, 2021 @06:19PM (#26380741)
To clarify, it's up to the function being called; the convention set by Zig's standard library is that if a function needs to allocate memory, then the allocator (specifically, a struct of function pointers to an allocator's implementations of realloc and shrink) should be one of the function's arguments.

There is of course nothing stopping a function from ignoring this and using the C global allocator if need be (or, as I've done in some experiments, using a C library's custom allocator - in my case that of SQLite).

(EDIT: from what I understand, there's technically nothing stopping C from using this sort of strategy, either; a struct of function pointers ain't exactly exotic. It's just a matter of libraries being written with that convention in mind, which doesn't seem to be very common.)

Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by glandium on Sunday March 07, 2021 @06:36PM (#26380863)
Oh so it has, in fact, the same caveat as Rust's scheme.

Edit: I guess the difference is that Zig doesn't have years of not supporting it, making the ecosystem more prone to support it.

Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by roca on Monday March 08, 2021 @07:24AM (#26384924)
In a large complex application you are going to want to use the same allocator everywhere, or close to everywhere, and almost all functions may allocate memory directly or indirectly, in which case this Zig convention will require most every function to have a useless parameter. That sounds enraging.
Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by tjalfi on Monday March 08, 2021 @11:48AM (#26387879)
It's common to use multiple allocators in some domains; game developers often use a bump allocator[0] which is reset at the end of every frame.

[0] https://twitter.com/SebAaltonen/status/1080235671883841541

Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by roca on Monday March 08, 2021 @02:28PM (#26390360)
Thanks, that's a good counterexample.
Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by yellowapple on Monday March 08, 2021 @01:38PM (#26389670)
A large complex application seems like exactly the kind of environment where being stuck with a single allocator would be enraging. I personally like the idea of being able to give each component of a large system its own fixed chunk of memory (and an allocator over that chunk), such that if one component goes crazy with memory consumption it's isolated to that component instead of choking out the whole system.
Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by roca on Monday March 08, 2021 @02:45PM (#26390556)
That makes a little bit of sense if by "component" you mean not a software component but a unit of work users care about (e.g. a document).
Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by yellowapple on Monday March 08, 2021 @03:09PM (#26390816)
It's applicable in either case:

- As you mentioned, if I'm editing a document, it's useful to have an allocator on a chunk of memory dedicated to that document. When I close the document, I can then simply free that chunk of memory - and everything allocated against it.

- If I'm implementing an operating system, I'm probably going to want to give each application, driver, etc. a limited amount of memory to use and an allocator against that memory, both so that I can free a whole process at once when it terminates and so that a single process can't gobble up memory unless my operating system specifically grants it more to use (i.e. by itself allocating more memory and making the process' allocator aware of it).

Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by roca on Monday March 08, 2021 @03:45PM (#26391265)
> When I close the document, I can then simply free that chunk of memory - and everything allocated against it.

You probably don't want to do this directly. Instead you want to walk the object graph and run cleanup code for everything in that graph, because in general there will be resources that aren't just memory that need to be released, and for consistency with normal operation, it should deallocate memory as it goes.

You probably don't want to allocate an actual "chunk of memory" either. That just creates unnecessary fragmentation. All you really need is accounting and the ability to report when you're consuming too much memory.

Your driver example is not an example where you would allocate memory per software component. You would actually want to allocate per device, not per driver module; it's just confusing because in many cases there is only one device. But if you can plug in many devices that use the same driver, you'd want independent allocation accounting per device.

Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by yellowapple on Monday March 08, 2021 @07:04PM (#26393412)
> in general there will be resources that aren't just memory that need to be released

Zig already handles this with its "defer" feature; as a resource goes out of scope, it can be released automatically. In the document example, that document's existence would likely be a running function, and as that function terminates, it would likely have "defer" statements kick in that free the document's chunk of memory and release any file descriptors and such.

> You probably don't want to allocate an actual "chunk of memory" either. That just creates unnecessary fragmentation.

If anything that should help reduce fragmentation, or at least help reduce its impacts, since you have better control over whether that allocation exists as a contiguous block.

> All you really need is accounting and the ability to report when you're consuming too much memory.

Which is trivial to do when you know for sure that a given component can only work with a given chunk of memory.

But yeah, nothing stopping anyone from implementing an allocator that cares nothing about where its bytes actually live, and just keeping a running tab of how much memory it's used. That is: using custom allocators is an elegant and simple way to implement that accounting, since that's basically what an allocator already is.

> But if you can plug in many devices that use the same driver, you'd want independent allocation accounting per device.

We're probably talking about the same thing here, then, but with slightly different terminology (and perhaps different structure); I'd be pushing for each device to be controlled by an instance of a driver (much like how an ordinary process is an instance of a program), and it would be those per-device instances that would each have their own allocator. Those instances are what I'm calling "drivers" in this context; they might share the same code, but they run independently (or at least they should run independently; a single malfunctioning disk shouldn't bring down all the other disks).

Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by roca on Monday March 08, 2021 @08:25PM (#26394176)
> that document's existence would likely be a running function

No, that would mean an application managing multiple documents would need one thread per document, which is not normal practice for GUIs. In fact it would then need one event loop per document thread which is not even possible on many platforms.

"defer" simply doesn't serve as a wholesale replacement for destructors, but that's a tangent to this discussion.

> If anything that should help reduce fragmentation

No, there would be fragmentation at document granularity. For example, if you create a document, add a lot of content to it, then delete some of that content, then do that again for several documents, the memory used would be the sum of the maximum sizes of the documents.

I agree with the rest of your comment.

Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by yellowapple on Tuesday March 09, 2021 @05:22AM (#26397318)
> No, that would mean an application managing multiple documents would need one thread per document, which is not normal practice for GUIs.

Unless those functions are async, which Zig also supports (even on freestanding targets!). Single OS thread, single event loop, many concurrent cooperatively-scheduled functions. Or you can get fancy and implement a VM that in turn runs preemptively-scheduled userspace processes, in essence basically reinventing Erlang's abstract machine (and this is exactly a pet project I'm working on, on that note).

And even keeping each document in its own (OS) thread ain't really that unprecedented; browsers already do this, last I checked (each open tab being a "document" in this context) - in some cases (like Chrome) even doing one "document" per process.

> For example, if you create a document, add a lot of content to it, then delete some of that content, then do that again for several documents, the memory used would be the sum of the maximum sizes of the documents.

Would that not also be the case if all those documents used a single shared block of memory? Again, splitting things up helps avoid fragmentation here, especially if you know that most documents won't exceed a certain size (in which case fragmentation is only an issue for data beyond that boundary) - or, better yet, if you ain't storing the whole document in memory, in which case the buffer of actively-in-use data can be fixed. Further, if each allocation is a whole page of memory, then that's about as much control over fragmentation as an application can hope for beyond itself being the OS (and probably won't make much of a difference if those pages are scattered across RAM anyway; swapping would definitely suffer on spinning rust, but that's already bad news performance-wise anyway).

Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by roca on Tuesday March 09, 2021 @05:56PM (#26405603)
> And even keeping each document in its own (OS) thread ain't really that unprecedented; browsers already do this, last I checked (each open tab being a "document" in this context) - in some cases (like Chrome) even doing one "document" per process.

That is not correct. (Source: I am a former Mozilla Distinguished Engineer.)

Chrome (and Firefox, with Fission enabled) do one process "per site", e.g. one process for all documents at google.com. (In some cases they may use finer granularity for various reasons, but that's the default.) In each process, there is one "main thread" that all documents share.

> Would that not also be the case if all those documents used a single shared block of memory?

No. Memory freed when you delete content from one document would be reused when you add content to another document.

Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by steveklabnik on Sunday March 07, 2021 @06:01PM (#26380645)
That is being backfilled in; Vec already implements it on nightly, IIRC.

And really, what you're talking about here is "the standard library data structures," which aren't super likely to be used in firmware anyway. It's a lot easier for ecosystem data structures to add support, after all, they already would choose to call the global allocator, so now they can do either. And it is much easier for them to cut backwards-incompatible changes, if they have to.

Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by yellowapple on Sunday March 07, 2021 @06:25PM (#26380780)
> which aren't super likely to be used in firmware anyway.

Why not? Zig's std library is specifically designed to be usable for freestanding/baremetal targets (e.g. firmware), and the compiler is smart enough to only include the parts of a library (including std) that are actually used. If you do need to reimplement a part of std, you can just... reimplement that part, and import your own implementation instead of the one from std.

Unless you're talking purely about Rust?

Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by steveklabnik on Sunday March 07, 2021 @06:35PM (#26380856)
I am talking purely about Rust, yes. Firmware tends to use libcore, and if it does happen to dynamically allocate memory, liballoc. libstd assumes you have an OS, so...
Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by glandium on Sunday March 07, 2021 @06:12PM (#26380708)
I mean, in terms of Rust, it sounds like Zig allows to use any allocator for anything in any crate. Not only structs in std or other crates that explicitly allow a custom allocator. In Rust, and only talking about std, you'd need to change a lot of things to allow e.g. BufWrite, etc. to use a custom allocator. And every crate that uses types that allocate stuff under the hood. But maybe I'm misunderstanding what Zig allows.
Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by steveklabnik on Sunday March 07, 2021 @06:37PM (#26380866)
You are not misunderstanding what Zig allows, but Rust can do the same thing. https://doc.rust-lang.org/stable/core/alloc/trait.Allocator.... just isn't stable yet. And it's conventional for it to take this as an argument for everything that needs it in Zig.

BufWrite would do it the same as any data structure would, an additional parameter, all the same.

Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by glandium on Sunday March 07, 2021 @06:46PM (#26380919)
I actually did misunderstand. I thought it allowed callers to give an allocator and the callees didn't have to know for it to be used.
Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by yellowapple on Sunday March 07, 2021 @10:10PM (#26381975)
I mean, I'd say you were mostly right, in the sense that the callee doesn't know the implementation details of the passed allocator; it's only aware of the interface (i.e. the struct of function pointers that defines that interface).
Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by leshow on Sunday March 07, 2021 @09:05PM (#26381673)
What's the use case? I'm trying to think of a situation where you'd want to do that were you might not just make a separate binary that uses the other allocator
Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by chubot on Monday March 08, 2021 @02:15AM (#26383026)
Custom allocators are extremely common in C and C++ code, and often improve performance over general purpose allocators (though they can also make things worse if you're not careful).

The C++ STL has custom allocators (though the initial design was sort of botched; there is a new polymorphic allocator mechanism that aims to fix it IIRC)

Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by yellowapple on Sunday March 07, 2021 @10:21PM (#26382021)
Zig's docs cover a few different scenarios, but a couple of interest to me at least:

- Arena allocators, where your code allocates a chunk of memory and then creates its own allocator just for that chunk; when that chunk gets freed, so does everything in it. Handy for short-lived data.

- Using different allocators for different regions of memory makes it trivial to compartmentalize things; you could use the OS allocator (or a straight pointer if you're implementing your own OS) to preallocate chunks of memory, slap allocators on those chunks, and hand those to different components, preventing any given component from bringing down the whole program due to a memory leak.

- It's possible to use allocators for verifying code correctness (e.g. detecting memory leaks, testing code under memory exhaustion conditions, etc.).

Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by pron on Sunday March 07, 2021 @06:26AM (#26375006)
Zig's main feature is what it doesn't have, and the languages you mentioned don't have that feature.

Other languages also have more-or-less general partial evaluation constructs, but they're not revolutionary because they didn't realise they can express traditional constructs in terms of partial evaluation. Zig is revolutionary in that its simple partial evaluation construct replaces generics/templates, typeclasses/concepts/traits, macros and conditional compilation. The result is something that is consistent, extremely powerful, and yet exceptionally simple.

Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by philosopher1234 on Sunday March 07, 2021 @02:26PM (#26378895)
Hacker news has a really hard time valuing simplicity. I think it’s an egotistical thing: I’m smart so I don’t need a simple language.

What people miss is that a simple a language allows you to apply your smarts to solving the actual problems in front of you instead of puzzling over language features. You can only handle so much cognitive load at once, and ideally the vast majority of that should be devoted to whatever problem you’re solving, not to the language itself.

Ironically, this fact is the same fact that makes complex languages more fun for hobbyists. There’s simply more to explore, and to try, and to solve, when your object isn’t building a product but instead playing with a language. It’s a different purpose, but people very rarely acknowledge this fact, likely because they’d rather pretend their purposes are clearly mechanical and business oriented. It’s ok to just want to have fun sometimes.

Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by the_duke on Sunday March 07, 2021 @03:02PM (#26379232)
That's an interesting observation.

It definitely has merit. I've run into this exact same thing in type-system heavy languages like Haskell/Scala/Rust, where I spend more time juggling abstractions than implementing features.

But there is an additional dimension: abstractions often make the first implementation much more cumbersome. But most code is maintained and read much more than it is written, and abstractions can make extending and maintaining a code base much easier.

It's also good to remember that the existence of certain language abstractions doesn't mean you have to use them.

You have to find the right tradeoff.

Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by philosopher1234 on Sunday March 07, 2021 @03:11PM (#26379329)
Another point I disagree with the consensus about! Abstractions have their place, but a bad abstraction ends up spending more of its life getting torn down than it does productively simplifying the code. In my opinion each abstraction increases system cognitive load, and so they should only be added “lazily”, ie when empirical experience with the code proves that a particular abstraction would have broad and deep utility.
Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by pron on Sunday March 07, 2021 @06:12PM (#26380706)
But simple languages don't mean more cumbersome abstraction -- take Scheme for example -- while C++ and Haskell's maintenance record isn't particularly great (Rust doesn't have one yet).
Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by cjohansson on Sunday March 07, 2021 @04:13PM (#26379818)
In all programming languages you must understand how the parser will understand and translate what you write, it sounds like Zig will always know your intentions and maximize your code, it sounds too good to be true
Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by pron on Sunday March 07, 2021 @06:08PM (#26380691)
If Zig were a high-level language it wouldn't have been so impressive. There are plenty of simple high-level languages. But in low-level domains there is a lot of extra accidental complexity because you need to precisely define layout, control memory allocation and placement, and try to avoid, or at least control branches or dynamic dispatch. And you need to do all that in the worst-case, and you don't have a JIT. So far there have been two approaches -- C, which is linguistically simple but inexpressive and spectacularly unsafe, and C++/Ada/Rust, which is much safer and much more expressive but incredibly complex. Zig finds a new way, not through some magic, but through really exceptional and radical design.
Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by cjohansson on Monday March 08, 2021 @01:27AM (#26382792)
But low-level languages also get translated by a parser so I don't see the difference.. Maybe you mean that languages has less abstractions and because of this simplier translations? Think I need to try Zig out to get a grip on it..
Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by pron on Monday March 08, 2021 @04:01AM (#26383581)
I mean that in low-level languages the name of the game is how to work with low-level details, and Zig's design makes it exceptionally pleasant as it uses the same language for those details as it does for the logic. There is no magic here, just very clever and careful design.
Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by the_duke on Sunday March 07, 2021 @06:38AM (#26375066)
As mentioned in my other (wall of text) comment, that's not strictly beneficial.

It forces you to implement a lot of logic in "userspace" that other languages do for you automatically.

Complexity for certain abstractions moves from the language to user code, at the expense of consistency, cohesion, totality and (auto-generated) documentation.

It will be interesting to see how things play out for Zig once the ecosystem grows a little and libraries appear, but there are very significant downsides to this approach.

Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by pron on Sunday March 07, 2021 @06:53AM (#26375162)
Which of those is more beneficial indeed remains to be seen and might end up being purely a matter of personal taste; the very thing you call a downside I see as an upside. I think that talking about the positives "consistency and cohesion" where composing primitives works as a positive is merely a matter of habit. Zig treats some aspects that other languages sees as primitives as if they were any other part of the language, where code and libraries rule rather than a growing collection of primitives. I do agree that in principle a language could be too unstructured for some domains (Lisp?) but interestingly, Zig didn't go as far as syntax macros, whereas Rust did. Anyway, Zig finds a surprising middle-ground that is, as yet, hard to definitively judge, whereas Rust, for better or worse, is more of the same.
Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by anp on Sunday March 07, 2021 @12:03PM (#26377505)
I was thinking of The Lisp Curse[1] while reading your comment and then you mentioned the language! I’m quite excited by Zig (even if safety is lower priority for it than for Rust, it is really pushing the tooling envelope) but I do wonder whether the “anti-composition hypothesis” here holds up for relevant projects today. Many C programs include shims for compatibility between multiple different “library object models” (hell even strings count here) and they seem to be a common source of security and performance issues. Maybe in the domains where Zig is most competitive that dynamic won’t play out? Or maybe comptime provides tools that will still enable composition or at least allow for lower overhead “object model shims”? I suspect that it will be hard to know more about how it plays out until there is more language stability and code sharing, maybe even a repository like npm or crates.io.

[1] http://www.winestockwebdesign.com/Essays/Lisp_Curse.html

Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by nyberg on Monday March 08, 2021 @02:25AM (#26383083)
Can the "safety is lower priority than rust" trend stop? Zig is not less concerned with safety and will catch illegal behaviour at runtime for that which isn't caught at compile-time. It's only ReleaseSmall and ReleaseFast that elide this where you're able to toggle safety checks via a builtin if you wish. There's ongoing work to provide more of it within the standard library with the GeneralPurposeAllocator being an example of it.
Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by anp on Monday March 08, 2021 @07:40PM (#26393742)
I'm not suggesting that Zig isn't concerned with safety, but it's not a language designed first and foremost to offer certain safety properties. For Rust you can say "as long as you don't write the `unsafe` keyword in Rust, you'll never introduce memory or thread unsafety". Is there an equivalent for Zig? Not AFAIK but I'd actually be quite happy to be shown wrong.
Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by dilap on Monday March 08, 2021 @08:53AM (#26385695)
Use after free is the big one Zig still doesn't protect against. Yet?

So I think it's a fair statement. Rust is safety obsessed. Zig is doing its best while paying top tribute to other gods.

Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by pron on Monday March 08, 2021 @02:41PM (#26390511)
I would put it differently. Rust sacrifices anything -- including things that may hurt other aspects of correctness -- to soundly guarantee (assuming the compiler is correct) no undefined behaviour in its safe subset (and yet makes some concessions, as a large percentage of Rust programs do employ unsafe code, and so don't make such a strong guarantee), while Zig finds a different balance, at times sacrificing possible UB for the sake of helping with functional correctness. Even if you look at correctness only, it is unclear which approach, if any, offers a better story.
Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by varajelle on Sunday March 07, 2021 @06:45AM (#26375106)
As the language evolve and people will want to do actual things with it, features will be added.
Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by loup-vaillant on Sunday March 07, 2021 @12:25PM (#26377700)
Assumption 1: people don't actually use Zig.

Assumption 2: the more we do with a language, the bigger the language has to be.

I am sceptical about (1), and the only way (2) can possibly be true is if the standard library is part of the language (which it really is not: it's user space stuff, curated approved by whoever's in charge). Don't be excessively pessimistic. It's just as irrational as misguided optimism.

Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by n30phyte on Sunday March 07, 2021 @12:40PM (#26377854)
In regards to assumption 1: zig's documentation isn't the greatest, and it's still pre 1.0 which may have turned many potential users away for the moment.

I think the person you were replying to implies that after a certain point (1.0 release?), Zig's userbase will increase to such levels that it can be considered "used" by (many) people

Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by loup-vaillant on Sunday March 07, 2021 @03:21PM (#26379405)
My guess is, a few hundred users are enough to identify and correct most of what's missing in the language. Going from there to a million users is unlikely to make a big difference. Especially if the language's features are orthogonal (apparently they are), and the scope of the language is clear (the intended use case at least seems to be).

We'll see how it goes. I won't bet my hat on it, but Zig does seem to be on a good path to stay simple even as it matures.

Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by zozbot234 on Sunday March 07, 2021 @09:40AM (#26376161)
> Zig is revolutionary in that its simple partial evaluation construct replaces generics/templates, typeclasses/concepts/traits, macros and conditional compilation.

This is what C++98 did, except they called their one true comptime evaluation construct "templates", and they did it by accident. There's a reason why Rust introduced generics and typeclasses separately: C++98 templates as bespoke comptime evaluation was a disaster, and this was clear already in the C++ community.

Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by pron on Sunday March 07, 2021 @09:44AM (#26376188)
> This is what C++98 did, except they called their one true comptime evaluation construct "templates", and they did it by accident.

Right, except not at all, because templates' syntactic elements are distinct from the "object" part of the language, so it is not a partial evaluation construct for C++, but rather a separate (and rather complex) meta-language for C++. In Zig there is just Zig (with its superb error reporting mechanism), and comptime partially evaluates it. Zig distances itself from C++'s problematic design much more than Rust, which, when all is said and done, is pretty darn similar to C++.

But that's the problem with revolutionary design. Your ability to compare it to what came before it is limited because it isn't really similar to anything. Luckily, Zig can be fully learned in a day or two, so there's no need to rely on comparisons for long. You can quickly learn it and decide if it's your cup of tea or not; even if it isn't, you'd have learned something quite refreshing and inspirational, and without spending too much time.

I do agree that there is something more mysterious about Zig. Nobody knows how "good" Rust is yet, either, but it's probably no worse than C++ when we factor all elements that matter to C++/Rust developers, and we're willing to accept that it's also probably not drastically better, except maybe when it comes to undefined behaviour. Zig is more of an unknown because it is so different. It has the potential to be worse than C++, but it can also be much better. At the very least, it is very interesting in that it offers a completely new vision for how low-level programming could be done.

Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by roca on Monday March 08, 2021 @08:18AM (#26385372)
I really don't understand why someone would think Rust is "pretty darn similar to C++". I think about my code and data in Rust very differently to C++. C++ doesn't have tagged unions, Rust does. C++ does have inheritance, Rust doesn't. C++ templates are quite unlike Rust generics. Rust enforces safety and (mutable XOR shared), C++ doesn't. All of these lead to quite different design decisions for same-shaped problems.
Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by pron on Monday March 08, 2021 @02:13PM (#26390166)
You're looking at the details, while I look at the overall "feel" and find them almost indistinguishable. They're both low-level languages -- and so, like all low-level languages, suffer from low-abstraction, i.e. the difficulty to hide internal implementation details from consumers behind APIs -- that decided to invest their complexity budget to get the appearance of high-level code once you read it on the page (while the difficulty of changing it is the same as with all low-level languages), and don't hesitate to employ a fair bit of implicitness, grow a large set of features, and let compilation be slow. The details of how they do that are less important; what's most apparent is their shared design philosophy of low-level programming (although I think that Rust improves on C++ and certainly cleans it up). Zig offers a radically different approach, and one that is also radically different from C's philosophy.
Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by roca on Monday March 08, 2021 @02:53PM (#26390647)
I don't think your critiques are accurate, but anyway, the "similarity" here is that you have the same high-level critique of both languages. This does not make Rust "pretty darn similar to C++".

For me, memory and data-race safety and absence of undefined behavior are critical features, but it would be misleading if I were to go around saying "C, C++ and Zig are all pretty darn similar".

Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by pron on Tuesday March 09, 2021 @05:48AM (#26397488)
> the "similarity" here is that you have the same high-level critique of both languages.

The similarity is that they both espouse the very same design philosophy for low-level programming. It's a pretty big similarity.

> For me, memory and data-race safety and absence of undefined behavior are critical features, but it would be misleading if I were to go around saying "C, C++ and Zig are all pretty darn similar".

It would be misleading, because memory safety and undefined behaviour in Zig is much closer to Rust than to C++. Even where it's not the same as Rust, it's still very different from C/C++. Safety and correctness are as emphasised in Zig as in Rust; they just go about achieving them differently. It is not clear at all which of them achieves correctness better.

Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by roca on Tuesday March 09, 2021 @05:50PM (#26405525)
> Safety and correctness are as emphasised in Zig as in Rust

This is so far from true I cannot take you seriously.

Zig doesn't have any kind of lifetime analysis, so it's as vulnerable to use-after-free/dangling pointers as C and C++ are. That alone rules out Zig from ever being considered "memory safe" in any meaningful sense.

[Yes, I'm aware of GeneralPurposeAllocator, but that is not something you want to ship in production. "Never reuse any virtual address space" is a disaster for the OS (VMA fragmentation, TLB shootdown IPIs) and the hardware (TLBs, caches). That's why no-one ships such a thing in production for C/C++. GeneralPurposeAllocator will no doubt be useful for debugging (though less effective than ASAN or Valgrind) but safety in production is the game here; ASAN doesn't make C/C++ "memory safe".]

Zig also allows data races so Zig programs can have undefined behaviour via data races on non-atomic values. Again, this cannot be fixed.

Even smart pointers (e.g. reference counting) are nasty in Zig. Zig doesn't have destructors so cleaning up an owning pointer or a refcounting pointer requires developers to write manual "defer" statements. Worse, these only work at function scope so you also have to write manual cleanup code for every data structure containing a smart pointer. Without idiomatic smart pointers Zig will likely be more prone to UAF bugs (and leaks) than C++.

Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by pron on Tuesday March 09, 2021 @06:07PM (#26405706)
You've misunderstood. There is no doubt that Rust eliminates more undefined behaviour than Zig (though not completely), but it does it at the cost of harming other aspects of correctness. Zig does not try to eliminate UB as much as Rust, but it focuses more on reducing other types of bugs. At the end of the day, you don't care if your program fails due to UB or another bug, and it is unclear which approach results in more correct programs overall.
Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by roca on Tuesday March 09, 2021 @06:24PM (#26405827)
> you don't care if your program fails due to UB or another bug,

Actually you do, because memory safety bugs are more likely to be exploitable than some arbitrary correctness bug, because they can be weaponized to take full control of the program.

The reality is that UAF/dangling pointers are a major source of CVEs in mature software. Rust prevents those in practice, Zig doesn't. You think Zig is going to be much better than Rust at preventing other kinds of bugs. I see zero evidence of that.

Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by pron on Tuesday March 09, 2021 @06:46PM (#26406016)
I don't think Zig is going to be better than Rust at preventing other bugs. I don't know. No one knows. Software correctness is a very tricky thing about which we don't know much more than we know. UB are a cause of many bugs, and Zig eliminates many kinds of UB; Rust eliminates more. But Zig is also better at things we also know reduce bugs: simple semantics with simpler analysability, and shorter turnaround, which means more tests. In formal methods research we also have an analogous choice of approaches: more soundness at the cost of higher complexity and effort or vice-versa. There is no point hypothesising about which works better because even the experts have no idea, and it's certainly possible they are about even. The only thing that can settle this is empirical research.
Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by roca on Tuesday March 09, 2021 @07:56PM (#26406520)
As someone who cares about safety in my software I'm going to take Rust's proven benefits over Zig's unproven hopes.
Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by pron on Wednesday March 10, 2021 @03:14AM (#26409005)
Again, you misunderstand. Both Zig and Rust have much less UB than C. The delta between Rust and Zig comes at a cost to language simplicity and to more testing. You're guessing that that cost's negative impact on correctness is smaller than that positive delta. It's a reasonable guess, but so is the opposite one, and neither is more proven than the other, which would be my guess (while I don't write safety-critical code these days, I worked on safety-critical realtime software where a bug or even a later response could cost the lives of many people; in such correctness-critical domains C is preferred over C++ despite being less safe), although I would even more confidently bet that the real difference, whichever way, is small.
Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by skohan on Sunday March 07, 2021 @06:01AM (#26374891)
The approach to arbitrary compile-time execution seems like a particularly novel feature.
Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by logimame on Sunday March 07, 2021 @06:14AM (#26374958)
D, Nim, and Haxe had it for quite some time (Along with Jai, which is yet unreleased to the public), although you can argue that Zig’s implementation is conceptually the simplest (it has merged compile time semantics with generic types in a unified way).
Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by kristoff_it on Sunday March 07, 2021 @06:19AM (#26374988)
One subtle but extremely important feature of Zig's comptime is that is emulates the target architecture. Fundamental for implementing correct cross compilation.
Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by elcritch on Sunday March 07, 2021 @04:22PM (#26379893)
That's pretty impressive. It's always annoying to get bit struct alignment issues. :/
Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by pron on Sunday March 07, 2021 @06:30AM (#26375030)
Zig's revolution is not in adding a partial evaluation feature, but in removing many other separate features that can be expressed as mere applications. As Antoine de Saint-Exupery said, "Perfection is achieved not when there is nothing more to add, but when there is nothing left to take away."
Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by Rochus on Sunday March 07, 2021 @06:19AM (#26374984)
And Lisp since even a much longer time.
Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by pron on Sunday March 07, 2021 @06:29AM (#26375027)
I see some abstract aesthetic similarities between Zig and Lisp, or some Lisp's at least -- especially their minimalism -- but Zig's partial evaluation (comptime) works nothing at all like Lisp's syntactic macros (there is no quoting construct, and you don't manipulate syntactic terms at all), and, in fact, has much simpler semantics. The result is intentionally weaker than macros -- macros are "referentially opaque" while comptime is transparent[1] -- but Zig's realisation is that you don't need macros -- with their complexities and pitfalls -- to deliver everything a low-level language would need.

[1]: I.e. if x and y have the same meaning ("reference"), in Lisp -- and in any other language with macros -- you could write a parameterised expression e, such that e(x) does not have the same meaning as e(y); you can't do that in Zig.

Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by Rochus on Sunday March 07, 2021 @07:38AM (#26375417)
Thanks. I'm not familiar with Zig. I responded to the "arbitrary compile-time execution" by adding Lisp to the proposed list of D, Nim, and Haxe. Also the latter might raise some concerns when looking at details as you did with Lisp.
Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by kazinator on Sunday March 07, 2021 @09:29PM (#26381794)
For instance, even if x and y have the same meaning, (let (x) ...) and (let (y) ... ) will not. So if you can't do that in Zig, that implies you can't write a custom binding construct.
Re:Why I rewrote my Rust keyboard firmware in Zig: consisten... by pron on Monday March 08, 2021 @04:10AM (#26383628)
A let expression isn't parameterised by x and y but binds their meaning, and the bound variable isn't free in the expression, and you can't substitute it at all. A simple example is an expression that prints the syntactic name of its argument. You can do that in C, C++, Lisp and Rust, but not in Zig (this is OK, because Zig obviates the need for that with excellent backtraces). But this means that you can understand all Zig code as simple Zig code, and that there is no complex meta-language with super-powers as there is in all those other languages I mentioned.