News for hackers. Stuff that matters.
Re:Some Thoughts on Zig by ghoward on Friday July 22, 2022 @07:52AM (#32190967)
Not GP.

> I'm very skeptical that the lack of memory safety is not going to be an issue "in practice".

For what it's worth, I agree with you. And I'm a C programmer that managed to put out a non-trivial piece of C code ([1]) that has had no memory safety issues triggered in the wild after 1.0 (which was released on 26 Oct 2018, so almost 4 years, yay!) that I know of.

The truth is that I am not able to write such code; I just fuzz the ever-living daylights out of my code with lots of asserts to test all of the assumptions that my code makes, including preconditions, postconditions, everything. I also use static analysis, -Weverything on Clang (without -Wpadded; that's a stupid warning), and a massive test suite with near 100% coverage. In my new projects, I'm even using fuzzing to generate almost all of my test cases so that my path coverage is as good as possible.

That's a lot of work, and it's not fun.

But what would make it even more effective would be to effectively automatically add asserts for me to test the assumption that an index is in range whenever it is used! And that's why memory-safe languages are the only way to go: not only do they save people from bad code, they help programmers turn bad code into good code with the tools at their disposal.

I think the Zig people know this, which is why they have ReleaseSafe for testing and for those cases where you have to be sure. But I also don't see them test their code enough for me to trust that it's not needed. And I doubt that Zig users test their code enough for it too.

All of this is to say, I completely agree with you despite (so far!) being the exception to the rule you articulated, and I dread the day when my reign of "being the exception" comes to an end. I really want to stay being the exception.

[1]: https://git.yzena.com/gavin/bc

Re:Some Thoughts on Zig by Schroedingersat on Friday July 22, 2022 @10:12AM (#32192027)
The goal of the Zig project isn't to get rid of that work. It's to make it easier.

ReleaseSafe is supposed to be the 'normal' way to compile (the same role O2 takes now albeit with a performance penalty compared to O2), with releasefast or releasesmall for those times performance is unacceptable and where you've satisfied yourself with the level of testing you've done.

The philosophy of the project is to build some of those static analyses and runtime checks that help fuzzing into the compiler and stdlib. And build a culture and toolchain where it's easy to start using the rest as part of the status quo (rather than them being the exotic thing used by weird people). Finally they get rid of some of the most egregious footguns ( ostly relevant to beginners), passing slices is much more ergonomic and less error prone for a beginner, the error handling is best in class, and there are optional types and less use of void pointers.

Of course doing the actual hard work is always going to be hard work. But you can turn it from hard and gruelling work where the majority of people have no idea of the things they should even be doing and where you uave to keep a laundry list of gotchyas in your head into hard but satisfying work that feels fathomable.

It's a different philosophy from rust, but one that is no less valid.

Re:Some Thoughts on Zig by ghoward on Friday July 22, 2022 @08:56PM (#32199060)
The error handling is so far from best-in-class that it's not even funny. Lisp conditions and restarts, which can be implemented in C, are best-in-class. Beyond that, does Zig allow arbitrary values as errors now? Anything that doesn't is not best-in-class; they would be worst-in-class.

Footguns are never just relevant to beginners. I say this as something close to an expert C programmer. I hit footguns all of the time. I just don't release them because I always assume I hit them and hunt them down.

Making things easier to fuzz is table stakes; even C can do that now. I set up fuzzing for my code with a simple build option. It's not hard. It's also effective at finding test cases for you, if you use it right.

Does Zig do Design by Contract? That would make fuzzing easier. Does it do Property-Based Testing? That's a more focused version of fuzzing. In fact, it's the same to unit testing as fuzzing is to integration testing. Until Zig has Property-Based Testing, I don't believe it's making testing easier.

For that matter, as far as I know, Zig only makes unit tests easy. Unit tests are okay, but not really effective. The test suite for my code is only integration tests, which makes things easier to refactor. And yet it is effective at finding bugs.

tl;dr: What Zig has is table stakes in 2022. It is nothing special for helping people write correct code.

Re:Some Thoughts on Zig by Schroedingersat on Saturday July 23, 2022 @09:54PM (#32209498)
> The error handling is so far from best-in-class that it's not even funny. Lisp conditions and restarts, which can be implemented in C, are best-in-class.

Adding gotos is not an upgrade to readability and debugability, and comefrom was supposed to be a parody.

Re:Some Thoughts on Zig by ghoward on Saturday July 23, 2022 @11:09PM (#32209954)
I suggest you read up on Lisp conditions and restarts. They are not exceptions. There is no goto other than a function call and a loop back to the spot the programmer picks. And there is no comefrom.

The C equivalent would look like this:

    do
    {
        err = some_operation_that_can_return_errs();
    }
    while (err != NULL && handleError(err) == ERROR_HANDLED);

    if (err != NULL)
    {
        panic("Could not handle error: %s\n", err_str(err));
    }
That is not goto or comefrom. Far from it. And of course, the panic at the end is not necessary; you could then return the error, or throw an exception, etc.
Re:Some Thoughts on Zig by Schroedingersat on Sunday July 24, 2022 @06:26AM (#32212005)
I'm struggling to understand how your example is not just something any language with error types can do trivially.

Every explanation of conditions and restarts I can find explicitly mentions searching the call stack for a handler and jumping into it, then reserving the capability to return to the function the error originated in.

This is explicitly non-local and hidden control flow (ie. the much maligned goto of exceptions).

The only advantage I can see asserted of having handlers (exceptions or conditions) that way is that you don't have to look for the error and pass it up manually but ! does that anyway so it doesn't seem worth throwing out your ability to reason about invariants in the context of a language with mutability and such.

Then there's returning to the stack frame you started in (the part I somewhat unfairly called comefrom). You need to drag around all the machinery required for continuations in order to do this. If the goal is to have available some of the context of an outer stack frame, you could pass the relevant references down and just handle it where you wished to resume.

Further thoughts: In some ways errdefer is the polar opposote of this concept. It allows the function to clean up and preserve invariants whilst passing control up the call chain.

Re:Some Thoughts on Zig by ghoward on Sunday July 24, 2022 @11:32PM (#32220240)
> Every explanation of conditions and restarts I can find explicitly mentions searching the call stack for a handler and jumping into it, then reserving the capability to return to the function the error originated in.

> This is explicitly non-local and hidden control flow (ie. the much maligned goto of exceptions).

No, it's a function call. That's how you jump to code while reserving the capability to return.

I mean, you explicitly "raise" an error, right? What's wrong with code like:

    raise AllocationFailure;
desugaring to a function call? I thought function calls were local and non-hidden control flow, so why do you think otherwise?

> The only advantage I can see asserted of having handlers (exceptions or conditions) that way is that you don't have to look for the error and pass it up manually but ! does that anyway so it doesn't seem worth throwing out your ability to reason about invariants in the context of a language with mutability and such.

One example is that perhaps you do want to ignore certain errors, but let others through.

I have code that returns an error on EOF. Usually. In reality, it calls an error handler and lets the handler decide.

When I open and read files to the end, I'll encounter EOF, but I want to. So I'll push a handler onto the stack of error handlers that ignores EOF but handles all other errors. If I did that with exceptions, the call stack would be destroyed. Not good.

Another example is memory allocation failure. Maybe one part of your program caches allocations and needs to be told to give them up if allocation failed. So in the handler for allocation failure, it calls a function to release those cached allocations, and when the handler returns, the allocator can try again. Hence, the restart.

> You need to drag around all the machinery required for continuations in order to do this. If the goal is to have available some of the context of an outer stack frame, you could pass the relevant references down and just handle it where you wished to resume.

No, you don't. If you did, I wouldn't have been able to implement it in C, but I did. [1]

> Further thoughts: In some ways errdefer is the polar opposote of this concept. It allows the function to clean up and preserve invariants whilst passing control up the call chain.

Passing control up the call chain on error should be a last resort because you're destroying context, but I agree, it should still exist.

The cool thing is that you can do this in C too, even with cleaning up and preserving invariants. See [2], [3], and [4].

[1]: https://git.yzena.com/Yzena/Yc/src/branch/master/src/concurr...

[2]: https://git.yzena.com/Yzena/Yc/src/branch/master/src/alloc/s...

[3]: https://git.yzena.com/Yzena/Yc/src/branch/master/src/alloc/s...

[4]: https://git.yzena.com/Yzena/Yc/src/branch/master/src/alloc/s...

Re:Some Thoughts on Zig by Schroedingersat on Monday July 25, 2022 @03:52AM (#32221646)
> raise AllocationFailure;

This is exception semantics which is hidden control flow present in every language with exceptions. Even in checked exceptions it is often obfuscated (ie. a function call that can raise is not distinguishable from one that can't wkthout manually looking at the types, and if they are distinct, then you just have something equivalent to error types). The point is that, upon calling a function that might transitively call a function that raises, you have no idea if the next line will execute or whether you will wind up in some other piece of code altogether.

It's not equivalent to calling a function because it destroys the stack context and violates the invariants and ability to reason about control flow for every function above it in the call stack.

I am starting to grok algebraic effects, and how they can make exceptions-but-not-objectively-incorrect by preserving context during the interrupted control flow and returning everything how it was left, but they come at a price.

You could guarantee that the stack will be reconstructed and execution will continue where it left off. But now 'raise' implicitly does a bunch of stuff including allocating memory on the heap and copying every stack frame between your error and your handler onto it. How does this function if your error is that you are unable to allocate?

Even then, you don't know where control will end up without hunting for the registration of the handler, which may not even be near the call site.

Sure you can build a stack on the heap in userland in C and dump everything there so as to reconstruct it (I skimmed your code and it seems like some implementation of coroutines or similar, I'm notnsure if this is part of the error machinery). There's nothing in zig that prevents you from doing this, or even using that exact library.

> The cool thing is that you can do this in C too, even with cleaning up and preserving invariants. See [2], [3], and [4].

I either can't follow your code with the amount of effort I'm willing to expend (likely) or there isn't anything resembling the descriptions I can find for conditions and restarts there.

Are you saying that the following can happen?

I can have some function foo that is regular C code with the exception of registering that it handles some condition.

Then foo can call, bar, which calls baz which calls qux, all of which are regular C code.

Then qux can call something which raises a condition of the kind foo registered.

Then control flow passes to foo which handles the error using the context in its current stack frame.

Then control passes back into qux with information provided by foo's handling code.

Then qux returns, followed by baz and bar returning without having to have any knowledge of or participation in your system.

And that all of this happens without constructing a closure over foo's context and sneaking it into qux via a global.

Re:Some Thoughts on Zig by ghoward on Monday July 25, 2022 @12:20PM (#32226591)
> This is exception semantics which is hidden control flow present in every language with exceptions.

This is not exception semantics. A language can make the keyword "raise" do whatever it wants.

Should I have used some other keyword? Fine:

    handle AllocationFailure;
In fact, let me expand it a bit:

    restart
    {
        err = allocate(size);
        if (err != NULL) handle AllocationFailure;
    }
This code calls the AllocationFailure error handler when allocation fails, and has the option of "restarting" (i.e., going back to the restart keyword and trying again). There is no hidden control flow, unless you think `errdefer` is hidden control flow.

> It's not equivalent to calling a function because it destroys the stack context and violates the invariants and ability to reason about control flow for every function above it in the call stack.

No, it doesn't. Again, I implemented this in C. I don't use longjmp(). It does not destroy stack context. You seem to have no idea what you are talking about.

> You could guarantee that the stack will be reconstructed and execution will continue where it left off. But now 'raise' implicitly does a bunch of stuff including allocating memory on the heap and copying every stack frame between your error and your handler onto it. How does this function if your error is that you are unable to allocate?

Yes, by doing a function call. It's a function call. Really.

> Even then, you don't know where control will end up without hunting for the registration of the handler, which may not even be near the call site.

The registration of the handler does not need to use the stack. In my code, it uses a separate heap-allocated stack, which is in the thread-local data.

Did you not read the actual C code I sent you? It's all in there.

> Sure you can build a stack on the heap in userland in C and dump everything there so as to reconstruct it (I skimmed your code and it seems like some implementation of coroutines or similar, I'm notnsure if this is part of the error machinery). There's nothing in zig that prevents you from doing this, or even using that exact library.

It's not coroutines. It's simple function calls. It's using function pointers, yes, but it's just function calls.

> I either can't follow your code with the amount of effort I'm willing to expend (likely) or there isn't anything resembling the descriptions I can find for conditions and restarts there.

Because your mental model of conditions and restarts is completely wrong.

> Are you saying that the following can happen?

No, that is not what happens. This is what happens:

You have some function foo that is regular C code with the exception that it registers ipsom as a handler for some condition until foo returns.

So ipsom is a registered handler until foo is done executing.

Then foo can call bar, which calls baz, which calls qux, all of which are regular C code.

Then qux can call something which raises a condition of the kind foo registered.

Then control flow passes to ipsom, which gets a new stack frame, and which handles the error using the context in its new stack frame.

Then control returns back into qux with information provided by foo's handling code.

Then qux returns, followed by baz and bar returning without having to have any knowledge of or participation in your system.

And that all of this happens without constructing a closure over foo's context and sneaking it into qux via a global because foo's context is never used as the error handler!

Again, it's just function calls and a possible jump to the beginning of a marked block.

To show this, I have fully implemented restarts now ([1]), which I didn't do because I didn't need them yet.

Now, if you had a hypothetical foo that registers ipsom as an error handler, you would do this in foo:

    y_REGISTER_HANDLER(ipsom);
    // The rest of the code.
    y_UNREGISTER_HANDLER;
I need to hook error handler registration into my RAII system so that you only have to register it (and it will automatically be unregistered on function exit), but you get the idea.

Now, in the function that actually raises the condition:

    y_RESTART(restart)
    {
        err = function_that_returns_err();
        if (err != y_ERROR_SUCCESS) y_HANDLE(err, status, restart);
    }
Of course, it's C, so the macros are a little ugly and requiring passing in variable names, but it works.

In this code, if `err` is not a success value, then ipsom is called, which will either not handle the error, handle the error but not allow a restart, or handle the error and allow a restart. If the last one is what happens, the code will jump back up to the beginning of the y_RESTART block and start over.

Edit: I realized my commit forgot to add y_strucon_handleErrorWithRestart(). It was added in [2].

[1]: https://git.yzena.com/Yzena/Yc/commit/95312b059132dc3e8614b4...

[2]: https://git.yzena.com/Yzena/Yc/commit/61c4a3ef735446a9478c58...

Re:Some Thoughts on Zig by Schroedingersat on Monday July 25, 2022 @08:12PM (#32232675)
Then you haven't made conditions and restarts.

You've made error values. This is just one trivial thing you can do with error values without adding keywords. As you said, it's just a function call.

This is the error system that languages like zig and rust have.

Ie. the best paradigm for error handling you can get without all thenoverheadnof a pure FP language.

Except Zig has a bunch of erganomic enhancements to make them far more pleasant to deal with and reduce the mental overhead and scope for mistakes. You can't get the value out of a failable function without acknowledging and passing up or handling the error, and you don't need the verbiage of if err != nil adding noise everywhere. And it comes with batteries included so you're not going to use code that uses something stupid like errno instead without noticing. And you get errdefer to mitigate one of the biggest downsides to saying 'I can't handle this here, passing an error up'.

They can't carry arbitrary data, because where would you put it in a language that has an error system capable of treating allocation failure as a handleable error? There's nothing stopping you from putting some state or context elsewhere, but you have to make those decisions.

Similarly you can have the error get handled in a parent scope without unainding as I demonstrated if that's really really important, but it shouldn't be the default.

Re:Some Thoughts on Zig by ghoward on Monday July 25, 2022 @11:14PM (#32233841)
> You've made error values. This is just one trivial thing you can do with error values without adding keywords. As you said, it's just a function call.

Well, yeah, with some "ergonomics." Like walking a stack of error handlers. Across threads. Go back and read my code; that's what it's doing: a stack walk across threads, followed by a jump backwards depending on certain conditions.

Does Zig do that? Not builtin, for sure.

Could you implement that in Zig? Sure, but then you're outside its "best-in-class" error system.

> This is the error system that languages like zig and rust have.

So you're admitting that Zig does not have any better error handling than C? Because C has return values.

At least Rust lets you use Option values and Result values.

> Ie. the best paradigm for error handling you can get without all thenoverheadnof a pure FP language.

Sigh...I just implemented something from a functional programming language in C. Overhead is minimal.

> Except Zig has a bunch of erganomic enhancements to make them far more pleasant to deal with and reduce the mental overhead and scope for mistakes. You can't get the value out of a failable function without acknowledging and passing up or handling the error, and you don't need the verbiage of if err != nil adding noise everywhere.

That's just manual exceptions with the compiler complaining if you get it wrong. I mean, sure, it's great that the compiler will check your work, but Clang and GCC can give me warnings about unused return values, so...that's not much better than C.

And they are still exceptions. You're still destroying stack frames until some function higher in the stack handles the error. When it does, all of that context is gone.

Using conditions and restarts means that you can make the decision to tear down the context after* handling the error. Exceptions are much less worse in that case. I've actually found them to be useful in certain cases, but only if the error is handled first.

Also, you can add those ergonomics in C. Like so:

    #define try(var, expr) do { typeof(var) ret_ = (expr); if (is_err(ret_)) return ret_; var = ret_; } while (0)
> And it comes with batteries included so you're not going to use code that uses something stupid like errno instead without noticing. And you get errdefer to mitigate one of the biggest downsides to saying 'I can't handle this here, passing an error up'.

I will say that errdefer is a good idea. I did implement basically that.

But no, Zig's error system doesn't come with batteries included because it can't carry arbitrary data. And it doesn't do a walk of a stack of registered handlers. And it doesn't do restarts. And it doesn't have Option or Result, nor interfaces so you could implement the same thing for other types.

Whatever you might think of conditions and restarts, if you seriously think that Zig's error system is best-in-class when Rust's does the same thing and more, while also allowing user code and types to use the same infrastructure, I just don't know what to tell you.

And I'm no fan of Rust. But it's better than Zig.

> They can't carry arbitrary data, because where would you put it in a language that has an error system capable of treating allocation failure as a handleable error?

Preallocate on thread startup. This is not hard.

And on allocation error, every function that allocates is controlled by the language (through the standard library), so that's not a problem in practice. And even if something tries to wrap an allocation failure in more data, and allocation fails again, just panic(). Really. You can even document that behavior in the language.

> There's nothing stopping you from putting some state or context elsewhere, but you have to make those decisions.

Yes, but that's an admission that Zig's error system is not ergonomic. Yet you claim that it is best-in-class simply because it's slightly more ergonomic than C's?

Sigh...

Re:Some Thoughts on Zig by Schroedingersat on Tuesday July 26, 2022 @12:00AM (#32234102)
> And it doesn't have Option or Result, nor interfaces so you could implement the same thing for other types.

It has both of those. The included error types are exactly result with added erganomics (passing up one frame is the easiest thing and is erganomically the same as checked exceptions)

Maybe learn what something is before arguing about it?

Re:Some Thoughts on Zig by Schroedingersat on Sunday July 24, 2022 @06:35AM (#32212050)
Actually, come to think of it, I'm pretty sure this can be achieved with a combination of async/await and error types.

If the inner function yields either a value (or none) or an error, the outer scope handles the error amd returns control to the async function (optionally with state gleaned from the handler) you have all the pieces.

I believe yield lacks any kind of syntactic sugar (planned), but is possibly in current async functions.

Viewing it through that lens, I can see it as a viable alternative to eschewing exceptions altogether.

Re:Some Thoughts on Zig by ghoward on Sunday July 24, 2022 @11:32PM (#32220241)
Adding async to your error handling would just mean that you now have two problems.
Re:Some Thoughts on Zig by Schroedingersat on Monday July 25, 2022 @05:49AM (#32222362)
That you said that seems to heavily imply that you already agree with the point that I'm trying to make about control flow.

Here's my best attempt at demonstrating conditions and restarts with zig-native tools.

Edit: fixed my dumb. https://zig.godbolt.org/z/creoT47Mj

Note that it'd get substantially more erganomic if yield were finished with all of the usual niceties of the monad-ish error keywords.

Also edit: Upon further reflection to be actually correct conditions/restarts, you'd have to unwind starting from inside the inner function if the handler gave up. This is doable by passing some state back down (handled/unhandled) from the outer scope, and then just returning an error from the bottom scope to begin either unwinding or let a non-restartable error handling mechanism do its thing.

Additionally I don't think this is compatible with also using async for its intended purpose without comptime shenanigans.

Re:Some Thoughts on Zig by ghoward on Monday July 25, 2022 @12:26PM (#32226677)
No, I don't really agree. I don't like async because of something you accused conditions and restarts of: hidden control flow.

In your example, I don't see any, which is great. But async can have hidden control flow. (I wonder why Andrew Kelley added it when he is so against even hidden function calls.)

On top of that, I don't think async is a good paradigm, but that's neither here or there to the conversation.

Re:Some Thoughts on Zig by Schroedingersat on Monday July 25, 2022 @08:02PM (#32232603)
That's the entire point I'm making though...exception style semantics are the worst way of handling errors. Algebraic effects make it a little better. Error values are much better as a default paradigm.

Ie. Zig's error system which is not based on algebraic effects is best in class.

Async does carry that same cost, but it comes with capabilities you can't get otherwise and is optional. Any objections to it are a demonstration of my point, not a disagreement with it.

Re:Some Thoughts on Zig by ghoward on Monday July 25, 2022 @10:49PM (#32233695)
Error values suffer from the pit of failure. If you forget to handle them correctly, you have a bug. See [1].

Exceptions are better than async because they do not "reify" stack frames, an incredibly terrible idea. I mean, it works, but it's not a good model, mentally or otherwise.

[1]: http://joeduffyblog.com/2016/02/07/the-error-model/

Re:Some Thoughts on Zig by Schroedingersat on Monday July 25, 2022 @11:54PM (#32234064)
> Error values suffer from the pit of failure. If you forget to handle them correctly, you have a bug. See [1].

Then you put your value behind the error in the type...like rust or zig where you cannot ignore them.

You keep describing the upsides of Zig's error system as reasons why something else is better.

Re:Some Thoughts on Zig by pjmlp on Friday July 22, 2022 @09:18AM (#32191590)
Congratulations for being part of the minority that cares,

On my experience, only small teams with top devs do happen to care for doing the security work you describe, as tends to be proven by surveys regarding the use of unit testing or static analysis tooling,

https://www.jetbrains.com/lp/devecosystem-2021/c/

https://www.jetbrains.com/lp/devecosystem-2021/cpp/

So yeah, pity that memory-safe languages have to fight back to regain what was already possible before UNIX took over the computing world and brought C for the ride.

This is why I would like that Zig could offer more than Modula-2 already did in 1978, in regards to memory corruption.

Re:Some Thoughts on Zig by ghoward on Friday July 22, 2022 @08:42PM (#32198972)
Yeah, I agree, and I like C!

I think the lack of bounds checks where they are not a problem is a massive design flaw nowadays. Zig should have made bounds checks universal and spent effort on compiler passes to elide them.

Allowing people to remove bounds checks is not smart (though Rust's unsafe is a good compromise), especially in such a coarse way as a build option.

In fact, a supposedly memory-safe language, Julia, has memory bugs because it allows users to turn bounds checks off. [1] This should never have happened.

[1]: https://yuri.is/not-julia/