News for hackers. Stuff that matters.
Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by Arnavion on Thursday July 15, 2021 @07:07PM (#27851150)
Having used defer in golang, I'm firmly in the camp that scope-based cleanup like defer is bad and value-based cleanup like destructors is better.

Say you have this (pseudocode):

    function foo() {
        let bar = make_bar();
        defer cleanup_bar(bar);
        let baz = make_baz();
        defer cleanup_baz(baz);
        let quux = make_quux();
        defer cleanup_quux(quux);

        do_something_with(bar, baz, quux);
    }
Now imagine you have multiple functions like this that all create bar, baz and quux's. Say these are unit tests and the bar-baz-quux are mocks or whatever.

So you decide to DRY by moving the creation to a common function.

    function create_mocks() -> (Bar, Baz, Quux) {
        let bar = make_bar();
        defer cleanup_bar(bar);
        let baz = make_baz();
        defer cleanup_baz(baz);
        let quux = make_quux();
        defer cleanup_quux(quux);

        (bar, baz, quux)
    }
... except this doesn't work any more, because the cleanup happens within create_mocks and create_mocks ends up returning cleaned-up values.

You could fix this by switching to a callback approach:

    function run_with_mocks(f: function(Bar, Baz, Quux)) {
        let bar = make_bar();
        defer cleanup_bar(bar);
        let baz = make_baz();
        defer cleanup_baz(baz);
        let quux = make_quux();
        defer cleanup_quux(quux);

        f(bar, baz, quux)
    }
... but now you have to make the caller use a callback that necessarily returns void and not any other type. (This is mostly a golang problem, since languages with generics can be generic on the return type of the callback.) It also means you can't easily do things like set variables in an outer scope easily, depending on how the language deals with closure captures.

Worse, if not all tests use the same mocks, you actually end up with multiple of these functions:

    function with_bar(f: function(Bar)) {
        let bar = make_bar();
        defer cleanup_bar(bar);

        f(bar)
    }

    ...

    function foo() {
        with_bar(bar => {
            with_baz(baz => {
                with_quux(quux => {
                    do_something_with(bar, baz, quux);
                });
            });
        });
    }
All these problems are avoided by having destructors.
Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by axaxs on Thursday July 15, 2021 @11:00PM (#27852630)
I'm not arguing against destructors, but this example is ultra contrived. Why would you defer cleaning up objects you are returning? That's no longer your responsibility.

Instead you'd return an object, and make it the callers responsibility to close it. Destruct all three in the close call.

It may not fit your mental model, and that's fine. But let's not pretend it's a serious hurdle.

Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by fauigerzigerk on Friday July 16, 2021 @05:36AM (#27854524)
>It may not fit your mental model...

No, it's not a psychological problem. This is real complexity, and it's one of the most serious and widespread issues in software development.

The key issue is that we often have rules that say: Whenever A happens then B must happen as well. It is absolutely critical that rules like this are expressed in exactly one place and that enforcement is opt-out not opt-in.

Everything else is error prone and creates significant mental burden.

Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by dnautics on Thursday July 15, 2021 @07:21PM (#27851240)
This is not really a problem in zig. If you really need a destructor, write a .deinit() function in your struct, and call that. For your case with mocks (note mock as a noun, not a verb, which is a bit of an antipattern anyways, see: http://blog.plataformatec.com.br/2015/10/mocks-and-explicit-...). That aside:

Go doesn't have generics so you can't really do this, but in the case of zig, your function would be parametrized on the type, so you would issue it a struct to be shared by the tests that has the common initialization upfront, make_ functions that return the pre-initialized stuff, followed by running the function, followed by no-op deinits, followed by whatever forensics you'd like to run later.

Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by Arnavion on Thursday July 15, 2021 @07:30PM (#27851316)
>This is not really a problem in zig. If you really need a destructor, write a .deinit() function in your struct, and call that.

This is missing the point. The point of a destructor is that it runs automatically. .deinit() is the same as the three cleanup_*()s in my example and thus has the same problems.

>but in the case of zig, your function would be parametrized on the type, so you would issue it a struct to be shared by the tests that has the common initialization upfront

The third example in my post shows why this is inadequate.

Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by dnautics on Thursday July 15, 2021 @09:26PM (#27852129)
Then im not really sure what your point is, because I've written zig code that reuses code in a similar way to what you are talking about, with three different forms of parallelism calling into some common runner code each of which has different needs for resource cleanup times (single threaded, cooperative, and independent parallelism). It wasn't a problem.

Have you tried doing what you propose in zig? Ultimately, the only difference between a .deinit() and a destructor is that one is explicit and the other is not. There are cases where you can easily make a mistake for both an explicit destructor and an implicit destructor.

Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by AndyKelley on Thursday July 15, 2021 @07:33PM (#27851334)
> except this doesn't work any more

it does in zig:

    fn create_mocks() !struct {bar: Bar, baz: Baz, quux: Quux} {
        var bar = try make_bar();
        errdefer cleanup_bar(bar);
        var baz = try make_baz();
        errdefer cleanup_baz(baz);
        var quux = try make_quux();
        errdefer cleanup_quux(quux);

        return .{
            .bar = bar,
            .baz = quux,
            .quux = quux,
        };
    }
Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by Arnavion on Thursday July 15, 2021 @07:36PM (#27851364)
You've misunderstood. The thing that's being tested is not make_*, it's do_something_with()
Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by dnautics on Friday July 16, 2021 @12:25AM (#27853042)
You didn't read the function signature.
Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by Arnavion on Friday July 16, 2021 @04:15AM (#27854029)
I did.
Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by lhorie on Thursday July 15, 2021 @07:31PM (#27851324)
But that's because you're doing continuation passing style. You could do this instead:

    func make_foo() *Foo {
      return &Foo {
        bar: make_bar(),
        baz: make_baz(),
        quux: make_quux(),
      }
    }
    func cleanup_foo(foo *Foo) {
      cleanup_bar(foo.bar)
      cleanup_baz(foo.baz)
      cleanup_quux(foo.quux)
    }

    // elsewhere
    foo := make_foo()
    defer cleanup_foo(foo)
    // [rest of logic goes here]
Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by Arnavion on Thursday July 15, 2021 @07:38PM (#27851376)
Please see the third example I gave, which talks about the set of mocks not being the same for every test.
Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by lhorie on Thursday July 15, 2021 @07:50PM (#27851499)
There are several options

Poor man's:

    func make_foo_with_mock_bar() *Foo {
      return &Foo {
        bar: make_mock_bar(),
        baz: make_baz(),
        quux: make_quux(),
      }
    }
Explicit injection:

    func make_foo(bar: Bar, baz: Baz, quux: Quux) *Foo {
      return &Foo {
        bar: bar,
        baz: baz,
        quux: quux,
      }
    }
Or use some DI framework

Even if you have reservations about where verbosity lies, I don't see how this affects the cleanup side.

Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by Arnavion on Thursday July 15, 2021 @07:52PM (#27851517)
You've just restated your original comment, without addressing the fact that this does not work with the third example in my original comment.
Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by lhorie on Thursday July 15, 2021 @07:59PM (#27851583)
How so? You can mock any of bar/baz/quux however you want using the patterns/techniques above. Am I misunderstanding something?
Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by Arnavion on Thursday July 15, 2021 @08:03PM (#27851621)
Again, your Foo always contains a bar, baz and a quux. And again, the third example in my original post talks about how not every test needs all three.
Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by lhorie on Thursday July 15, 2021 @10:02PM (#27852327)
Ah, I understand now. My understanding is that with zig and go, one does not DRY deferring, it's just a part of the idiom trade-offs between verbosity and explicitness.

So if bar, baz, quux aren't intrinsically related as a unit and might be mixed and matched arbitrarily, you just forego the whole ceremony and call each thing or the respective mock individually (the argument being that if you're making multiple different permutations, you're not repeating yourself per se; e.g. mocks don't need to be cleaned up in the first place)

There's also something to be said about tests needing several arbitrary mocks. Normally, I'd consider that a smell. I've actually seen the pattern of making a kitchen sink foo for scenarios like that, but with nils/noops for unused things (can't say I like that approach though).

Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by dnautics on Friday July 16, 2021 @12:21AM (#27853032)
It's a huge code smell. It's funny because just today I was listening to a podcast about testing and they talk about exactly these sorts of contrivances as being a good sign you should not be mocking something in a test.

(https://smartlogic.io/podcast/elixir-wizards/s6e6-matthias-l...) if you're curious

Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by AlexCoventry on Thursday July 15, 2021 @07:53PM (#27851521)
I thought you were looking for ways to DRY up creation of a set of mocks?
Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by Arnavion on Thursday July 15, 2021 @07:57PM (#27851564)
I am.

With the three mocks I gave, there are 2^3 - 1 = 7 possibilities of which mocks a particular test needs. Of these, three are degenerate cases of a single mock each which would be served by the individual make_* already, so that leaves four cases of make_foo* that you'd need to implement.

With more than three mocks, this number grows rapidly.

Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by AlexCoventry on Thursday July 15, 2021 @07:46PM (#27851463)
The common approach in go is something like

    func newMocks(
    ) (
      foo Foo, bar Bar, baz Baz, 
      cleanup func(),
    )
and making the caller responsible for calling the cleanup function, possibly in a defer. To me, this seems like a good trade off between making a little more work for the caller and keeping things explicit and under the caller's control.
Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by Arnavion on Thursday July 15, 2021 @07:51PM (#27851509)
This doesn't work with the third example, ie where the set of mocks is not the same for every test. This suggestion devolves to the original code that we were trying to refactor.
Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by Mawr on Friday July 16, 2021 @05:49AM (#27854584)
Then the original code is perfectly fine and does not need refactoring. You create the mocks you need with cleanup defers and you're done.

You could just make a function that creates all the mocks needed for all tests, but you don't like that. So the only option left is to specify which mocks you need for each test... which is exactly what the original code is doing.

You're running into a wall because you're trying to abstract code that's already maximally abstracted.

Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by oconnor663 on Friday July 16, 2021 @04:58AM (#27854300)
I wonder if "conditionally send a local variable over a channel" is another good example of something that's hard to clean up with defer.

One upside of destructors is that adding destructor cleanup to a type isn't necessarily a breaking change. (In Rust terms, as long as the type isn't "Copy".) But in languages with defer or with/using statements, it is.

Another upside of destructors is that the cleaned-up state can be mostly hidden from other code. But when cleanup is a regular function, it's easier to access an object after cleanup, and you have to reason more about what happens in that case. (A Rust example here might be "What happens if you use a Box after it's freed?" The answer is that you just can't do that, even though Box doesn't have a freed/null state.)

Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by j1elo on Friday July 16, 2021 @09:30AM (#27856022)
Wait, the moment a creator function returns its created values, it is also returning their ownership. So at least with that concrete DRY refactor you are showing, you're effectively not only moving code but also responsibilities around.

But I get your point: with destructors you are free to change the ownership responsibilities, because the link between a variable and its releasing is implicit and carried over by the runtime, instead of explicitly being cared for by the code.

Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by cmrdporcupine on Thursday July 15, 2021 @10:27PM (#27852478)
100%. And another problem with defer is that it only works in function scope, not block scope. This bit me in a piece of Go I had to refactor into multiple functions recently when all I wanted to do was a few lines change around a 'for' loop. The code review ended up being more substantial and it made the code less readable than just being able to sprinkle in some additional defers.

With the C++ destructor RAII pattern I can use it at any lexical scope I need it at, which is super handy.

Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by Jtsummers on Thursday July 15, 2021 @10:33PM (#27852499)
Zig's defers, at least, are block scoped. So they are executed at the end of whatever block they're created in, in contrast to Go's version of defer which is function scoped only.
Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by naikrovek on Thursday July 15, 2021 @11:47PM (#27852870)
RAII is very obviously a bad idea, and I'm shocked that more people can't see this. it's "fine" until it isn't, and when it bites you, it crushes your arm. you are unlikely to ever forget this after it happens. it's like driving a car fast while blindfolded; you will crash, and it will hurt, it's just a matter of when.
Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by pcwalton on Friday July 16, 2021 @12:07AM (#27852963)
Care to explain why? In Rust RAII works very well, and the common failure mode of RAII--use-after-free--is statically prevented by the borrow check.
Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by cmrdporcupine on Friday July 16, 2021 @10:45AM (#27856753)
Use-after-free is definitely the major footgun portion of RAII in C++ (in my experience) but seems to me like the same issue would be present in any non-GC'd language that offers a 'deferral' mechanism, asynchronous or not (unless it has ownership/borrow management like Rust as you point out).
Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by marton78 on Friday July 16, 2021 @02:06AM (#27853444)
Your claims would benefit from you substantiating them.
Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by naikrovek on Friday July 16, 2021 @03:49PM (#27860468)
until you experience the problems yourself, you won't believe me, anyway, no matter what evidence I provide.

it is way too easy to critique an approach that led to a problem that hasn't happened to you, and is described to you by someone else.

Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by kitd on Friday July 16, 2021 @02:31AM (#27853539)
You can create "block-scope" with an immediately-called function.
Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by andrenth on Thursday July 15, 2021 @07:33PM (#27851336)
I would put the mocks in a Mock struct with a Cleanup() method and have the caller call defer mocks.Cleanup()
Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by Arnavion on Thursday July 15, 2021 @07:37PM (#27851374)
Please see the third example I gave, which talks about the set of mocks not being the same for every test.
Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by axaxs on Thursday July 15, 2021 @11:58PM (#27852920)
You keep saying this, but it doesn't make a lot of sense.

Your initial pretense was that you always make 3 objects, so move it into a function.

When people show how to accomplish that, you state 'well what about my test that doesn't need all three?'

Why are you testing things your code doesn't do?

Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by Arnavion on Friday July 16, 2021 @04:30AM (#27854135)
>When people show how to accomplish that, you state 'well what about my test that doesn't need all three?'

You've phrased this as if I said this after I'd been given a solution for the initial problem. In reality, I mentioned this in the original post itself, but it seems many people didn't read it.

It's almost like I have prior real-world experience of this problem, where I already went down the path of making common functions for all the creation and cleanup respectively, only to find that it doesn't help when all the functions don't create the same subset of things.

>Why are you testing things your code doesn't do?

I have no idea how you came to this conclusion.

Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by axaxs on Friday July 16, 2021 @11:32PM (#27863381)
Hey, I didn't mean to come across as rude but when I reread it, it was. I wasn't expressing myself well, I apologize.

I've been a Go mentor for years, and see how people take concepts they are used to and try to shoehorn them in, and complain they are ugly.

In this case, when I first saw your examples, my thought was 'youre doing it wrong'. But not wrong as in incorrect, but now how the language was designed.

If you truly care we can go through scenarios and I can tell you how I'd approach it. But in my past experience, when people find a paradigm they like, they tend to not listen.

Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by ghoward on Thursday July 15, 2021 @07:43PM (#27851421)
These are great examples. I knew that I preferred destructors over defer, but I could not explain why. You put it into words.

I think people have a bias against destructors because of C++, but they really are pretty nice. Speaking as a guy that uses C instead of C++, destructors are the one thing I would add to C.

Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by Arnavion on Thursday July 15, 2021 @07:49PM (#27851489)
They certainly have their problems. They're functions that get invoked without any code to explicitly invoke them. You have to consider whether a destructor can be invoked on a partially constructed object. You have to consider whether a destructor can be invoked more than once. You have to consider providing the ability to not call the destructor for performance. etc etc

It's understandable why a language trying to be close to C in spirit does not have them.

Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by ghoward on Thursday July 15, 2021 @09:33PM (#27852181)
Yes, those are all problems with destructors, but they are well understood. However, in a language I am currently creating, calling a destructor on a partial object is impossible because the object is created in "one" step. In other words, constructors themselves don't have access to the object in a partially created state.

It does make things much easier.

Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by hnakamur on Thursday July 15, 2021 @08:13PM (#27851696)
I agree. For example, most of Rust libraries uses the drop handler (something like desctructors) to close a file descriptor. However it has problems since in async model closing is also async and it may return an error. So in tokio-uring explicit async close will be provided. https://github.com/tokio-rs/tokio-uring/pull/1/files#diff-3d...

And you can call close like: file.close().await.unwrap(); https://github.com/tokio-rs/tokio-uring/pull/17/files#diff-b...

Re:Zig, Skia, Clojure, Geometry and the Japanese TV Show: IC... by pcwalton on Friday July 16, 2021 @12:08AM (#27852967)
Note that there are ideas to fix this issue in the async foundations WG. It's arguably more of a problem with async as implemented in Rust than with the concept of RAII.