News for hackers. Stuff that matters.
Re:Switching from C++ to Rust by burntsushi on Monday March 13, 2023 @07:52PM (#35145053)
Nope, not unions. Sum types. Sum types would be more analogous to tagged unions or discriminated unions.

> i have almost never needed to use such types

Well sure. When is an abstraction "needed"? Before Fortran, nobody "needed" a programming language either. So is a programming language necessary?

That's the funny thing about the word "need." It has very narrow application, and it's precisely why I didn't mention the word a single time in my top level comment.

Re:Switching from C++ to Rust by zabzonk on Monday March 13, 2023 @08:01PM (#35145166)
> Nope, not unions. Sum types. Sum types would be more analogous to tagged unions or discriminated unions

so, yes, unions.

Re:Switching from C++ to Rust by cornstalks on Monday March 13, 2023 @08:09PM (#35145254)
The union part is just an (obvious) optimization. An implementation detail. Sum types are not required to use a union; they can be implemented without using a union.
Re:Switching from C++ to Rust by zabzonk on Monday March 13, 2023 @08:15PM (#35145313)
how, exactly?
Re:Switching from C++ to Rust by cornstalks on Monday March 13, 2023 @08:18PM (#35145349)
By using a struct with lazily initialized members, for example.
Re:Switching from C++ to Rust by zabzonk on Monday March 13, 2023 @08:34PM (#35145510)
if that is your example, provide some code.
Re:Switching from C++ to Rust by cornstalks on Monday March 13, 2023 @09:07PM (#35145891)
Pretend we have the following sum type:

    enum SumType {
        Foo(foo),
        Bar(bar),
        Baz(baz),
    }

    void DoSomething(SumType sum_type) {
        match sum_type {
            Foo(foo): foo.do_x();
            Bar(bar): bar.do_y();
            Baz(baz): baz.do_z();
        }
    }
This could be lowered to:

    struct LoweredSumType {
        optional<Foo> foo;
        optional<Bar> bar;
        optional<Baz> baz;
    }

    void DoSomething(LoweredSumType sum_type) {
        if (sum_type.foo.has_value()) {
            sum_type.foo->do_x();
        } else if (sum_type.bar.has_value()) {
            sum_type.bar->do_y();
        } else {
            sum_type.baz->do_z();
        }
    }
This isn't real code; it's a pretend language that's kind of a mix of Rust and C++. But it illustrates that the sum type can be lowered into a non-union representation.
Re:Switching from C++ to Rust by zabzonk on Monday March 13, 2023 @09:33PM (#35146116)
> Pretend we have the following sum type:

which looks remarkably like a union to me, if i understand your made-up language.

Re:Switching from C++ to Rust by cdcarter on Monday March 13, 2023 @10:56PM (#35146806)
It's not a union, though, it stores references to three memory locations, not just one (as a union represents a single memory location which can have multiple possible interpretations). And the lowered code has nothing protecting it from an invariant where more than one reference is non-null!
Re:Switching from C++ to Rust by the-smug-one on Monday March 13, 2023 @08:52PM (#35145712)
Why does the members be lazily init:ed?
Re:Switching from C++ to Rust by cornstalks on Monday March 13, 2023 @09:00PM (#35145817)
If the data types have constructors and/or destructors that have side effects then it has to be lazy. Otherwise IMO it's not really behaving like a sum type.
Re:Switching from C++ to Rust by the-smug-one on Monday March 13, 2023 @08:50PM (#35145697)
struct List {}; struct Cons : public List { Cons(int v, List* nxt); int v; List* nxt; }; struct Nil : public List {};

This is how you'd do it in Java.

Re:Switching from C++ to Rust by zabzonk on Monday March 13, 2023 @08:58PM (#35145792)
and you would use it ... how?
Re:Switching from C++ to Rust by dcow on Tuesday March 14, 2023 @01:33AM (#35147773)
It's not exactly Java, but:

    Cons(1, Cons(2, Cons(3, Nil)))
Re:Switching from C++ to Rust by kaba0 on Tuesday March 14, 2023 @05:42AM (#35149156)
Actually, Java does have sum types nowadays, and you can write a recursive list implementation like “new Cons(2, new Cons(3, Nil.nil()))”.

Here is a definition for a Maybe type, converting it to a List is left as an exercise to the reader :) : https://news.ycombinator.com/item?id=35133670

Re:Switching from C++ to Rust by dcow on Tuesday March 14, 2023 @11:08AM (#35152708)
I just meant because I was eliding the `new`s.
Re:Switching from C++ to Rust by the-smug-one on Tuesday March 14, 2023 @04:55AM (#35148859)
I've showed you an alternative encoding, maybe get to work and learn something yourself?
Re:Switching from C++ to Rust by burntsushi on Monday March 13, 2023 @09:22PM (#35146026)
Based on your other replies in this thread, as far as I can tell, you're not here to try and understand something. You're here to fuck around and play games with people over definitions. So, I'm not responding to you. I'm responding to the people following along who might get confused by the mess you're making here. More to the point, the Wikipedia article for "sum type" redirects to "tagged union," which just honestly makes this more of a mess.

A tagged union is a specific kind of data structure. It essentially refers to a pair of information: the first is a tag and the second is the union. The union might represent many different kinds of values. The important bit is that the size of that memory is the size of the largest value. The tag then tells you what kind of value is in that memory, and crucially, tells you how to interpret that memory. Getting this wrong, for example, in a language like C can lead to undefined behavior.

Tagged unions show up in all sorts of places. The classical example is an AST. You usually have one "node" type that can be one of many possible things: a literal, a binary operation, a function call, a type definition, whatever. And for any non-primitive node, it usually has pointers to children. And what is the type of a child? Well, a "node"! It's a recursive data type. It lets you represent children as just more nodes, and those nodes might be any other AST node. (And as you might imagine, an AST can be quite a bit more complicated than this. You often want restrictions on which types of children are allowed in certain contexts, and so you wind up with many different tagged unions.)

The use of a tagged union like this can be conceptually thought of as a sum type. A sum type is not a data structure. A sum type is a concept. It comes from type theory and it is the "dual" of product types. Product types represent the conjunction of values while sum types represent the disjunction of values. In type theory notation, you usually see 'A x B x C' to refer to the product type of A, B and C. Similarly, you usually see 'A + B + C' to refer to the sum type of A, B or C.

A sum type can be represented by just about anything, because it's just a concept. You can take something that is typically used for product types and use it to present an API that gives you a sum type:

    type OptionInt struct {
        exists bool // private
        number int // private
    }
    
    func OptionIntNone() OptionInt {
        return OptionInt{false, 0}
    }
    
    func OptionIntSome(number int) OptionInt {
        return OptionInt{true, number}
    }
    
    func (o OptionInt) IsSome() bool {
        return o.exists
    }
    
    func (o OptionInt) Number() int {
        if !o.exists {
            panic("option is none, no number available")
        }
        return o.number
    }
This is a very contrived and hamstrung example, because it really doesn't give you much. But what it does give you is the API of a sum type. It is a value that can be either missing or present with an integer. Notably, it does not rely on any particular integer sentinel to indicate whether the value is missing or not. It encodes that state separately. So it can fully represent the presence of any integer value.

As you can see, despite it offering an API that represents the sum type concept, it is actually implemented in terms of a product type, or a struct in this case.

In a language like Go, which does not really support either unions or sum types, this sort of API is really hamfisted. In particular, one could object that it is a sum type because it doesn't really represent the "sum typeness" in the type system. Namely, if you get its use wrong, then you don't get a compilation error. That is, you can call the 'Number()' method without checking whether 'IsSome()' returns true or not. In other words, correct use of OptionInt looks like this:

    var opt OptionInt := doSomethingThatMightReturnNone()
    if opt.IsSome() {
        // OK, safe to access Number. It won't panic
        // because we just checked that IsSome() is true.
        fmt.Println(opt.Number())
    }
But nothing in Go stops you from writing this instead:

    var opt OptionInt := doSomethingThatMightReturnNone()
    fmt.Println(opt.Number())
This is why "support for sum types" is critical for making them as useful as they can be. In Rust, that same option type is defined like this:

    enum OptionInt {
        None,
        Some(i32),
    }
That's it. The language gives you the rest. And crucially, instead of splitting out the 'IsSome()' check and the access via 'Number()', they are coupled together to make it impossible to get wrong. So the above code for getting a number out looks like this instead:

    let opt: OptionInt = do_something_that_might_return_none()
    if let OptionInt::Some(number) = opt {
        println!("{}", number);
    }
You can't get at the 'number' without doing the pattern matching. The language won't let you. Now, you can define methods on your sum types that behave like Go's 'Number()' method above and panic when the value isn't what you expect. And indeed, Rust's Option and Result types provide this routine under the names 'unwrap()' and 'expect()'. But the point here isn't just about Option and Result, it's about sum types in general. And you don't have to define those methods that might panic. You can just let the language help guide you along instead.

Sum types are an idea. A concept. A technique. Tagged unions are an implementation strategy.

Notice also that I haven't mentioned exhaustiveness at all here. It isn't required for sum types. Haskell, for example, doesn't mind at all if your pattern matching isn't exhaustive. You actually have to tell it to warn you about it, and even then, it's not a compilation error unless you treat it as one. In Rust, exhaustiveness checking is enabled by default for all sum types. But you can disable it for a particular sum type with the '#[non_exhaustive]' attribute. The benefit of disabling it is that adding a new variant is possibly semver compatible change, where as adding a new variant in an exhaustive sum type is definitely a semver incompatible change.

Re:Switching from C++ to Rust by signa11 on Monday March 13, 2023 @10:59PM (#35146822)
this is just ... beautiful ! thank you !
Re:Switching from C++ to Rust by youainti on Monday March 13, 2023 @11:17PM (#35146962)
Thank you for such a great explanation. I found it to be really useful.
Re:Switching from C++ to Rust by kd5bjo on Tuesday March 14, 2023 @02:28AM (#35148057)
> But you can disable it for a particular sum type with the '#[non_exhaustive]' attribute.

I’m not sure this is the best description of what `#[non_exhaustive]` does in Rust. It doesn’t so much disable exhaustiveness checking as it marks the given list of variants as incomplete.

In Rust, some pattern matching contexts are required to be exhaustive (e.g. ‘let pattern = …;’, ‘match … {}’), and others are not (e.g. ‘if let … = … {}’). When an enum is tagged as ‘#[non_exhaustive]’, the former are required to include a wildcard matcher that will accept unknown variants, even if all known variants are explicitly matched.

Rust also allows structs to be `#[non_exhaustive]` to mark that the listed fields might not be sufficient. This disables the ability to (outside of a privacy boundary) directly construct an instance of the structure, or to destructure it with an exhaustive pattern.

Re:Switching from C++ to Rust by tialaramex on Tuesday March 14, 2023 @06:17AM (#35149381)
The privacy boundary applies in both cases by the way. Despite labelling your AmericanCity enum as #[non_exhaustive] since it only has Houston and Chicago in it so far, you are allowed in your own code inside the boundary to write a match which handles only Houston and Chicago. And that's actually probably a good idea, because when you add SanFrancisco to the enum, the compiler will point out anywhere your match clauses don't handle it, whereas if you had a default case (as your 3rd parties will) you'd drop into the default.
Re:Switching from C++ to Rust by burntsushi on Tuesday March 14, 2023 @08:19AM (#35150284)
My goal wasn't to describe it thoroughly, it was to call it out as a means of disabling exhaustiveness checking. Regardless of whether that's really what the essence of '#[non_exhaustive]' actually is, it does disable exhaustiveness checking. It's right in the name. It's an important part of what it does.

Basically, we're talking past each other. I called it out for what it does, but you're pointing out why someone would want to use it. That is, "#[non_exhaustive] on enums indicates that the list of variants is incomplete, and it does this by disabling exhaustiveness checking outside of the crate in which it is defined." Although, of course, not even that is precisely correct. Because your match arms are still forced to be technically exhaustive via the compiler requiring a wildcard arm. But the end effect to the user of the enum is that exhaustiveness checking on the variants of the enum is disabled.

Re:Switching from C++ to Rust by kaba0 on Tuesday March 14, 2023 @05:39AM (#35149136)
Thanks for the writeup!

Slightly unrelated, but re

> More to the point, the Wikipedia article for "sum type" redirects to "tagged union," which just honestly makes this more of a mess

I often see Wikipedia articles in CS topics severely lacking, or claiming exact definitions, when no such thing exists. Many definitions we regularly use are not exact at all, different papers/books use them differently, e.g. how would you even define OOP, low-level languages, etc? I would prefer them to use less strong language, as too many people argue on the internet based on Wikipedia definitions.

Re:Switching from C++ to Rust by burntsushi on Tuesday March 14, 2023 @08:13AM (#35150235)
I agree that they are lacking. In theory, I have the disposition to improve articles like that, but in practice, whenever I've tried, I get shut down by editors. It's been a long time since I've tried though.

It happens on other wikis too, like the Arch wiki.

I realize it's a tired complaint that a lot of people have and editors don't have an easy job, but I am so completely done with contributing to wikis. I know a lot of other people with the same complaint and same conclusion, including experts in the domain of type theory. One wonders whether this is one of the reasons why the articles are not as good as they could be.

Re:Switching from C++ to Rust by noobermin on Monday March 13, 2023 @10:38PM (#35146663)
Can you explain the utility of them over unions then? Asking sincerely.
Re:Switching from C++ to Rust by burntsushi on Monday March 13, 2023 @10:49PM (#35146749)
Unfortunately it's actually kind of difficult to parse your question because of the ambiguity in the surrounding comments. You could mean "tagged union" instead of "union," or maybe not and "them" means "tagged union"...

If you're asking to compare tagged unions and unions, then...

A union is not a tagged union. A union is part of a tagged union. A union on its own is just a region of memory. What's in that memory? I dunno. Who does? Maybe something about your program knows what it is. Maybe not. But what if you need to know what it is and some other aspect of your program doesn't tell you what it is? Well, you instead put a little bit of memory next to your union. Perhaps it an integer. 0 means the memory is a 32-bit signed integer. 1 means it's a NUL terminated string. 2 means it's a 'struct dirent'. The point is, that integer is a tag. The combination of a union and a tag is a tagged union.

An abstract syntax tree is a classic example of a tagged union.

If instead you're asking to compare tagged unions and sum types, then...

Tagged unions are one particularly popular implementation choice of a sum type. Usually sum types have additional stuff layered on top of them that a simple tagged union does not have. For example, pattern matching and exhaustiveness checking.

Re:Switching from C++ to Rust by noobermin on Monday March 13, 2023 @11:01PM (#35146844)
Sorry, yes I meant tagged union, a C style union (as in the language's concept) is so useless by itself that I thought being tagged was an obvious given.

In that case, the pattern matching and exhaustiveness means that the value the sumtypes in rust offer is in combination with those other language features, not the mere usage of the sumtypes themselves then, I think that makes more sense.

Re:Switching from C++ to Rust by burntsushi on Monday March 13, 2023 @11:11PM (#35146921)
Yes. When you say "sum type" and you're talking about, say, Ocaml or Standard ML or (mostly) Haskell or Rust, then what comes with that is also the infrastructure around sum types in those languages. They aren't really separable. That includes pattern matching and exhaustiveness checking (the latter is an opt-in warning in Haskell, and can be opted out of in Rust on a per-type basis).

The discussion around "sum type" the concept is to disentangle it from the implementation strategy. Tagged unions are the implementation strategy, and they don't come with pattern matching or exhaustiveness checks.

Of course, many of these terms are used interchangeably in common vernacular. But if you look at the comment that kicked off this annoying sub-thread:

> so, um, unions? i have to say that in many years of programming, i have almost never needed to use such types.

Then this is clearly trying to draw an equivalence between two different terms, but where no such equivalence exists. There's no acknowledgment here of the difference between what a "union" is (nevermind a tagged union) and the "sum types" being talked about in my original top level comment. And the last part, "i have almost never needed to use such types" is puzzling on multiple levels.