I've seen exactly one Rust type which is actually a union, and it's a pretty good justification for the existence of this feature, but one isn't really enough. That type is MaybeUninit<T> which is a union of a T and the empty tuple. Very, very, clever and valuable, but I didn't run into any similarly good uses outside that.
Sure, but the comparable Rust feature is enum, not union.
That is essentially the motivation, primarily in the context of FFI where matching C's union behaviour using transmute is tricky and error-prone.
Also you can manually tag them and get s.th. more like other high level languages. It will just look ugly.
If you want disjoint types, something like Pascal's discriminated variants or Rust's enums is the way to go. It's embarrassing that C never had this.
Many bad design decision in C come from the fact that originally, separate compilation was really dumb, so the compiler would fit in small machines.
ColdString is easier to explain, the whole trick here is the "Maybe this isn't a pointer?" trick, ColdString might be a single raw pointer onto your heap with the rest of the data structure at the far end of the pointer, this case is expensive because nothing about the text lives inline, but... the other case is that your entire text was hidden in the pointer, on modern hardware that's 8 bytes of text, at no overhead, awesome.
CompactString is more like a drop-in replacement, it's much bigger, the same size as String, so 24 bytes on modern hardware, but that's all SSO, so text like "This will all fit nicely" fits inline, yet the out-of-line case has the usual affordances such as capacity and length in the data structure. This isn't doing the "Maybe this isn't a pointer?" trick but is instead relying on knowing that the last byte of a UTF-8 string can't have certain values by definition.
All ColdStrings which look like 8 bytes of UTF-8 text really are 8 bytes of UTF-8 text, just the type label on those 8 bytes isn't "[u8; 8]" an array of 8 bytes but instead "mut *u8" a raw pointer. "Validate" for example is 8 bytes of ASCII, thus UTF-8, and Rust is OK with us just saying we want a pointer on a 64-bit machine with those bytes. It's not a valid pointer, but it is a pointer and Rust is OK with that, we just need to be careful never to [unsafely] dereference the pointer because it's invalid
OK, so there are two cases left: First, what if there are fewer bytes of text? Zero even?
Since there are fewer than 8 bytes of text we can use the whole first byte to signal how many of the remainder are text, we use the UTF-8 over-long prefix indicator in which the top five bits of the byte are all set, bytes 0xF8 through 0xFF for this, there are eight of these bytes corresponding to our 8 lengths 0 through 7 inclusive. Because it's over-long this indicator isn't itself a valid UTF-8 prefix. Again we can pretend this is a pointer while knowing it's invalid.
Lastly, the seemingly trickiest problem, what if the string didn't fit inline? We use a heap allocation to store the text prefixed by a variable size integer length and we insist this allocation is aligned to 4 bytes. This means a valid pointer to our allocation has zeroes for the bottom two bits, then we rotate that pointer so those bottom two bits are at the top of the first byte position (depending on machine word layout) and we set the top bit. This is now always invalid UTF-8 because it has the continuation marker - the top bit is set but the next is not, which cannot happen in the first byte of any UTF-8 text, and so our code can detect this and reverse the transformation to get back a valid pointer using the strict provenance APIs if this marker is present.
This type is tomtomwombat's idea, credit to them:
As for C, it is a sharp blade on its own.
I can't even think of any prominent C FFI problems where I'd reach for the union's C representation. Too many languages can't handle that so it seems less useful at an FFI edge.
One of the reasons .NET had Managed C++, replaced by C++/CLI (nowadays C++20 compliant, minus modules), is exactly that P/Invoke (and RCW/CCW) cannot represent everything.
Which they don't want to expose on .NET type system directly.
If you're interested, i can point you to specs i'm writing that address the area you care about :)
Again, very rough. We go round and round on things. Loving to decompose, debate and determine how we want to tackle all these large and interesting areas :)
I admit that I haven’t actually used C# in 20 years or so.
Just wanted to add just another opinion on a few things.
I think many people already mentioned it, but I also don't feel to good about non-boxed unions not being the default. I'd personally like the path of least resistance to lead to not boxing. Having to opt-in like the current preview shows it looks like a PITA that I'd quickly become tired of.
Also, ad-hoc union types could be really nice. At least in Python those are really nice, stuff like `def foo(x: str | int)` is just very nice. If I had to first give this union type a name I'd enjoy it way less.
But I'm aware that you are trying your best to find a good trade-off and I'm sure I don't know all the implications of the things I wish you'd do. But I just wanted to mention those to have another data point you can weigh into your decision process.
My belief is that we will have a `union struct` just like we have `record` and `record struct`. Where you can simply say you want value-type, non-boxing, behavior by adding the `struct` keyword. This feels very nice to me, and in line with how we've treated other similar types. You'll only have to state this on the decl point, so for each union type it's one and done.
The problem is that the only safe way for the compiler to generate non-boxed unions would require non-overlapping fields for most value types.
Specifically the CLR has a hard rule that it must know with certainty where all managed pointers are at all times, so that the GC can update them if it moves the referenced object. This means you can only overlap value types if the locations of all managed pointers line up perfectly. So sure, you can safely overlap "unmanaged" structs (those that recursively don't contain any managed pointers), but even for those, you need to know the size of the largest one.
The big problem with the compiler doing any attempt to overlap value types is that if the value types as defined at compile time may not match the definitions at runtime, especially for types defined in another assembly. A new library version can add more fields. This may mean one unmanaged struct has become too big to fit in the field, or that two types that were previously overlap compatible are not anymore.
Making the C# compiler jump though a bunch of hoops to try to determine if overlapping is safe and even then leaving room for an updated library at runtime to crash the whole things means that the compiler will probably never even try. I guess the primitive numeric types could be special cased, as their size is known and will never change.
I suppose an LLM could do a pretty good job at this.
An example that may show the difference: if you have a language with nullable types, then you basically have a language with union types like String|Null, where the Null type has a single value called `null` and String can not be null.
Now if you pass this around a function that itself may return `null`, then your type coalesces to String|Null still (you still get a nullable string, there is no doubly nullable). This is not true for Maybe/Option whatever you call types, where Some(None) (or Optional.of(Optional.empty())) is different from None only.
Rich Hickey once made a case that sort of became controversial in some FP circles, that the former can sometimes be preferred (e.g. at public API surfaces), as in for a parameter you take a non-nullable String but for returns you return a String|Null. In this case you can have an API-compatible change widening the input parameters' type, or restricting the return type - meanwhile with sum types you would have to do a refactor because Maybe String is not API compatible with String.
This is a really good point. I'd love to be able to have a sum type of two strings ("escaped" and "unescaped"); or any two kinds of the same type really, to model two kinds of the same type where one has already passed some sort of validation and the other one hasn't.
Edit to add: I figure what I want is for enums to be extended such that different branches are able to carry different properties.
Edit again (I should learn to think things through before posting. sorry): I suppose it can be faked using a union of different wrapper types, and in fact it might be the best way to do it so then methods can take just one of the types in arguments and maybe even provide different overloads.
- enum: essentially just a typed uint tag collection
- union: the plain data structure that can contain any of several types
- tagged union: combines enums and unions, so you can dispatch on its tag to get one of the union types
Read from this section and they appear in order:
Other languages have unions with named choices, where each name selects a type and the types are not necessarily all different. Rust, Haskell, Lean4, and even plain C unions are in this category (although plain C unions are not discriminated at all, so they’re not nearly as convenient).
I personally much prefer the latter design.