Did you, uh, do the thing I can't ask if you did on HN? Because it's entirely about how to get better performance than sentinel values by specially structuring your discriminant.
ORIGINAL:
Not sure what you mean, so I'll try again.
An Option<T> in Rust should occupy the size of T plus the size of a discriminant. On a 64-bit machine for example an Option<i32> will take up 8 bytes. Four bytes for the i32 and four bytes (including alignment) for the discriminant. A 100% increase in space is clearly a performance hit.
If, however, T is a non-nullable type, the Rust compiler is smart enough to use this null value as a sentinel and you end up with an Option<T> that only takes up the size of T.
So, for example, both a &i32 and an Option<i32> will take 8 bytes, no difference, zero waste.
for example?
It might be more instructive to list some exceptions:
- As described elsewhere in this thread, Julia is able to optimize Array<Option<X>> internally into a bitfield and a tightly-packed array of X
- Rust can use knowledge of a type's valid values to optimize Option<T> into something the same size of T for common T (pointers, arrays, etc, also notably I think user-defined enums which aren't exhaustive of their value space) - but as far as I know it cannot do what Julia does, you need to reach for e.g. https://docs.rs/vec-option/latest/vec_option/
- This is somewhat the raison d'etre for Jai, if it ever gets released.
to make it obvious, post some code.
bool m_initialized ;
storage_type m_storage ;FWIW, we have had this discussion a few months ago on HN.
(The compiler is allowed to improve this, but it won't.)
But those languages are very good at pretending they mix the two concepts.
This is about how to represent a null values in memory (for a columnar data structure), not how null values are expressed and accessed in a particular language's type system. However null values are represented in memory, you'd want an appropriate language binding/reppresentation.
And in practice, if someone has an odd sentinel value like that in the data format, library authors tend to be lazy and not take the time to do that wrapping; instead, they just expose the i32 directly, maybe Option<i32>, and sharp edges abound around the sentinel.
(That is, the data format — a sentient value — encourages buggy code. And that is the billion dollar mistake: that a nullptr pretends to be a pointer, while it is not, and gets swept up in the same operations.)
This is only true if that summation is not used or compared anywhere else which isn't usually the case. For example if in sql you are reporting the SUM(column_A)/ SUM(column_B) and one column can be null without the other one being null you're likely reporting the wrong number
But you don't have to use what the article calls a "sentinel value" (i.e., you don't have to dedicate -2489391827 as "null") to do it: simply "null" for None/Null/Nothing, and the integer for Some(<int>) / Just int would work.
It's just another variant of that: having INT32_MIN is just as bad: these things inevitably end up unrepresented in the typesystem, someone writes a naïve function over that, and suddenly your "nulls" are being summed or computed on, particularly when they're rare in the dataset, or at least in the one the coder tested with.
(Even in SQL, where it's a separate value — NULL — it's still an awfully weak type system, so sanity is by no means assured.)
Give me must-be-explicitly-handled Option<T>.
(So, I'd argue that even if there were a performance loss, in most cases, the gain in correctness is the right tradeoff.)