It is now common when creating a library to rely on manual checking and coders explicitly writing code to check and throw NullArgument exceptions for each object parameter in every public method, and then writing dozens or hundreds more pointless unit tests to ensure this null exception behaviour works as expected; this is totally unnecessary in languages that demand explicitly stated Option<T> when an Option is meaningful.
Instead of requiring all the above for safety -an approach which will notably fail to report any errors if the null argument checks are partially or completely omitted- we have -and we should prefer- languages where the compiler or interpreter will not let us pass null into a function where null makes no sense, because that fact is encoded in the function definition.
At the library boundaries, which are comparatively rare to internal code, we can simply use the explicit Option type.
I should note I have one objeciton with the F# implementation - in that I think that ideally the properties { .IsSome, .IsNone, .Value } should not exist, as they promote bad programming practices.
[1] https://fsharpforfunandprofit.com/posts/the-option-type/
For instance a `std::string` or a `double` in C++ can never be null ; a pointer to either can be but that's far from idiomatic.
My understanding so far is that option types have two advantages over the way current mainstream languages handle the null case: a) you can find the potential uses of null values statically, and also ensure that the programmer writes some code that at least nominally handles the case; b) you can avoid this burden in cases where null is not an option. Does null-as-its-own-type improve on this?
In Crystal we go a step further and allow type unions between arbitrary types (of which nil is just an ordinary empty struct in the stdlib which you can call methods on). I love how it works in practice.
(I don't like inclusive union types because they're noncompositional - code you write with them isn't parametric any more - which seems particularly awkward in a compiled language - how do you compile code that forms unions involving type parameters?)
I'm not 100% sure how you can integrate sum types with a flow-typing system, perhaps with pattern matching?
So inside the body of a generic method (or class) you can't form unions involving the method's type parameters? That works but makes them much less useful as a language feature - most language features work as normal within a generic method.
> I'm not 100% sure how you can integrate sum types with a flow-typing system, perhaps with pattern matching?
Whatever you do for union types should work, surely. Indeed it ought to be simpler since you have more information to work with - with a sum type if the thing is B then you know it's not A, whereas with a union type it's possible for the thing to be both A and B.
And regarding sum types I was just thinking syntactically instead of in terms of the type system.
Hmm. Does that mean you can't have a generic method in a separate compilation unit from where it gets used?
> I really am interested in knowing more about what you said about union types being non compositional because I'm definitely not a type theorist.
I'm not really a type theorist, I'm just thinking about e.g. if you have a method like:
def doSomething[A](userSuppliedValue: A) = {
val x = if(someCondition) Some(userSuppliedValue) else None
x match {
case Some(a) => y ...
case None => z ...
}
x match {
case None => v ...
case Some(a) => w ...
}
}
I'm confused by your example too. If you mean replacing matching None with testing for nil, then you'd have to replace Some(T) with an `else`, then the compiler absolutely can prove that if you take one branch you take the other - given that x is not assigned to (not sure why the compiler would want to prove that though). And after all that I'm not sure how your example relates to, or what you even mean by "noncompositional".
If you use an inclusive union, the type of x becomes A | nil (or however you write it). The trouble is that you don't know whether A might also be a type that includes nil, and userSuppliedValue might be nil already. So if you write in match/case style your branching behaves inconsistently in that case (because x is both an A and a nil) - and if you write in "if(x == nil)" style then you can't infer that branch y will be taken when someCondition is true.
> And after all that I'm not sure how your example relates to, or what you even mean by "noncompositional".
What I mean is that viewing these types as type-level functions, they don't compose. For option/maybe-style types, composition works as expected: you can know the behaviour of X[Y[A]] by reasoning about X and Y independently. That doesn't work for inclusive unions, because whether A | nil is a different type from A depends on what type A is.
Being able to move all your concerns into the type system is really nice, and the difference between a type system that does what you expect 95% of the time and one that does 100% of the time is huge - but I don't know any way to get there except gradually. I've been doing Scala for 8 years and it probably took 4 before I realised how important this stuff was even in the simple case of Option - and that's coming from a background of Java before then and already being dubious of things that step outside the type system.
Language inconsistencies matter, but only in large codebases, and by the time you have those large codebases it's probably too late to change the language. I think Scala gets a lot right, I think something like Idris gets more right. I can make the case for why those things are the right things in terms of theoretical purity/consistency. But when it comes to practical differences I don't know how to convince anyone other than by saying "go maintain a 300kloc system in an ML-family language for 5 years and see how much easier it is" :/.
Best of luck. I hope Crystal finds some things that work well, and that we all add something to the language design state of the art. I do think we're getting gradual progress and consensus - if nothing else, almost every new language has at least some form of sum typing and pattern matching these days - but I guess progress on the aspects that matter for big systems is inherently going to be slow, because it's only after we've built those big systems that we can see what's wrong (and what's right) with them.
And yeah, nobody has any experience maintaining 300kloc+ systems in crystal. And i'm sure we get loads of things wrong and there are lots of pain points. I hope we find some type theorists to pick through the problems when that happens too, because crystal's type system is really quite different to anything I've ever seen before. And it would be nice to see what comes out of that theory wise.
Well, the use cases will be there, and any general-purpose language will need answers for them. One can of course achieve the same effect with other language features (e.g. Java's famous use of the "visitor pattern" for those cases), but there's a cost in verbosity and maintainability.
And pattern matching isn't really needed if you have flow typing + OO (at least I haven't found a usecase for pattern matching which crystal doesn't already have an answer to)
Sum types usually only have two or three entries too.
> And pattern matching isn't really needed if you have flow typing + OO (at least I haven't found a usecase for pattern matching which crystal doesn't already have an answer to)
How does "flow typing + OO" solve the problem? You can use OO to emulate pattern matching via the visitor pattern, yes, but that's notoriously cumbersome; I don't see how flow typing makes any difference to that?
Crystal has method overloading (dynamic dispatch), which is a form of pattern matching (but not destructuring) but that's it. So I guess we have pattern matching but definitely not to the extent say haskell or elixir has it.
Visitor pattern in Crystal is just a class with a bunch of `def visit(node : Type)` and then just call `visit(foo)` and let dynamic dispatch handle the rest. The visited data structure has no knowledge about the visitor.
Hmm, I guess flow typing lets you do an exhaustiveness check of an if-else style set of cases. Still seems like a syntactic shortcut would be nice, just as one misses "switch" in Python.
I'd take it a step further, and posit that The Fine Article's rage comes from its author not grokking the distinction, or even recognizing there is one.
Of course null is going to seem frustratingly and dangerously wrong if that's your concept of it.
PS: To disembiguate I was speaking of the null a value in a nullable type. Maybe I got you wrong.
Also I guess Null is way less a problem in strictly typed language than in JavaScript for instance.
Null values and null pointers aren't fundamentally the same thing, though some programming languages may be implemented in a way such that use of one exposed the other. A language can have a Null value that never results in NPEs.
The problem with nulls isn't strictly null references, in the sense of "a pointer that points to nothing" or "a pointer that throws a null pointer exception", it's that languages with nulls implicitly make null an element of every type. They tend to do this because variables are allowed to have null references, but if you changed the semantics of variables to technically forbid null references, but still had a null value as an implicit element of every type, you'd still have the problem of needing to do runtime checks for null everywhere.
sum = 0
for entry in entries:
case (val):
null: pass
number n: sum += n
endcase
The fun part is that the typesystem enforces that you check all possible cases -- or only lets you explicitly un wrap potentially null values.
So it's not like a C programmer checking if (x == NULL) manually.
Alternatively, you can only use languages which offer actual tagged unions, and avoid the ridiculousness of these 'hasvalue' fields.
I tried to address explicit-null-handling programming concepts such as Option<T> or tagged unions in my last paragraph.
Nevertheless, it didn't seem like the essay was about non-nullable concepts but instead, he was writing about the idea of "null" as a data representation for unknown values to be fundamentally flawed because of various boolean operations strangeness.
My point is that "null" as a data representation mapping to "missing" is an irreducible complexity which is orthogonal to Option<T>. E.g. F# Option<T> still has ".None" which is a "null by another name". Yes, the programming language support is superior because it forces the programmer to handle possible nulls instead of crashing. Safety around nulls enforced by the compiler is progress. (The null handling as a verb assisted by compiler checks.) However, it didn't remove the need to represent a null as a state -- or null as a noun. Why do I separate those 2 verb-vs-noun concepts? Because he used examples with constants such as "10* null" returning a "null" instead throwing an error as violating math logic. And "10< nil" violating 2-value boolean logic. Therefore he plays with the idea that nulls in the language shouldn't exist. Let's rewrite one of his examples using variables instead of constants:
x = Option<int>
y = Option<int>
x = null
y = 10 * x
This irreducible complexity is why "null as noun" will get reinvented as "99999", "-1", etc if you don't have null (whether spelled as ".None", "Nil", or some other standard way). Even if you have an array of Option<int>s and want to serialize to disk, you still need to write "null as data concept" in some fashion (empty string, sentinel value, extra "hasdata" field etc).
As side note, I find it interesting that some commenters believe the author is talking about Optional<T> or similar concept and that I missed his point. I ask those readers to look at the essay again and confirm that it makes no mention of "Option", "Optional", "Maybe", "sum types", "tagged unions", etc. His essay seems to be based on the flawed concept of data representation of unknown values. A key sentence near the top:
>Logically-speaking, there is no such thing as Null, yet we’ve decided to represent it in our programs.
But he's right. It shouldn't be an inherent part of any data. Sometimes, something not being present is a reasonable logical state to be in. For these times, there are tagged unions. In other cases, it is absolutely insane to simply add on the possibility of the NULL value to every data type in a language.
Again, tagged unions do not solve the author's examples of Aristotle logic puzzles and inconsistencies around null/None/Nil/Nothing/etc.
He's not talking about "type safety" enforced by static compiler checks, or opt-in nulls as the default vs opt-out, etc. Therefore, compiler enforced explicit pattern matching code on tagged unions misses the point.
I suggest you click on the Stackoverflow q&a link in the author's essay to get a better idea of what the his philosophical conversation is about.
>In other cases, it is absolutely insane to simply add on the possibility of the NULL value to every data type in a language.
That's a totally valid position but the author isn't talking about that. He's questioning the _meaning_ of "null" in light of observing (in his view) non-intuitive behavior with math and boolean operators.
Your "tagged unions" doesn't address his philosophical conversation at all.
My point is that if the author removed "null" from a hypothetical language because it's "weird", the users of such a hypothetical language would recreate the "null" again. This has nothing to do with discriminated unions or Option<T>.
Your example would be nonsensical in any language I’m aware of.
It makes some sense for interoperation when living inside .NET or the JVM, but this was a known problem when those systems were designed.
2. Restricting a language to non-circular data structures will not be popular.
3. Having a nullable and non-nullable version of every pointer or reference type is not usually super popular either.
Phrased differently: nullable types are actually a fairly reasonable compromise from a set of not fully appealing choices.
This was discussed in rust some time ago: https://mail.mozilla.org/pipermail/rust-dev/2013-March/00330...
And you may start needing fancier versions of polymorphism to express things like "this accepts a T=Foo or Optional<Foo>, and returns Bar<T>".
In passing, I remain to be convinced that that x?.length/etc syntax (and equivalents in other variations of Optional) is really a good idea - to me, it feels like an error-prone way of papering over the fact that non-explicit null(aka absent-optional) checks are just too painful to program with. Phrased differently, I see no reason to believe that the sufficiently-common-to-warrant-special-support appropriate handling of null is just to feed it up through as the computation of the computation if it occurs in any part.
I'd assume that while Kotlin's String? is equivalent to Optional<String>, Kotlin has no equivalent to Optional<Optional<String>>.
Which, personally, I think I'd prefer Kotlin's approach, because that means you can do `maybeString = "exists"`, which is more readable than `maybeString = Optional::Exists("exists")`
Also, you can still nest them in Kotlin, since you can have a nullable type inside generics.
Also, regarding your third point, IMO the question mark syntax in for example TypeScript and C# is fine.
ftp://ftp.cs.princeton.edu/reports/1989/220.pdf
There are various possible solutions for circular structures that do not require destroying all static safety with nulls everywhere.
Doing so *nullifies*
[0] https://channel9.msdn.com/Blogs/Seth-Juarez/A-Preview-of-C-8...
But essentially they're not solving it: you will get a warning on explicit default-ing of a non-nullable reference, you will get a warning on implicitly initialising/defaulting class fields, but you will not get warnings when doing so for structs or arrays.
As an aside here, there is also a school of thought that NULL in SQL was a mistake, see e.g. https://www.dcs.warwick.ac.uk/~hugh/TTM/Missing-info-without... and http://thethirdmanifesto.com/ (Hugh Darwen and Chris Date)
Worse, instead of blowing up and yelling "this is wrong, fix it!", they'll pretend to work and give you the wrong results.
I imagine if null is the "billion dollar mistake", then null never having existed - and relying on sentinels - would've been the trillion dollar mistake.
Doing things that way means every time you check the value of a date, the very first check needs to be "is it the sentinel value?". But that's kind of the concept of a sentinel value anyway.
That doesn't work as most of the program logic simple checks of a date is between two dates. It basically just treats 2025-12-31 as the end of all time. I suspect they will patch it before the end and change that date to something else.
The whole idea is that you don't have to rely on sentinels -- and with optionals and pattern matching we didn't have, ever since the 80s or so.
On the contrary it’s perfectly avoidable. You just need to use an option type with a language that supports comprehensive pattern matching and won’t even compile if you don’t explicitly consider the case of a missing value. If your model is flawed and you don’t want to explicitly specify that some value is optional then the problem is in your way of thinking. And there is no need to invent a language that doesn’t support nulls, they already exist.
As the op says - is a "logic-variant of Null"
No, it's not.
> Or a "typed null" to be more precise.
To the extent this is true, it is a fundamental difference.
> It's got some nice properties, but really it's the good old tri-state.
No, it's not. Because the Boolean type contains only True and False, not Null and the Option<Boolean> type (likewise, Option<Option<Boolean>>, etc.) does not contain True or False. There's no weirdness in doing logic operations on an Option<Boolean>, or a pair of them, because it is a type error to attempt it because Option<Boolean> is not Boolean.
I agree that for other reasons it's better, but conceptually they are used to represent the same thing. (Logic variant of null)
Yes it is, the point they make is that Option<Boolean> is explicitly tri-state in contrast to regular Boolean which should be two-state.
void f(char *x) {g(&x); printf(x);}
It's possible that calling g(&x) might cause x to no longer be null, even if x originally was null. It would then be inappropriate for the compiler to complain that the next instruction attempts to use x without checking whether x is null.
So... yeah I dig it man :) I just left some things out to keep on a pace.
Languages which lack a way to signify an error other than null can be an issue. Most modern languages have option types and/or exceptions, both of which provide good ways to deal with error conditions.
NULL in SQL is also problematic -- e.g. as it behaves wrt aggregations and such. Explicitly user defined unknown values are better.
I'm struggling to understand what you mean here. My understanding is that there is no distinction between "logical null" and a "null reference" in terms of the problem we're discussing--as soon as you introduce nulls as a placeholder for values that are not yet initialized you have to deal with the logical implications of null being a member of those types, no? It's been a while since I watched the talk, but scanning the transcript from the talk we're discussing (https://www.infoq.com/presentations/Null-References-The-Bill...) the way Prof. Hoare talks about them seems to be the same in terms of their logical impact on the type system. I quote:
25:55 One of the things you want is to be able to know in a high level language is that when it is created, all of its data structure is initialised. In this case, a null reference can be used to indicate that the data is missing or not known at this time. In fact, it's the only thing that can be assigned if you have a pointer to a particular type.
...
27:40 This led me to suggest that the null value is a member of every type, and a null check is required on every use of that reference variable, and it may be perhaps a billion dollar mistake.
And another thing you wrote which I don't understand:
We need nulls in programming languages because they are useful to model real-world (lack of) information. All those other clunky techniques will reinvent the same kinds of "null" programming errors!
Isn't that exactly backwards? The entire point Prof. Hoare was trying to make is that using nulls to model the "real-world (lack of) information" causes us to have to contend with the logical problems inherent in making null a member of every type. Additionally, sum types seem to model this in a logically consistent way very well, as other commenters have noted.
These are not trivial distinctions; knowing the semantic intent of a NULL can either hamper or aid in reasoning about a code-base and avoiding subtle errors
That said, the memory layout of Maybe/Optional (assuming it can be stack allocated) will look much like what you outlined, though of course the API can be more user friendly.
But it's really not. First of all, most Haskell compilers will optimize away usages of Maybe that are known to be Nothing or Just. Secondly, GHC at least uses tagged pointers for small sum types, including Maybe.
It catches so many stupid cases where I'd have call bar on foo where foo could be undefined.
TypeScript handles it pretty well given the constraints it has to work in.
Any other number of states automatically ensure that the logic is non-Boolean or non-Extended Boolean.
The interesting thing is that positive powers of 2 will also allow consistent extended Boolean logic and semantics.
The essay is conflating 2 different concepts of "null."
The Tony Hoare quote is about null _references_ which is about aliases to computer memory. (E.g. the dreaded NPE or null pointer exception.) He's not talking about _logical_ nulls such as "tri-state booleans" or "missing data".
The logic-variant of Null usage for "unknown" "missing" "invalid" values is unavoidable in programming. This is why that null concept shows up repeatedly in different forms such as NaN in floating point, NULL in SQL language, etc. If you invented a theoretical language without logical Nulls, the users of your language would reinvent nulls using worse techniques such as homemade structs with an extra boolean "HasValue" field. E.g.:
The "hasvalue" field becomes a "null by convention". Other programmers might not even code a verbose extra boolean variable and instead use (dangerous) sentinel values such as INT_MAX "32767" or negative value such as "-1" as the "pseudo null" to represent missing data. A lot of old COBOL programs had 99999 as some out-of-range value to represent missing data. We need nulls in programming languages because they are useful to model real-world (lack of) information. All those other clunky techniques will reinvent the same kinds of "null" programming errors!The orthogonal aspect is making the type system more powerfully aware of nulls so that the compiler sees that possible null conditions were not checked. A compiler error forces the programmer to put in the explicit defensive code to handle the possibility of null values.