News for hackers. Stuff that matters.
Re:OpenJDK mailing list: Null-safety and value types by topspin on Tuesday February 07, 2023 @05:37PM (#34700847)
I don't care how they get there, as long as they get there. In this case they're realizing that NULL makes optimization difficult and are swerving into solving this 27 year old instance of the "billion-dollar mistake." NPEs shouldn't exist in work-a-day code.
Re:OpenJDK mailing list: Null-safety and value types by sam_lowry_ on Tuesday February 07, 2023 @05:46PM (#34700963)
What syntactic sugar they offer is not clear from the mail.

There are already lots of techniques, from annotations to Optional.ofNullable but none matches the simplicity of != null.

They can also go the route of `effectively final` and make the compiler deduce nullness instead of marking it explicitly.

Re:OpenJDK mailing list: Null-safety and value types by cogman10 on Tuesday February 07, 2023 @06:04PM (#34701191)
It'll allow the JVM to more aggressively optimize the code (the main benefit).

Annotations are, unfortunately, all over the board. There's a dozen different `NotNull` annotations out there. Having it in the language will make it the last solution to this problem.

You can't do deduction at compile time because you'll end up with different method signatures if someone calls a method with a null vs not. (not to mention the problem with someone calling these methods with reflection)

You can do deduction at runtime (and they likely will), but then you have to deal with the problem of unwinding an optimization when the hypothesis is proven wrong. That can be really costly when the optimization is dealing with memory layout. (Imagine an array of value types where you deduced they should be non-null, yet some unexpected path inserts a null. In that case you have to reallocate the entire array and it's elements in order to handle that).

It's more than just syntactic sugar.

Re:OpenJDK mailing list: Null-safety and value types by kaba0 on Tuesday February 07, 2023 @06:01PM (#34701159)
Discussions on the mailing list usually tries to avoid the exact syntax for as long as possible to avoid bike-shedding. But according to this mail there would be a default nullability (hopefully not-nullable as the default, but it may depend on whether it is a value class or not), and ! and ? as modifiers (and * for type variables).

So `Point p = null` won’t type check, you would have to mark it as `Point?` where Point is a value (primitive) class. There were also points that it should work with the usual type inference, e.g. `final var p = methodThatReturnsNullablePoint()` would be inferred to `Point?`.

Re:OpenJDK mailing list: Null-safety and value types by aardvark179 on Tuesday February 07, 2023 @08:12PM (#34702693)
Reference types probably can’t be made non-billable by default as that would break too much. But

  String! myString = “Definitely not null”;
doesn’t look too bad.

Edit: it’s worth saying that a lot of careful thought around compatibility goes into features like this. That means the core libraries should be able to transition to non-billable types where this is already a run time requirement, and existing compiled jars must continue to work.

Re:OpenJDK mailing list: Null-safety and value types by vips7L on Tuesday February 07, 2023 @09:10PM (#34703193)
I don’t think anyone would really use String! everywhere.
Re:OpenJDK mailing list: Null-safety and value types by aardvark179 on Wednesday February 08, 2023 @03:26AM (#34705567)
I think you’ll see it creep into parameter types in the core JDK and it will work its way out from there.
Re:OpenJDK mailing list: Null-safety and value types by kaba0 on Wednesday February 08, 2023 @12:45AM (#34704588)
I think it can work in a binary-additive way — the class file gets a new attribute for nullness information, while missing it (older class files) default to “unspecified”. That can be safely consumed by new code, which will get the updated syntax with a new option (specified on a module/package/class basis) where ‘T’ means ‘T!’ and that should be a proper subset of previously compiling code, so I don’t think anything would break that way. Similarly to how generics and unchecked warnings were rolled out.
Re:OpenJDK mailing list: Null-safety and value types by pjmlp on Wednesday February 08, 2023 @01:55PM (#34712906)
See the !! debacle for C#11 to see how changing the default went.

So now we are back at warnings if non nullable references are enabled on the compiler settings, and eventually we can turn them into warnings as errors.

The biggest issue is the large ecosystem of existing libraries.

Re:OpenJDK mailing list: Null-safety and value types by vbezhenar on Tuesday February 07, 2023 @06:02PM (#34701172)
I think the general idea is pretty clear.

`MyClass! myvar` is not supposed to accept nulls. Assigning null to `myvar` would throw exception, comparing `myvar` with null is pointless and probably should emit a warning. Or maybe even error.

`MyClass? myvar` is supposed to accept nulls. Dereferencing this value without checking for null is either warning or error.

`MyClass myvar` is supposed to be unspecified with regards to nulls. So basically current code. You can do whatever you want but you should refactor it to `MyClass!` or `MyClass?` some day.

Re:OpenJDK mailing list: Null-safety and value types by yarg on Tuesday February 07, 2023 @05:58PM (#34701114)
There's not really a nice way of implementing the syntax without changing the language.

Although perhaps they could resolve this with magic imports?

    import java.lang.magic.optional;

    public class Foo{
        public static void main(String...args){
            Foo? f = ...;
            if(f){
                f.doThings();
            }
            f.doThings(); //compile time error
        }
    }
The idea being that importing the magic class enables the cleaner syntax.
Re:OpenJDK mailing list: Null-safety and value types by usrusr on Wednesday February 08, 2023 @04:37AM (#34706046)
I doubt that it's possible to open the door for "language version by imports" without turning that decision into a forever-floodgate. Note that I am not entirely sure that it would be a bad decision (I do lean in that direction however), but it would definitely be a severe one.

It would be like a variant of inviting Dracula across the doorposts, only that this one may or may not actually be a cool guy and not a vampire, but in any case he will sure bring all his friends, and their friends and so on forever.

Re:OpenJDK mailing list: Null-safety and value types by yarg on Thursday February 09, 2023 @12:26AM (#34720107)
With great power comes great responsibility.
Re:OpenJDK mailing list: Null-safety and value types by yarg on Tuesday February 07, 2023 @05:48PM (#34700981)
I've alway felt "billion dollar mistake" is unreasonably hyperbolic.

In retrospect it would be very nice to have a language level ability to define something as optional, and require existence checking for such values;

But it wasn't so obvious at the time.

Re:OpenJDK mailing list: Null-safety and value types by topspin on Tuesday February 07, 2023 @06:37PM (#34701599)
> I've alway felt "billion dollar mistake" is unreasonably hyperbolic.

I guess you can take that up with Turning Award winner C. A. R. Hoare FRS FREng. I'm going to go ahead and keep citing it despite being a epic underestimate.

> But it wasn't so obvious at the time.

It was, just not to the hackers that invented the languages plagued by these mistakes.

Re:OpenJDK mailing list: Null-safety and value types by TJSomething on Tuesday February 07, 2023 @06:06PM (#34701224)
I mean, it doesn't take much for a class of errors that has existed for 55 years to cause a billion dollars in damage. Just in US labor costs, that's less than 5 hours per developer year. And I probably spend about 10 hours a year on average dealing with NPEs.
Re:OpenJDK mailing list: Null-safety and value types by yarg on Tuesday February 07, 2023 @09:00PM (#34703134)
Sure, but most people see $1,000,000,000 and think only of the incomprehensible abstract - they see huge; they don't see what the value means.

They certainly don't see the amortised cost across the decades long operation of a highly successful industry.

So it comes across as a far deeper cut than the stubbed toe that it actually is.

Re:OpenJDK mailing list: Null-safety and value types by usrusr on Wednesday February 08, 2023 @04:16AM (#34705905)
Convenient hand-wave to distract from the question of wether or not it has been a mistake at all: "look, reverse-extrapolated to the individual man-year it's such a small number! Who cares about the sign!"

That hour you spent chasing an NPE? Yeah, it happened. (really not that often though) The time saved by having a single, universally accepted way of implementing optionality, even it's inconveniently always on? That's invisible because we did not live through the alternative.

Re:OpenJDK mailing list: Null-safety and value types by highwind on Tuesday February 07, 2023 @05:56PM (#34701093)
I believe ML family of languages do not have null. And it was developed well before Java. So I don't think timing was the issue.
Re:OpenJDK mailing list: Null-safety and value types by pjmlp on Wednesday February 08, 2023 @01:51PM (#34712825)
Eiffel also didn't, and additionally it supports value types (expanded classes).
Re:OpenJDK mailing list: Null-safety and value types by pkulak on Tuesday February 07, 2023 @06:03PM (#34701176)
Can you expand on that a bit? Why would having it built in to the language (Kotlin-style, for example) be better than not having the concept of null in the language at all and using something like the Rust Option?

Personally, I like the Rust approach. It seems a bit more flexible and can evolve easier over time.

Re:OpenJDK mailing list: Null-safety and value types by cogman10 on Tuesday February 07, 2023 @06:10PM (#34701266)
I think the main issue with the Rust option is they ended up with language syntactic sugar anyways. So in some ways, it's worse than the kotlin-style because reading `let baz = foo?.bar` could have all sorts of weird type implications.

What is baz? Is it Option<Bar>? Is it Result<Bar>? Is it some user defined enum<Bar>? Who knows! You have to find that by looking up the foo definition.

For kotlin, the answer is simple. "baz is a nullable Bar"

Rust did this because interacting with the enum directly was cumbersome.

The concept of "no value" is so integral to day to day programming that elevating it into the type system with "nullable types" makes more sense to me vs using generics trickery. Even if it's slightly less "pure".

Re:OpenJDK mailing list: Null-safety and value types by swsieber on Tuesday February 07, 2023 @06:23PM (#34701427)
I rarely if ever use the question mark with 'Option' when working with rust, and it's been just fine.

I do use it a lot with Result though. But if you have any question about the return type you could[1] just look at the return type for the current function you're in.

[1] I think there's a way to define custom types on nightly, and I wouldn't be surprised if it let you map custom types on Result/Option, but I don't actually know.

Re:OpenJDK mailing list: Null-safety and value types by ramranch on Tuesday February 07, 2023 @06:49PM (#34701736)
I am a fan of Rust, but I feel obligated to point out the return type of the current function does not provide any guarantees about the type which '?' was used on, even on stable. Since the generic requirement to use '?' with a return type of `Result<T,E>` is just `Into<E>`, you would still need to look at the called function since `From<E> for T` could be satisfied for (almost) any T.
Re:OpenJDK mailing list: Null-safety and value types by swsieber on Wednesday February 08, 2023 @08:49AM (#34707948)
Ah, I did not realize that. Thanks for pointing that out!
Re:OpenJDK mailing list: Null-safety and value types by lmm on Tuesday February 07, 2023 @09:52PM (#34703477)
> So in some ways, it's worse than the kotlin-style because reading `let baz = foo?.bar` could have all sorts of weird type implications.

Syntax sugar that works with a normal library type is much nicer than dedicated syntax for a special-case builtin, IME.

> What is baz? Is it Option<Bar>? Is it Result<Bar>? Is it some user defined enum<Bar>? Who knows! You have to find that by looking up the foo definition.

Sure, or if you have a decent IDE you just mouseover it. But that's no different than any other method. `let baz = foo.add(bar)` doesn't tell you what type foo or bar is, and I don't think I've ever seen anyone argue that it should (e.g. by requiring method names to be globally unique).

> The concept of "no value" is so integral to day to day programming that elevating it into the type system with "nullable types" makes more sense to me vs using generics trickery. Even if it's slightly less "pure".

I've found this isn't really true. Once you don't have language-level support nudging you to use it all the time, wanting to have a possibly-absent value is actually pretty rare. (E.g. a lot of the time you want to include a "reason" for why it's absent, so you want an Either/Result-like type - but if you're using Kotlin you end up using a nullable type because you're lazy and the language makes that easier. That's bad for long-term maintainability IME, especially because if you want to switch the nullable type for a result you have to change all your code - unlike Rust where you can switch fairly easily because ?. works the same way for both types).

Re:OpenJDK mailing list: Null-safety and value types by yarg on Tuesday February 07, 2023 @06:14PM (#34701314)
I really don't think that you want null in a modern language, at the runtime level there will eventually be a zeroed out reference field - but a programmer shouldn't have to deal with that.

So I think we largely agree.

(I left example syntax in another comment.)

Re:OpenJDK mailing list: Null-safety and value types by kaliszad on Tuesday February 07, 2023 @07:58PM (#34702521)
Having nil (NULL, undefined whatever) that is falsey as is customary in Lisp/ Clojure is quite useful - the language family depends on it. Most things deal with it very well and it is a useful distinction that something wasn't setup or didn't return any meaningful value for whatever reason.

Clojure has a powerful meta-data system which could be used for checking for errors if Clojure didn't depend on the host platform for that. Also, it seems, Common Lisp also has some ideas about how to handle things more or less in-line with other code without having a special system that you can't see most of the time.

Btw. some of this can go very deep in the technology stack, there were computers (https://en.wikipedia.org/wiki/Ternary_computer) with ternary logic (https://en.wikipedia.org/wiki/Three-valued_logic). Such a computer could improve many aspects of computing, e.g. density and therefore efficiency. It could improve reliability (things could be more explicit). You could divide by 3 precisely and efficiently. It could improve our understanding of logic by making this for most of us alien concept to a more practical and widespread tool.

Re:OpenJDK mailing list: Null-safety and value types by kaba0 on Wednesday February 08, 2023 @12:35AM (#34704520)
Is there a difference between excluding null from a type and having basically syntactic sugar for ‘T | Null’ in the form of ‘T?’ ?
Re:OpenJDK mailing list: Null-safety and value types by MrJohz on Wednesday February 08, 2023 @01:44AM (#34704953)
Yes. Having an explicit `Option<T>` type allows you to write things like `Option<Option<int>>`. This type is not possible in, say, Kotlin or Typescript. (Well, technically it is, but not using the native null types provided by the language.)

Most of the time this isn't a very useful type to have, but it has the big advantage of being mechanically obvious. For example, consider a hashmap of optional types, something like `Map<String, Option<Player>>`. If we write a get method for the map, it should return an Option to indicate whether the value was present or not. But what happens if the value is present, but it is explicitly Null?

The advantage of having an explicit option type is basically that it's easier to compose generics without having to understand what the values might be, which is usually what you want when using generics. That said, most of the time, if you've got an option of options of something, you're just going to flatten that type down anyway.

E: Thinking about it, the other advantage is that it's easier to create a separate namespace for "methods that should exist when T might be null but are meaningless the rest of the time". For example, Rust's `Option` type has methods to map the internal value into another value, unwrap the value and panic if it was null, swap the value with a different one, etc. In languages which use null | T, the equivalent is usually to use Elvis operators and similar (obj?.field), but that requires more special casing.

Re:OpenJDK mailing list: Null-safety and value types by kaba0 on Wednesday February 08, 2023 @04:01AM (#34705815)
I guess one could have ‘(T | None) | None’, which can automatically be flattened to ‘T | None’, couldn’t it?

Also, the Map::get method that can return null is the problem here (even though your example is great and thanks for that, I didn’t think of it), something like ‘Option<Player?>’ should allow to differentiate between those cases.

But I do remember reading that a bottom type does make typing rules harder (e.g. scala 3’s explicit nulls feature which is basically “T is non-nullable, write it as T | Null” is not sound)

Re:OpenJDK mailing list: Null-safety and value types by islon on Wednesday February 08, 2023 @07:12AM (#34707025)
No, you cannot.

var foo: (T | None) | None = None

Which None do you mean here?

This is just basic logic: (X or Y) or Y <=> X or Y

Untagged unions cannot model that. Contrast it with tagged unions:

var foo: Option<Option<T>> = Some(None)

var bar: Option<Option<T>> = None

Now you can reason about where the None means.

Re:OpenJDK mailing list: Null-safety and value types by kaba0 on Wednesday February 08, 2023 @08:32AM (#34707738)
I may have not been clear, but that “flattening” is what I meant, and mentioned it as a “feature” (when Some(None) and None is equally useful/useless).

Surely, if you want to distinguish between 3 states you need more data, hence my recommendation of Option<T?>

Re:OpenJDK mailing list: Null-safety and value types by MrJohz on Wednesday February 08, 2023 @08:39AM (#34707824)
The other commenter related the types to normal binary operations, which I think explains why (T | Null) | Null can't be distinguished from T | (Null | Null). Another way of thinking about it is by asking what "untagged unions" actually means, and in the context of languages like Typescript, it generally means "unions where the tag is the type of the object". ("Untagged" here is a bit of a misnomer.) But if the tag is the type of the object, how do I distinguish between two different nulls? They both have the same type (the Null-Type), which means they both have the same tag in the union, which means they're the same.

What you say about `Option<Player?>` is a good point though. If we want to distinguish between the different results, we need an explicit Option type. But now we've got Option _and_ we've got nullable types. Which should we use? They're both doing the same thing (i.e. marking where a type may be present but might not be), so why do we need both?

In practice, my impression that nullable types are really good for integrating with languages that already have unchecked nulls in then, either for historical reasons (like Java) or because they're dynamic languages (like Python or Javascript). It's a way of acknowledging the null value in the type system without demanding that all the code that been interacting with nulls be rewritten.

However, if you were going to write your own language from scratch, it's difficult to see why you would allow nullables to exist when the Option type does pretty much everything that nullables can, but with more clarity for cases of "nested nullability". You can also still add syntax sugar for it (Rust is going down this route, for example), but you don't have to special case nullability to the same extent.

Re:OpenJDK mailing list: Null-safety and value types by kaba0 on Wednesday February 08, 2023 @11:22AM (#34710314)
Thanks for the answer, I was genuinely curious about the advantages/disadvantages of these approaches. But indeed, types that completely exclude ‘nulls’ seem to be the cleaner solution — though I wonder whether that is even possible to retrofit to Java.
Re:OpenJDK mailing list: Null-safety and value types by insanitybit on Tuesday February 07, 2023 @08:52PM (#34703060)
> But it wasn't so obvious at the time.

Sure it was. That was how it worked before. Hoare created null because of pressure from other programmers to write shitty code that looked like it worked even when it didn't.

Re:OpenJDK mailing list: Null-safety and value types by saagarjha on Tuesday February 07, 2023 @06:36PM (#34701595)
> In this case they're realizing that NULL makes optimization difficult

How so? Null checks are mostly “free” on the JVM, they trap on the zero page and get converted to an exception via a signal handler.

Re:OpenJDK mailing list: Null-safety and value types by Someone on Tuesday February 07, 2023 @06:50PM (#34701744)
It’s not only the checks, but allotted storage. An Integer that’s guaranteed not to be null can be stored in 4 bytes, a “Point” class with two “int” fields that is guaranteed to be not null can be loaded in a 64-bit register, etc. That makes it easier for the compiler to generate good code.
Re:OpenJDK mailing list: Null-safety and value types by pjmlp on Wednesday February 08, 2023 @04:45AM (#34706105)
Eiffel was already doing this with expanded classes and non void references, but yeah, who would have thought such details are relevant. :)
Re:OpenJDK mailing list: Null-safety and value types by marginalia_nu on Wednesday February 08, 2023 @03:59AM (#34705805)
Do NPEs exist in work-a-day code?

I remember working on a codebase where everything had null check pyramids like

  if (foo != null) {
    if (foo.getBar() != null) {
       Bar bar = foo.getBar();
       if (bar.getBaz() != null) {
          for (Quux q : bar.getBaz()) {
             if (q.getFoobar() != null) { ... }
          }
       }
    }
but that was a long time a go in a code base that operated by passing humongous mega-objects with hundreds of fields (running into the limit of how many parameters could be in the constructor was a constant headache), it solved a lot of its problems by printing a stacktrace and shrugging, and objects would be half-constructed and methods would return null in all sorts of scenarios.

In comparison, the entire codebase for marginalia search has about 65 nullchecks in total. A decent chunk of them are in dealing with older APIs that sometimes idiomatically use nulls to communicate things. java.util.Map, BufferedReader and the Servlet api in particular, the rest in dealing with the inherently noisy nature of crawl data. So while there are a few examples where there are nulls in the data, but they are almost always contained to the module that produced them and don't cross interface boundaries. It's very far removed from some pinnacle of code quality, but just gets a few engineering principles right that that other code base didn't.

Re:OpenJDK mailing list: Null-safety and value types by dangets on Wednesday February 08, 2023 @12:22PM (#34711360)
Maybe this is from my epoch of education, but with early Java null checking everything was promoted as a good practice. Even today, depending on what your teacher said was correct, your coding experiences (e.g. if you came from C++, if you read 'Effective Java'), you might still null check everything.

Personally, I lean towards the "use nulls sparingly and be very explicit where null values are possible", use immutable objects etc. This reduces the unnecessary null-check noise and makes things more robust. In java, I even prefer to add @Nullable to fields to try to switch the default mindset to "assume non-null unless @Nullable annotation exists" - though another developer made a note on a code review that "that is not how Java is designed and @Nullable is just noise".

Having nullability as part of the type makes everything more explicit, so I'm a huge fan. Beyond just the perf and safety benefits, it's a win for domain modeling.

Re:OpenJDK mailing list: Null-safety and value types by pjmlp on Wednesday February 08, 2023 @01:49PM (#34712803)
ML solved it in the 70's, Eiffel in the mid-90's, however just like NPE there were quite a few things that Java didn't adopt, and here we are.