As do TypeScript, and Kotlin, and Dart, and Python (with MyPy), and C#, and Swift...

It's honestly inexcusable that Go, as a relatively new statically-typed language, doesn't have strict null-checks

This isn't to say it's useless, just that it's not as useful as it _could_ be. I've seen codebases in all three languages where there have been real production issues due to null values, even though the type decls say otherwise. Kotlin probably does the best job here but using any JVM library introduces potential (K)NPEs.

Languages like Rust, Haskell or ML do not have this issue since they do not need to support such a legacy layer.

You can override TypeScript's checks for anything, not just nulls, and yet people still get tons of benefit from it. Same goes with many of Rust's safety checks in unsafe { } blocks.

Additionally, languages like C# and typescript have to be a little more flexible because they added this feature after the fact. Go had plenty of opportunity to do it up front and not require any compromises, and it chose not to.

Still, typescript equips you to catch these errors, even if you can technically circumvent it. In practice it can be nearly bullet-proof if you follow good practices.

If you're using either a library that wasn't written in pure TS (maybe JS or JS with .d.ts) or interacting with some unconverted JS from your own codebase, you can easily pass a null through entirely by accident. The problem really stems from the JS end of things, but 9 times out of 10 you're going to be touching JS at _some_ level when using TS so I think it's fair to point out this gap.

- !

- as x

- @ts-ignore

E.g. nullable types are not options - you can't "map" them (you can't invoke Select, Where etc). It's easy to "get value or default" (via operator `??`), but you can't do the other, equally frequent thing, of "apply transformation on value if not-null". Or... well, you can, but here's the syntax:

    var newVal = val == null ? (resultType?) null : compute(val.Value);

    var newVal = val.Select(compute);

Of course that doesn't fix all the other issues with null handling in C# (inconsistency between null references and null value types, inability to nest options, ...)

While I like Rust, it would be good if people actually did their research while crediting Rust for things it didn't implement.

I did not read the parent comment as "rust invented nun-nullables and is great", but as "rust, for example, removes the class of problems this way...". I think its neither helpful nor realistic to require every mention of a feature to be backed up by a proper research into who invented in when.

So worth giving it credit where credit is due for choosing to use something other languages showed were a good idea. But it's all design choices now, there's few major new language ideas in the mainstream, so while I agree it would be nice to see a little more awareness of the legacy, I wouldn't phrase it as confrontationally as "if people actually did their research" when most people only write short comments and replies that probably assume you also know the legacy.

I think here you mean that C-style nullable pointers should be “constrained to low-level programming” and high-level programs should always have option types and non-nullable pointers, that correct?

If that’s the case then I’d say “nay”… because it’s not that hard to have your cake and eat it: while Rust implements niche-value optimisation somewhat generically, nothing precludes special-casing optional pointers such that optional and non-optional pointers have exactly the same ABI.

And then if the language is memory-safe it probably gains in performances, because it doesn’t have to check non-nullable pointers for nulls before dereferencing them.

If so, yes, that makes complete sense. But if it's on the level of system languages, well, Rust is a perfectly capable low level system language, and constraints nulls only to where they matter.

At the end of the day, null is a value for pointers, and a system language does need pointers. But if you have a reasonable type system, not everything needs to be a pointer, and the type system is a compile-time feature, so it doesn't matter for your code target.

Yes pointers are there, for when they are really needed for interfacing with the hardware, dynamic datastructures and reference parameters can be dealt in more type safe ways.

* invent

(Other than this minor typo, I'm not sure why you're being downvoted. It's crazy that people in 2021 think non-nullable types are novel/"crazy". Why is nullability the default in most people's brains?)

More generally, I don't think many programmers get to see the better way because they're not exposed to it. I can rhapsodise about Rust, but I doubt my company is going to buy into it because they already picked Go.

Go is not a language that "just happened" (what could be said of JS and PHP). Go is designed. Designed by heavyweight language designers (from Wikipedia): Robert Griesemer, Rob Pike and Ken Thompson.

These people knew of Haskell, type systems, and the merits of type safety. I would expect them to know of "billion dollar mistake[1]" that is null. But sadly Go still carries on with the mistake.

I too care more about null-safety and proper sum types (in combination with nice switch/match statements and/or pattern matching) than generics. In the Elm language I found an experience that not having generics is perfectly okay (just a little annoying sometimes).

I find "not null safe languages" not okay nowadays, and I hold this opinion since before Go's first appearance (2009). I really wonder how the designers came to this decision.

I'm afraid this mistake can never be fixed, as null checks are already idiomatic Go. Java also could not fix it (which may be one of the main reasons behind Kotlin). Maybe the best thing we can hope for is Kotlin kind of language for Go (question marks after types to indicate nullability). It's just sad.

[1]: https://www.infoq.com/presentations/Null-References-The-Bill...

C# added some null-safety features despite having a 20-something year baggage of legacy code. While it might not be the perfect solution (these checks work only at compile time and you can enable them per-file), I find that they work great for new projects. You have to be extra careful at boundaries (interfaces with third-party libraries which have not added ? annotations yet, API calls, and so on), but they save a lot of headache inside your own code.

The question mark is the best thing if you have not build your language with null safety to start with. Please look at Elm for a good example of what real null safety looks like.

With Go they had to change to do the right thing from the start. I really wonder why they didnt.

What makes you think that they didn’t know about it, rather than that they did know but decided they weren’t interested in the trade offs for this particular new language.

Now I consider nil safety as an essential feature of any modern language.

As a Rust security consultant, many bugs we find in Rust applications are crashes due to .unwrap()... it's easier to spot, but still essentially the same class of bug.

Isn't expect() supposed to cover that case?

expect("something bad we don't expect")

unwrap_or_else(|| panic!(...))

is clear and doesn't suffer from the naming mistake.

In Go some common types are forced to be nillable, and you can't express "this is never nil". In Rust, "never nil" is the default, even for slices and by-reference types, so right of the bat for the majority of types nullability disappears entirely. You can't make a mistake of `unwrap()`ing something that doesn't support unwrapping.

`unwrap()` is the laziest/worst way of handling optionals, but it's still better than Go's behavior. `unwrap()` is local and explicit, so you can find and review every potential failure point (unlike e.g. finding every use of a nil map in Go). And of course Rust has plenty of better, graceful ways of handling optionals, so you can also ban this method entirely with a lint.

It does yes. A fair number of other constructs can panic as well.

> I wonder if any codebases lint those away.

Clippy has a lint for indexing so probably.

For the general case, it's almost impossible unless you're working on very low-level software (embedded, probably kernel-rust eventually) e.g. `std` assumes allocations can't fail, so any allocation will show up as a panic path.

https://github.com/Technolution/rustig can actually uncover panic paths, but because of the above the results are quite noisy, and while it's possible to uncover bugs thanks to rustig it requires pretty ridiculous amounts of filtering.

The `for` loop is based on the Iterator trait. Iterators typically optimize better than indexing, and can't panic.

Yeah it did.

> Just traded it for something slightly different. ie, calling unwrap when no value exists causes the program to panic.

That betrays an absence of familiarity with option types.

First of all, an optional pointer is strictly more work, so you're not going to use an optional pointer when you don't have to, meaning most of your pointers will be non-optional and statically checked so.

This means when you do encounter an optional pointer, there's a reason for it. At which point you get to put your thinking hat on and wonder whether that's applicable to your situation:

* sometimes you don't care because it's a one-off or something and you just unwrap() and go on your merry way

* sometimes the pointer is set by construction but the typesystem is not expressive enough to understand that, usually by the second or third time you get around that code and don't remember why the unwrap's there you'll switch to `expect` or `unreachable!` in order to document your assumptions or logic (which is also helpful when those break and the code panics

* and most of the time there's a good reason why the pointer is nullable and it applies to your situation and you probably want to handle it properly and you do.

Plus `unwrap()` and friends are easily greppable so you can review them with little trouble, or flag them during review, or whatever.

In language with nullable pointers (and only that), you've got none of this.

There are simple, go-like solutions. There's a character for pointer, so why not a character for a non-nil pointer?

Tbf, this is simply solved by putting a generic optional in the stdlib.

In Rust, you're forced to match against the enum before you can look at the value, but in Go you even can't match. So unless it's implemented as a type cast, testing optionality would be just like testing against nil. The current internal "optional" package just panics, which is just as useful as a panic on dereferencing nil.

Give it an interface that allows you to attach behaviours for not null and null values.

[1] https://docs.oracle.com/en/java/javase/11/docs/api/java.base...

I like Swift’s approach personally, which is that there’s an Option type, but loads of syntax sugar to make working with it easy (just append a “?” after the type to flag it as optional, “if let”, optional chaining with “?.”, “?()”, etc etc.)

Option types require generics, which Go doesn't (currently) have and may or may not be enthusiastically embraced by the community.

Option types must also be baked into the language, standard library and culture from the beginning in order to be ergonomic; otherwise you're constantly having to wrap and unwrap them whenever you interact with code that was written before the type was introduced. Which is most the code you interact with.

Finally, my own experience has been that Option types generally make things more rather than less complicated when you introduce them to a language that already has null. You'll need to make a decision on whether Some(null) is an allowed value, both options will introduce language warts.

Kotlin's approach, on the other hand, does not require generics to work, and interacts very well with the JVM's existing libraries and its existing type system. It's not going to butter everyone's toast. You can't do anything monadic with Kotlin's approach, for example, but I don't get the impression that the Go community is just desperate for monads.

Why would a built-in option type require more genericity than the built-in array and map types?

The other big distinction between the Kotlin approach and optional types is that the Kotlin approach introduces no new run-time types. Null safety is checked statically, and then you're done. That's a big part of why it plays nicer with an existing standard library. It also means, though, that you introduce no extra run-time overhead, which I would assume is something that's considered pretty desirable to gophers.

  void foo(int? x) {}
  void main() {
    foo(5);
  }

  fn foo(x: Option<i32>) {}
  fn main() {
    foo(5);
  }

* val input: String? is a nullable String. It's basically String|null.

There's a third option, which is String!, platform types: since Kotlin has interop with Java, Kotlin will err on trying to be practical for you when calling Java code: it will (rightly so) assume that it is a non-nullable String, but still warn you that it doesn't have the necessary info to infer nullability/non-nullability, so it could technically be null. It's up to you to decide if you want to null-check it. This can be solved in two ways:

- Update your java code to include @Nullable/@NonNull annotations. You should already be doing that anyways, especially if you have control over it. - Don't call Java code/wrap it in Kotlin wrappers. Kotlin code does not have this type inference issue.

This equation makes sense for some mental models, but isn't quite the same as Option. In particular, Option[T] has the nice property that you can map over its inner type in a naive fashion `forall S. (T -> S) -> (Option[T] -> Option[S])` whereas the coalescing version above needs an additional assumption that `S` is non-nullable.

    forall S not null. (T -> S) -> (T? -> S?)

(Bonus point for Java that has introduced Option types, that can still be null)

  String? :: Option
  String :: Option.Some
  null :: Option.None

This interacts terribly with generics, since it means that only one side of the generic boundary can "own" the null value at any one time. To simplify a case where this can cause issues:

    class Loadable<T>(
      // null until value is loaded
      var valueIfLoaded: T?,
    )
    // returns null if user is not found
    fun getUser(id: String): Loadable<User?>

The "correct thing to do here would be to add a `T: Any` bound on Loadable, which prevents T from itself being a nullable type. That would notify the author of getUser that they need to box the User?, so that the cases are kept separate:

    data class Box<T>(val value: T)
    class Loadable<T>(
      // null until value is loaded
      var valueIfLoaded: T?,
    )
    // returns null if user is not found
    fun getUser(id: String): Loadable<Box<T?>>

Nullable types are a feature of the type system and require language support, but the benefit is that you don't touch typical programming patterns

   val foo: T? = x()
   if(foo == null) { return ... }
   // foo is now T, optionality gone

   val foo: Option<T> = x()
   // following must be wrapped and sprinkled in .map, .flatMap, .orElse

Really? I haven't used Java in a long time, but in the languages where I've used Optional types, you'd always check for presence once and then extract the value before further use.

That said, if one ever finds themselves seriously using a nested Option, it's probably time to write a new enum for that use case.

My own cranky opinion is that, in the 21st century, a static type system that can't even reliably distinguish between something and nothing isn't much of a type system at all. Say what you will about dynamic type systems, but it must be acknowledged that they do a much better job of maintaining a firm and logically consistent, if not statically verifiable, grasp of the most basic ontological distinction that the universe has to offer.

There's always the option to wrap the atomic value in a struct of some sort. But, without generics, that's going to feel more like gRPC's awkward, verbose way of doing it than the ML family's Option<T> type. That approach is (mostly) tolerable in a datagram format, because I/O is going to be a hot mess no matter what you do, anyway, but I'd hate to have to do it with the types I'm using in my actual code.

I'm generally fairly disdainful of the, "every language must cargo cult everything functional languages do," thing, but it is interesting to observe that generics and proper algebraic types would cover all of these use cases - and more - cleanly and without having to add a specific language feature to cover each one. Which has me wondering, if the goal of Go was to create a maximally simple and ergonomic imperative language, did they actually achieve that, or did they follow a greedy search into a local maximum?

A couple things I have tried:

- hope default values align with your business logic, eg an empty string isn't a valid name and 0 isn't a valid age.

- for partial updates, populate the existing values before unmarshalling, then unmarshal on top. Missing fields in the json won't overwrite the existing values

- unmarshall into a map[string]interface{}, which gives you the semantics you want.

If a function says it returns a struct and an error value, then that function is going to return things into a memory chunk sufficient to hold that struct and error value. "sizeof" is a well-defined operation in Go. Everything has a precise size the compiler uses. Slices may look dynamic, but they're actually a three-word structure for size, capacity, and pointer; the "dynamic size" happens behind that pointer. Maps may look dynamic, but they're a fixed struct as well under the hood. Channels may look like they could be dynamic, but they have fixed sizes as well for their internal components. Go is a value-based language, not a reference-based language.

The same thing happens when you declare a struct value in a function. Memory for it is allocated immediately.

"Initialize with empty value" is not a solution to any sort of type issue and has nothing to do with nil; "initialize with empty value" is a solution to C's uninitialized value problem. Unlike C, which simply grabs some RAM and gives it to you, and leaves it to you to figure out what to do with the garbage values in it, Go guarantees that all fresh values you receive are fully initialized to a well-defined "zero value".

In Go, when you say you have a struct, you do, right then and there, and so, it needs a value. You don't have an "undefined" value, because Go is too low level for that. For Go to have an undefined value for a struct isn't something it could just bodge on, it isn't something that was just an "oversight", it would actually be a fundamental overhaul to the language. Go would have to be adjoining values to the structs you declare, making it harder for you to know how much memory they take, or it would have to completely shift to a reference-based language, or it would have to do something else like that not in keeping with the nature of the language.

Rust, for instance, is doing more work than you may realize when you "Option" something. Think about what the memory representation of Option<byte> is. Without more information from the user, you can't use any value of the byte as the undefined value, because all 0-255 values are valid byte values. You have to stick it somewhere else. Rust, and some other languages, often do some magic to turn your byte into a 16-bit int instead and pack the invalidity into there. Having an array of Option<...>s is a non-trivial operation for the compiler, especially to do it maximally efficiently. Go doesn't do this kind of thing. The struct that is declared is what is in memory.

I welcome you to think that's still a problem in the language, but I hope this at least makes why Go works this way more comprehensible.

> Having an array of Option<...>s is a non-trivial operation for the compiler

This is just not true. Arrays are always a number of values laid out in memory, exactly like in C. There is no magic for an array of options. It works the same as an array of any other value: you put N of them in memory, one after the other.

Sum types in general usually have compiler magic associated with them, because the common case of options or sums between various integers can get good results with the compiler being smart enough to pack the "None" option somewhere clever. As the size of the thing being used as a sum type goes up that amortizes to being less important. The naive way of using some integer as a tag and then having a chunk of memory large enough to store the largest value in the sum type can get very inefficient for small "largest values", especially if you have to round to a full 64-bit machine word for some reason.

Also, to be abundantly clear, I think this is all a good thing. It is intrinsically part of the value of a sum type in a language that you're not forced to think about the modestly complicated memory layout such things entail. It is good that compilers have some special cases for when you're summing on small values.

Seems more likely you'd limit your enums to 256 entries and always use a byte than require a word by default, no?

niche-value optimisation is a much more complicated affair so it's unlikely as a baseline indeed.

Personally I prefer how Protocol buffers handles not present strings compared to C. If you stick to std::string, C++ isn't bad either.

It seems like one of those fundamental advancements in language design, like when computer science collectively decided goto should be deprecated in favor of flow-of-control expressions.

I don't see how making types non-nullable solves the problem, doesn't it just create a new problem? When I load the string from somewhere outside of the program, if it is not present, I would either have to check for that condition otherwise I will crash, otherwise I would assign it some magic value and have to check everywhere "if not magic value".

otoh, if the strings are not external then there is little danger of things ever being null no?

If the type is non-nullable then the compiler will refuse the program if you have not dotted your is and crossed your ts. So you'd have to check your expectations at the edge, and if the expectation is "that is indeed optional" then it's optional internally and the compiler will ensure that is acknowledged at every place it's used.

> otoh, if the strings are not external then there is little danger of things ever being null no?

It depends. Lots of things are internally nullable. If I look an account by a key that's been provided externally, the account may or may not exist, that's an optional.

However it's true that most of the internal values will be non-nullable, and then the advantage of non-nullable types is that is checked and validated by the compiler, so it avoids misuses and misunderstandings.

When you only have nullable types, then the only thing you can do is assume, everywhere, and pray that you're right.

But I sort of agree that excluding nulls is only part of the solution, because existing isn't the only requirement that values have to meet. I often need strings to be non-empty or have a minimum length.

Many modern type systems allow you to define new types that conform to an existing type's interface. But it's often a rather convoluted affair.

This is a very important programming pattern that should be used at every available opportunity, and probably one of the biggest programming failures that is rarely talked about in programming. Dynamically-typed languages encourage this style more than statically-typed languages but it can be done in either one. You need to do as much of your validation as close as you can to the time that data enters your system. If you don't, instead of validation living at your edge, it gets smeared through the entire program, and it will be done incorrectly when that happens.

(If you have a deeper layer that has more restrictions on the validity, then the additional validation should be done on that deeper layer. This can be applied recursively within a program if it is big enough to have multiple domains. But you always want edge validation.)

Architectures that fail to do this inevitably become a mess on the inside. Functions develop that are passed "strings" but they don't know if they're validated strings, or decoded strings, or what. Where things get validated and decoded becomes incredibly complicated. Functions start growing options describing what's being passed, but then the functions calling those functions end up eventually being wrong themselves. This eventually metastasizes into the sort of code that nobody can change because every attempt to change something screws up some delicately balanced code path. The only solution to this is to start over and do this edge validation I'm talking about, so that the entire rest of the code base just stops having to worry about it.

This also goes hand-in-hand with the idea that you should decode data at the edge, and pass around data internally only in its natural format, and encode data only at the edge as it leaves. If you're in the "middle" of a program and suddenly there's code to URL decode a string, there's an architectural problem in that code. (This code is likely to become code in the future to conditionally decode the string, or much worse, guess at whether the string needs to be decoded, which is getting perilously close to a You've Already Lost situation.) If you come to a deep enough understanding of what it means you can start to see that validation and decoding into a canonical internal format, and the encoding on the way out from the canonical internal format, are the same thing.

I know this is a common question since I also had it myself at the beginning, but see the strong types not as the assertion that the world will never have a nil string in it, because that's obviously an impossible assertion, but as a statement that any calling code is going to have to deal with what happens when there's a nil string (or whatever other invalid input), because I, this strongly typed library, am not going to deal with it. This statement is also almost always correct, too, because the library lacks the context to know how to handle things it can't handle. You shouldn't ask libraries to handle things they don't know how to handle, because, well, they don't know how to handle them.

In C# we were going from "every reference type can be null" to "only things marked with '?' can be null" with nullable reference types. When I started using Go it was nice to see that only something, that is explicitly created as pointer is nullabe, everything else can't be null / nil - and you can't get panic from nil referencing stuff

Well, it's the same in C# since 1.0 to some extent, except that people rarely used `struct` (value types) in C#. The ecosystem matters a lot, of course, but just at the language level, Go and C# structs have the same semantics (though in C# it's harder to get a reference to a struct).

Like, I'm not trying to be cute here, but the point of having a "non-nullable type" is that you can get an error message when you use a nullable type in its place. However, if you're not talking about non-nullable types and just about whether the compiler knows a value is nil, well, the compiler knows that.

The safety is contagious, preventing crashes from nil pointer dereferences at runtime. For ex: 99% of your program is functions accepting only non-nullable types then 99% of your program is guaranteed to not crash from nil pointer deref.

With Go, you have to check for nil in every function even if all your callers already check for nil. That's because Go can't declare a function's arg as non-nullable type.

I have no idea what you mean.

  func foo(a int, b someStruct, c *someStruct)

Now, it is true that there are multiple reasons for using a pointer aside from supporting nil (ability to modify, optimization not to copy a huge object).

Your claims are correct, but it feels like you're missing the point they are (ineffectively) making.

You make it seem like value types solve this problem, but they don't. C also has the same thing you're talking about, but it still has nullability problems.

I think other comments are not even asking for anything complex. Something akin to C++'s references (which cannot be null) would already be a step forward.

Are you sure? This would only work in a world with a single goroutine.