News for hackers. Stuff that matters.
Re:Things Go needs more than generics by baby on Monday October 04, 2021 @01:52AM (#28743497)
As an ex security consultant almost all the bugs I found in Go applications were crashes that happened because of nil. It’s crazy how Rust just gets rid of this whole class of bugs with option and result types (and sum types in general).
Re:Things Go needs more than generics by brundolf on Monday October 04, 2021 @01:57AM (#28743522)
> It’s crazy how Rust just gets rid of this whole class of bugs

As do TypeScript, and Kotlin, and Dart, and Python (with MyPy), and C#, and Swift...

It's honestly inexcusable that Go, as a relatively new statically-typed language, doesn't have strict null-checks

Re:Things Go needs more than generics by mbStavola on Monday October 04, 2021 @05:13AM (#28744667)
I don't know about C# and Swift, but Kotlin, Python, and Typescript are only null-safe-ish; there is always a relatively easy path to smuggling a null value into your program regardless of the guarantees of the types you use.

This isn't to say it's useless, just that it's not as useful as it _could_ be. I've seen codebases in all three languages where there have been real production issues due to null values, even though the type decls say otherwise. Kotlin probably does the best job here but using any JVM library introduces potential (K)NPEs.

Re:Things Go needs more than generics by unscaled on Monday October 04, 2021 @07:16AM (#28745281)
This situation is unfortunately quite hard to avoid in the languages above since they either added null safety (or indeed any kind of type-checking) at a later stage (C# and Python) or they need to maintain compatibility with a language or platform that is not null-safe (Kotlin with Java, Swift with Objective C, TypeScript with JavaScript).

Languages like Rust, Haskell or ML do not have this issue since they do not need to support such a legacy layer.

Re:Things Go needs more than generics by pjmlp on Monday October 04, 2021 @07:45AM (#28745474)
Kind of true, there is still the possibility to smuggle null values via unsafe, or FFI, but then the fault is on whoever certified the code as safe.
Re:Things Go needs more than generics by unscaled on Monday October 04, 2021 @09:29PM (#28754505)
I was hesitating whether to mention unsafe code blocks or FFI, but the no language that gets thing done can avoid abuse of its escape hatches. Even Haskell has to deal with nulls when doing FFI.
Re:Things Go needs more than generics by brundolf on Monday October 04, 2021 @11:12AM (#28747728)
There's still a huge difference between "you can smuggle a null value in" and "the language can't statically reason about null values at all"

You can override TypeScript's checks for anything, not just nulls, and yet people still get tons of benefit from it. Same goes with many of Rust's safety checks in unsafe { } blocks.

Additionally, languages like C# and typescript have to be a little more flexible because they added this feature after the fact. Go had plenty of opportunity to do it up front and not require any compromises, and it chose not to.

Re:Things Go needs more than generics by wildbook on Monday October 04, 2021 @06:02AM (#28744902)
As much as I like C#, your points apply to it too.
Re:Things Go needs more than generics by mtelgon on Monday October 04, 2021 @10:01AM (#28746800)
can you give an example of where typescript fails to protect you against nulls in your own code? (apart from the use of any
Re:Things Go needs more than generics by brundolf on Monday October 04, 2021 @11:07AM (#28747663)
You can turn off strictNullChecks, and there's also noUncheckedIndexAccess which unfortunately isn't enabled by default even in strict mode. You can also introduce flaws via @ts-ignore, casting, etc.

Still, typescript equips you to catch these errors, even if you can technically circumvent it. In practice it can be nearly bullet-proof if you follow good practices.

Re:Things Go needs more than generics by mbStavola on Monday October 04, 2021 @07:11PM (#28753551)
Aside from explicitly turning off null safety and tricky use of casting, an easy example is interfacing with JS.

If you're using either a library that wasn't written in pure TS (maybe JS or JS with .d.ts) or interacting with some unconverted JS from your own codebase, you can easily pass a null through entirely by accident. The problem really stems from the JS end of things, but 9 times out of 10 you're going to be touching JS at _some_ level when using TS so I think it's fair to point out this gap.

Re:Things Go needs more than generics by appletrotter on Monday October 04, 2021 @11:12AM (#28747735)
Abusable things:

- !

- as x

- @ts-ignore

Re:Things Go needs more than generics by skohan on Monday October 04, 2021 @07:10AM (#28745240)
How can you get an unchecked nil into a swift program without the unsafe API?
Re:Things Go needs more than generics by bobbylarrybobby on Monday October 04, 2021 @10:10AM (#28746931)
Implicitly unwrapped optionals
Re:Things Go needs more than generics by skohan on Monday October 04, 2021 @04:48PM (#28751890)
But this is exactly equivalent to calling `.unwrap()` in rust, no?
Re:Things Go needs more than generics by virgilp on Monday October 04, 2021 @06:06AM (#28744926)
C# only sort-of-fixed this problem and it did so rather inelegantly IMO.

E.g. nullable types are not options - you can't "map" them (you can't invoke Select, Where etc). It's easy to "get value or default" (via operator `??`), but you can't do the other, equally frequent thing, of "apply transformation on value if not-null". Or... well, you can, but here's the syntax:

    var newVal = val == null ? (resultType?) null : compute(val.Value);

instead of

    var newVal = val.Select(compute);
Re:Things Go needs more than generics by mathw on Monday October 04, 2021 @06:17AM (#28744978)
Various bits of C# and its libraries also variously ignore non-nullability when it suits them. I know it's done the way it's done for compatibility reasons, but it's really quite confusing around the edges as a result. And despite something being non-nullable in the code it could still be null at runtime thanks in my experience particularly to the joys of deserialisation libraries.
Re:Things Go needs more than generics by CodesInChaos on Monday October 04, 2021 @07:51AM (#28745516)
For that I use my own extension function like `?.Map(x => ...)`, defined as `Map(this T input, Func<T, TResult> map) => map(input)`. Still a bit ugly, but much nicer than the ternary operator.

Of course that doesn't fix all the other issues with null handling in C# (inconsistency between null references and null value types, inability to nest options, ...)

Re:Things Go needs more than generics by andy_ppp on Monday October 04, 2021 @05:47AM (#28744843)
As someone working on a Typescript codebase with "string? | undefined" everywhere I wish this were true.
Re:Things Go needs more than generics by tticvs on Monday October 04, 2021 @04:25AM (#28744371)
Unfortunately it's easy enough to confuse MyPy that you can't really fit in with the rest of this list.
Re:Things Go needs more than generics by rowanG077 on Monday October 04, 2021 @04:50AM (#28744532)
Unfortunately in any real world typescript or python the type checking is almost always partial. Which means it doesn't actually save you from these kind of errors.
Re:Things Go needs more than generics by cageface on Monday October 04, 2021 @05:07AM (#28744634)
I have not found this to be the case in Typescript code I've worked on. The main issue I've found with TS is stuff coming in from outside the system as JSON that doesn't conform to expected formats.
Re:Things Go needs more than generics by jakelazaroff on Monday October 04, 2021 @07:02AM (#28745200)
You can solve the JSON issue by validating that it deserializes to a known type at the point of ingress. I wrote a small library that I use in basically every project to help with that: https://www.npmjs.com/package/narrows
Re:Things Go needs more than generics by CryZe on Monday October 04, 2021 @05:24AM (#28744717)
One issue I've noticed is that indexing into a T[] array where T can't be undefined, results in the expression to be of type T, not T | undefined, despite an out of bounds index evaluating to undefined. I believe I've seen that they recently added a setting that changes this though.
Re:Things Go needs more than generics by cageface on Monday October 04, 2021 @05:39AM (#28744790)
A lot of statically typed languages leave array indexing as a loophole for convenience but yeah this can also be a source of errors.
Re:Things Go needs more than generics by andy_ppp on Monday October 04, 2021 @05:48AM (#28744845)
Consider using https://github.com/gcanti/io-ts to help with that...
Re:Things Go needs more than generics by pjmlp on Monday October 04, 2021 @02:38AM (#28743752)
As ML did already in 1976, and Ada in 1983, with Eiffel in the mid-90's being the first OOP language to have non-nullable references.

While I like Rust, it would be good if people actually did their research while crediting Rust for things it didn't implement.

Re:Things Go needs more than generics by maverwa on Monday October 04, 2021 @07:05AM (#28745213)
I think you are right, in that rust did not invent non-nullable references. But I do not see a problem is someone claming that they like that on a language.

I did not read the parent comment as "rust invented nun-nullables and is great", but as "rust, for example, removes the class of problems this way...". I think its neither helpful nor realistic to require every mention of a feature to be backed up by a proper research into who invented in when.

Re:Things Go needs more than generics by babel_ on Monday October 04, 2021 @07:35AM (#28745398)
I'd say that clearly there's a design choice in whether or not to have it, so Rust choosing to have these whereas Go choose not to is clearly something worth crediting as "probably good design". While I think there's a place for nullables, they're mostly in the area of compiler optimisations and constrained low-level programming rather than higher-level code, since you're at the level of caring about how many words your procedure returns whether you should be returning a pointer to a struct, or whether or not you should use fat pointers (which in high-level code is usually just an easy "yes"), which is a level of optimisation most people no longer concern themselves with and that the compiler can do quite well (heuristically speaking, similar to register allocation).

So worth giving it credit where credit is due for choosing to use something other languages showed were a good idea. But it's all design choices now, there's few major new language ideas in the mainstream, so while I agree it would be nice to see a little more awareness of the legacy, I wouldn't phrase it as confrontationally as "if people actually did their research" when most people only write short comments and replies that probably assume you also know the legacy.

Re:Things Go needs more than generics by masklinn on Monday October 04, 2021 @08:12AM (#28745665)
> While I think there's a place for nullables, they're mostly in the area of compiler optimisations and constrained low-level programming rather than higher-level code

I think here you mean that C-style nullable pointers should be “constrained to low-level programming” and high-level programs should always have option types and non-nullable pointers, that correct?

If that’s the case then I’d say “nay”… because it’s not that hard to have your cake and eat it: while Rust implements niche-value optimisation somewhat generically, nothing precludes special-casing optional pointers such that optional and non-optional pointers have exactly the same ABI.

And then if the language is memory-safe it probably gains in performances, because it doesn’t have to check non-nullable pointers for nulls before dereferencing them.

Re:Things Go needs more than generics by marcosdumay on Monday October 04, 2021 @11:02AM (#28747608)
By "constrained low-level programming" you mean things like the interfaces exported by CPUs and VMs?

If so, yes, that makes complete sense. But if it's on the level of system languages, well, Rust is a perfectly capable low level system language, and constraints nulls only to where they matter.

At the end of the day, null is a value for pointers, and a system language does need pointers. But if you have a reasonable type system, not everything needs to be a pointer, and the type system is a compile-time feature, so it doesn't matter for your code target.

Re:Things Go needs more than generics by pjmlp on Tuesday October 05, 2021 @02:56AM (#28756259)
C spoiled the mentality that everything needs to be a pointer, when in fact, other systems programming languages never did it like that.

Yes pointers are there, for when they are really needed for interfacing with the hardware, dynamic datastructures and reference parameters can be dealt in more type safe ways.

Re:Things Go needs more than generics by ebingdom on Monday October 04, 2021 @03:07AM (#28743951)
> while crediting Rust for things it didn't implement.

* invent

(Other than this minor typo, I'm not sure why you're being downvoted. It's crazy that people in 2021 think non-nullable types are novel/"crazy". Why is nullability the default in most people's brains?)

Re:Things Go needs more than generics by onei on Monday October 04, 2021 @04:27AM (#28744378)
I would guess null is the default because many of us start out with languages like C or Java. I've seen some interns have FP experience with Haskell, but it tends to be a single module and they often don't appreciate the nuance and FP is still not particularly mainstream in industry.

More generally, I don't think many programmers get to see the better way because they're not exposed to it. I can rhapsodise about Rust, but I doubt my company is going to buy into it because they already picked Go.

Re:Things Go needs more than generics by cies on Monday October 04, 2021 @07:19AM (#28745297)
> I would guess null is the default because many of us start out with languages like C or Java.

Go is not a language that "just happened" (what could be said of JS and PHP). Go is designed. Designed by heavyweight language designers (from Wikipedia): Robert Griesemer, Rob Pike and Ken Thompson.

These people knew of Haskell, type systems, and the merits of type safety. I would expect them to know of "billion dollar mistake[1]" that is null. But sadly Go still carries on with the mistake.

I too care more about null-safety and proper sum types (in combination with nice switch/match statements and/or pattern matching) than generics. In the Elm language I found an experience that not having generics is perfectly okay (just a little annoying sometimes).

I find "not null safe languages" not okay nowadays, and I hold this opinion since before Go's first appearance (2009). I really wonder how the designers came to this decision.

I'm afraid this mistake can never be fixed, as null checks are already idiomatic Go. Java also could not fix it (which may be one of the main reasons behind Kotlin). Maybe the best thing we can hope for is Kotlin kind of language for Go (question marks after types to indicate nullability). It's just sad.

[1]: https://www.infoq.com/presentations/Null-References-The-Bill...

Re:Things Go needs more than generics by 5e92cb50239222b on Monday October 04, 2021 @08:43AM (#28745924)
>Java also could not fix it (which may be one of the main reasons behind Kotlin).

C# added some null-safety features despite having a 20-something year baggage of legacy code. While it might not be the perfect solution (these checks work only at compile time and you can enable them per-file), I find that they work great for new projects. You have to be extra careful at boundaries (interfaces with third-party libraries which have not added ? annotations yet, API calls, and so on), but they save a lot of headache inside your own code.

Re:Things Go needs more than generics by cies on Monday October 04, 2021 @09:00AM (#28746090)
Cool. It's a bit like what Kotlin did. Yet after some experience with Kotlin I found it is not perfect. "You have to be extra careful at boundaries" exactly, with Kotlin too.

The question mark is the best thing if you have not build your language with null safety to start with. Please look at Elm for a good example of what real null safety looks like.

With Go they had to change to do the right thing from the start. I really wonder why they didnt.

Re:Things Go needs more than generics by lazulicurio on Monday October 04, 2021 @09:43AM (#28746591)
Also worth noting that Java may be able to fix it in the future after Valhalla drops (and this may mean better implementation for JVM-based languages too).
Re:Things Go needs more than generics by cies on Monday October 04, 2021 @09:52AM (#28746691)
I sure hope so. Not holding my breath though. :)
Re:Things Go needs more than generics by chrisseaton on Monday October 04, 2021 @07:35AM (#28745401)
> I would expect them to know of "billion dollar mistake[1]" that is null. But sadly they had not.

What makes you think that they didn’t know about it, rather than that they did know but decided they weren’t interested in the trade offs for this particular new language.

Re:Things Go needs more than generics by cies on Monday October 04, 2021 @08:55AM (#28746052)
I misworded what I thought. I think they knew but have no clue why they went ahead with the mistake non the less.
Re:Things Go needs more than generics by pjmlp on Monday October 04, 2021 @04:55AM (#28744578)
Thanks, I did not check my post correctly.
Re:Things Go needs more than generics by gameswithgo on Monday October 04, 2021 @07:24AM (#28745339)
Give the guy some credit, he is probably aware of all of that.
Re:Things Go needs more than generics by cageface on Monday October 04, 2021 @02:03AM (#28743552)
I did iOS dev for many years. When I switched from Obj-C to Swift almost all my runtime errors vanished because the compiler caught them for me.

Now I consider nil safety as an essential feature of any modern language.

Re:Things Go needs more than generics by lima on Monday October 04, 2021 @04:38AM (#28744457)
> It’s crazy how Rust just gets rid of this whole class of bugs with option and result types (and sum types in general).

As a Rust security consultant, many bugs we find in Rust applications are crashes due to .unwrap()... it's easier to spot, but still essentially the same class of bug.

Re:Things Go needs more than generics by rowanG077 on Monday October 04, 2021 @04:48AM (#28744518)
I disagree. A nil error in Go is most likely a dev forgetting a check. An unwrap() is a dev explicitly saying this should always contain a value. In fact fixing unwrap is as easy as search for unwrap in your codebase and factoring them all out.
Re:Things Go needs more than generics by vaylian on Monday October 04, 2021 @08:30AM (#28745808)
> An unwrap() is a dev explicitly saying this should always contain a value

Isn't expect() supposed to cover that case?

Re:Things Go needs more than generics by rowanG077 on Monday October 04, 2021 @09:09AM (#28746206)
No it's the opposite. Unwrap() means the dev said this will never not be set. Expect() is saying if this value is not available exit the program with this user facing error message.
Re:Things Go needs more than generics by ModernMech on Monday October 04, 2021 @09:12AM (#28746235)
expect() is like unwrap() but you can give a message as an argument.
Re:Things Go needs more than generics by mathw on Monday October 04, 2021 @06:19AM (#28744991)
It is a shame there are so many Rust examples out there with unwrap() all over them, because it's really not something I would want to see in production code. expect() with a message if you know that the only reason this fails is because something else messed up that should have prevented the call in the first place, sure. unwrap()? No thanks. That's why we have the ? operator now.
Re:Things Go needs more than generics by da39a3ee on Monday October 04, 2021 @08:57AM (#28746070)
Agreed, except I don't think people should use expect() since its name is a mistake made early on in Rust: people use it to write

expect("something bad we don't expect")

unwrap_or_else(|| panic!(...))

is clear and doesn't suffer from the naming mistake.

Re:Things Go needs more than generics by BoxOfRain on Monday October 04, 2021 @05:30AM (#28744739)
Option types in general are a wonderful thing, it's a big part of why I prefer Kotlin to Java. Kotlin's syntax for option types is really nice too, it's just `Int` vs `Int?` as opposed to Rust's `Option<i32>`. I definitely prefer Rust-style result types when it comes to error handling though.
Re:Things Go needs more than generics by jtaft on Monday October 04, 2021 @07:43AM (#28745456)
I don’t think rust got rid of the null dereference problem. Just traded it for something slightly different. ie, calling unwrap when no value exists causes the program to panic.
Re:Things Go needs more than generics by pornel on Monday October 04, 2021 @08:58AM (#28746075)
It did. It really makes a huge difference.

In Go some common types are forced to be nillable, and you can't express "this is never nil". In Rust, "never nil" is the default, even for slices and by-reference types, so right of the bat for the majority of types nullability disappears entirely. You can't make a mistake of `unwrap()`ing something that doesn't support unwrapping.

`unwrap()` is the laziest/worst way of handling optionals, but it's still better than Go's behavior. `unwrap()` is local and explicit, so you can find and review every potential failure point (unlike e.g. finding every use of a nil map in Go). And of course Rust has plenty of better, graceful ways of handling optionals, so you can also ban this method entirely with a lint.

Re:Things Go needs more than generics by moldavi on Monday October 04, 2021 @11:59AM (#28748494)
Doesnt Rust have implicit panics on indexing out of bounds? I wonder if any codebases lint those away.
Re:Things Go needs more than generics by masklinn on Monday October 04, 2021 @12:17PM (#28748836)
> Doesnt Rust have implicit panics on indexing out of bounds?

It does yes. A fair number of other constructs can panic as well.

> I wonder if any codebases lint those away.

Clippy has a lint for indexing so probably.

For the general case, it's almost impossible unless you're working on very low-level software (embedded, probably kernel-rust eventually) e.g. `std` assumes allocations can't fail, so any allocation will show up as a panic path.

https://github.com/Technolution/rustig can actually uncover panic paths, but because of the above the results are quite noisy, and while it's possible to uncover bugs thanks to rustig it requires pretty ridiculous amounts of filtering.

Re:Things Go needs more than generics by baby on Tuesday October 05, 2021 @05:20PM (#28765756)
It’s really hard if you don’t start linting them early on because they creep on your codebase. There are a set of functions that will panic on you surprisingly. Indexing is one example, copy_from_slice is another one, honestly it’s too bad that there are such functions in the library but at least clippy can find them.
Re:Things Go needs more than generics by pornel on Monday October 04, 2021 @05:59PM (#28752603)
It has both optional and panicking ways to index, but it steers users away from using indexing in the first place.

The `for` loop is based on the Iterator trait. Iterators typically optimize better than indexing, and can't panic.

Re:Things Go needs more than generics by arnsholt on Monday October 04, 2021 @07:57AM (#28745563)
Sure, but the crucial difference is that all the unwraps in your code are then hooks that a linter can find, or simply grepping for unwrap in your codebase and do a manual audit of those pieces of code. In my experiments with Rust that's been a very nice way of working: first build a very barebones version getting the basic happy path right, and then getting all the tedious stuff right afterwards by eliminating all the unwraps. By making it explicit, you now have a concrete thing you can audit specifically, rather than every pointer access in your entire program and every pointer you pass off to a library.
Re:Things Go needs more than generics by masklinn on Monday October 04, 2021 @12:05PM (#28748617)
> I don’t think rust got rid of the null dereference problem.

Yeah it did.

> Just traded it for something slightly different. ie, calling unwrap when no value exists causes the program to panic.

That betrays an absence of familiarity with option types.

First of all, an optional pointer is strictly more work, so you're not going to use an optional pointer when you don't have to, meaning most of your pointers will be non-optional and statically checked so.

This means when you do encounter an optional pointer, there's a reason for it. At which point you get to put your thinking hat on and wonder whether that's applicable to your situation:

* sometimes you don't care because it's a one-off or something and you just unwrap() and go on your merry way

* sometimes the pointer is set by construction but the typesystem is not expressive enough to understand that, usually by the second or third time you get around that code and don't remember why the unwrap's there you'll switch to `expect` or `unreachable!` in order to document your assumptions or logic (which is also helpful when those break and the code panics

* and most of the time there's a good reason why the pointer is nullable and it applies to your situation and you probably want to handle it properly and you do.

Plus `unwrap()` and friends are easily greppable so you can review them with little trouble, or flag them during review, or whatever.

In language with nullable pointers (and only that), you've got none of this.

Re:Things Go needs more than generics by rightbyte on Monday October 04, 2021 @06:38AM (#28745081)
Segfaults are a nice runtime assert. Better that than having erroneous error handling code that hides the bug.
Re:Things Go needs more than generics by kitd on Monday October 04, 2021 @04:32AM (#28744416)
With generics, Optionals in Go become much more ergonomic.
Re:Things Go needs more than generics by tgv on Monday October 04, 2021 @05:12AM (#28744663)
But they add an unnecessary overhead, and still don't prevent you actually writing x := nil; print(x.a);. And everyone can have their own Optional, meaning you'll going to install dozens of different generics libraries for just this feature.

There are simple, go-like solutions. There's a character for pointer, so why not a character for a non-nil pointer?

Re:Things Go needs more than generics by kitd on Monday October 04, 2021 @06:13AM (#28744961)
And everyone can have their own Optional, meaning you'll going to install dozens of different generics libraries for just this feature.

Tbf, this is simply solved by putting a generic optional in the stdlib.

Re:Things Go needs more than generics by tgv on Monday October 04, 2021 @08:49AM (#28745993)
How would it work, BTW?

In Rust, you're forced to match against the enum before you can look at the value, but in Go you even can't match. So unless it's implemented as a type cast, testing optionality would be just like testing against nil. The current internal "optional" package just panics, which is just as useful as a panic on dereferencing nil.

Re:Things Go needs more than generics by kitd on Monday October 04, 2021 @10:27AM (#28747171)
Similar to a Java Optional<T> [1]

Give it an interface that allows you to attach behaviours for not null and null values.

[1] https://docs.oracle.com/en/java/javase/11/docs/api/java.base...

Re:Things Go needs more than generics by tgv on Tuesday October 05, 2021 @02:48PM (#28763831)
Function passing/callbacks, no thanks. You still won't get a compiler error when passing a potentially nil value, and I'd rather have readable code. I hope Optional is not going to be in the way of progress in this area.