News for hackers. Stuff that matters.
Re:GoKart: A static analysis tool for securing Go code by the-smug-one on Wednesday August 18, 2021 @06:09PM (#28227773)
Go has some nice tooling which is quite easy to use w.r.t. static analysis. I started writing a nil pointer analysis tool which was going to take advantage of and provide some more advanced information*. I "unfortunately" had a lot more fun stuff to do during my vacation, but it was very easy to get started with! So kudos to the Go team for making this kind of stuff possible for a 1-man team.

* Just a forward-style abstract interpretation living on-top of Go's type system as an additional layer so you get explanations for why the tool believes that a nil-pointer dereference may occur, etc.

Re:GoKart: A static analysis tool for securing Go code by SquishyPanda23 on Wednesday August 18, 2021 @06:55PM (#28228170)
> I started writing a nil pointer

It still boggles my mind that Go decided to force programmers to worry about nil pointers.

Re:GoKart: A static analysis tool for securing Go code by tptacek on Wednesday August 18, 2021 @11:53PM (#28230157)
Then it boggles your mind that Go functions the way most popular languages work. You don't see so much dunking on Python, Java, Clojure, Ruby, &c, over this, even though these languages dominate the leaderboards.

Which is fine, except that this is probably the second-most boring critique of Go, one virtually everyone has heard before, and it has little if anything to do with the story we're actually commenting on, despite having spawned a huge thread about option types.

If GoKart had been my project, I'd be annoyed.

Re:GoKart: A static analysis tool for securing Go code by nyanpasu64 on Thursday August 19, 2021 @12:57AM (#28230472)
- MyPy doesn't have pervasive nullability, but distinguishes nullable and non-nullable types in the type system. A function declared to return int but randomly returns None has a bug in its type hints.

- I dunk on Java for pervasive nullability too (though there are tools that add @Nullable xor @NonNull annotations used for analysis, possibly sound). But Go has over a decade more hindsight and should've known better.

I haven't used the other languages.

Re:GoKart: A static analysis tool for securing Go code by hnlmorg on Thursday August 19, 2021 @06:28AM (#28232200)
> [Python] distinguishes nullable and non-nullable types in the type system. A function declared to return int but randomly returns None has a bug in its type hints.

Go distinguishes between the two too. You cannot pass nil as a value to int. In fact in Go you'd get a compiler warning[0] so you don't even need to rely on type hints and a properly set up CI/CD pipeline to catch said faults:

The problem with Go is that pointers can be nullable[1] as well as interfaces[2] (interfaces, crudely speaking, being Go's solution to generics and inheritance. Crudely speaking. So interfaces get used a lot).

There is some logic to them being nullable if you think about the code from a hardware perspective but given how opinionated the compiler and language is, I feel they could have done more to catch accidental nils to save the developer from having to consciously consider them each time.

[0] https://play.golang.org/p/BADNnw08hoo

[1] https://play.golang.org/p/b39tY1SDQtZ

[2] https://play.golang.org/p/Fsjsa_-o7Qb

Re:GoKart: A static analysis tool for securing Go code by maleldil on Thursday August 19, 2021 @09:08AM (#28233357)
The int example was a bit misguided, seeing as int is a primitive type in Go, different from pointer types. In Python, both are references.

Basically, it's the difference between returning None from a function `def f() -> MyClass` in Python (which is type error) versus returning nil from `func f() &MyClass`, which is completely normal in Go.

Having `Optional` in the function signature makes it explicit that one has to check for Nones. Go lacks that.

Re:GoKart: A static analysis tool for securing Go code by hnlmorg on Thursday August 19, 2021 @10:35AM (#28234332)
Optional parameters are a different thing. Yeah one workaround with the lack of optional parameters are to send nil values, but that's only going to work if the type you want optional is a nil'able type. Plus there's nothing stopping null values from being passed in Python code outside of optional parameters. My point is optional parameters are one reason why null values might creep in but not the cause nor reason for null values.

Plus I personally think if you need an optional parameter then 99% of the time the API is likely designed wrong from the outset (eg maybe you should instead be passing a class). The reason being is that optional parameters aren't always predictable (eg why is this value optional? When do I need to include a value for it?)

Re:GoKart: A static analysis tool for securing Go code by maleldil on Friday August 20, 2021 @12:00PM (#28247543)
You completely missed my point. Both my examples were when a function returned an object, and the caller has no idea whether that object ever null. It has nothing to do with parameters.

For what it's worth, I completely disagree with your point. Optional arguments are there to set default values. Default values are so useful that even Rob Pike went through some efforts to use them in Go[0] after refusing to have them in the language.

[0] https://commandcenter.blogspot.com/2014/01/self-referential-...

Re:GoKart: A static analysis tool for securing Go code by hnlmorg on Sunday August 22, 2021 @08:07AM (#28264801)
> You completely missed my point. Both my examples were when a function returned an object, and the caller has no idea whether that object ever null. It has nothing to do with parameters.

Type hints then? I don't write much Python so your point might have be clearer had you provided an example. But even in the case of type hints, it requires the developer to be diligent about setting those hints -- Python will happily plod along without them.

That's actually one of the reasons Python isn't my preferred language. I generally prefer something where typing a lot stricter. But this is personal preference and not a judgement on Python.

> For what it's worth, I completely disagree with your point. Optional arguments are there to set default values.

Default values can be set via constructors. Which then allows you to define far more explicit APIs. For example a method that explicitly sets the optional properties when needed might be called `CreateFooWithOnions()` and that method is aware that `.bar` is required to be set for "Onions". Other methods that know the defaults don't need to be set can leave those optional properties with the default. Thus the developer doesn't have to consider if an optional parameter is required in specific use cases, they instead call the method for their use case that is aware of what optionals are required for that use case.

I went through a phase of using optional parameters in the 90s and as I worked with more contributors, and other projects interfaced with my own APIs, I began to realise that optional parameters aren't self documenting. They're not descriptive. Instead it requires the user to understand what's happening under the hood of the API. So I learned from that and moved away from optional parameters.

> Default values are so useful that even Rob Pike went through some efforts to use them in Go[0] after refusing to have them in the language.

That link was from early 2014. Back then methods were only 6 months old (in Go). In fact Rob Pike (and others) tried a lot of things in the early days of Go. Some of those ideas sucked and were never heard from again. And some of the ideas worked and are now part of the core library.

Also the ironic thing here is while the API is called "Option" what Pike is doing is not creating optional parameters but actually creating objects with optional properties. Exactly like I've been describing as the better solution. Albeit Pike's work created a bunch of additional boilerplate and ultimately ends up with something that isn't any more readable. Which is likely why that idiom never made it to Go 1.3 (and beyond).

Re:GoKart: A static analysis tool for securing Go code by logicchains on Thursday August 19, 2021 @03:16AM (#28231232)
>You don't see so much dunking on Python, Java, Clojure, Ruby, &c,

One of the common arguments now for why C# is superior to Java is that it supports non-nullable references. As does C++, which for large latency-sensitive projects is generally picked over C.

Re:GoKart: A static analysis tool for securing Go code by pjmlp on Thursday August 19, 2021 @08:14AM (#28232824)
Which almost no one uses, because it is more trouble than it is worth on existing code bases, and requires everything to be on latest versions.
Re:GoKart: A static analysis tool for securing Go code by scns on Thursday August 19, 2021 @05:58AM (#28232064)
I'd say it has been a better designed language for sure, which wasn't that hard since they just needed to take a look where Java messed up. So many things are obvious in hindsight so it's not a fair comparison. Regarding platform and reach, Java still wins i guess.

Disclaimer: Using neither.

Re:GoKart: A static analysis tool for securing Go code by radicalbyte on Thursday August 19, 2021 @06:41AM (#28232261)
C# has a higher rate of change than Java and a very strong ecosystem - albeit not quite as strong as Java. The tooling - if you pay for it - is in my opinion much better.

The thing is, the Java ecosystem is insane. What else comes close to it in breadth and quality? Python, Go, Ruby etc certainly don't. C++?

Re:GoKart: A static analysis tool for securing Go code by pjmlp on Thursday August 19, 2021 @08:18AM (#28232847)
C++ yes. That is why despite its flaws and complexity, it will be around for decades to come.

Even on the places where Java and .NET languages took over C++ hegemony, it is still there on the implementation of native/extern methods and COM/UWP libraries.

Also don't forget if your favourite compiler is a LLVM/GCC frontend, many of its improvements require a bit of C++ code changes as well.

Re:GoKart: A static analysis tool for securing Go code by SquishyPanda23 on Thursday August 19, 2021 @07:58AM (#28232728)
Golang wasn't designed in the 1990s.

I'm sorry you're bored and also annoyed in the possible universe where GoKart is your project. But it seems like this is a thing people want to talk about in a post about Go static analysis tooling.

I don't see why you're trying to police HN conversations.

Re:GoKart: A static analysis tool for securing Go code by pjmlp on Thursday August 19, 2021 @08:12AM (#28232806)
ML was designed in 1976.
Re:GoKart: A static analysis tool for securing Go code by dllthomas on Thursday August 19, 2021 @04:04PM (#28238664)
There's plenty of room for some people to have got something right long before the bulk of people, as I think very much happened in this case.
Re:GoKart: A static analysis tool for securing Go code by Nullabillity on Thursday August 19, 2021 @05:52AM (#28232028)
Sure, but if they can't even get the boring stuff right then why bother moving on to a deeper evaluation?
Re:GoKart: A static analysis tool for securing Go code by SPBS on Wednesday August 18, 2021 @08:49PM (#28229087)
It’s not as bad as Java’s NullPointerException because primitive types and compound primitive types are much more prevalent (which are guaranteed to never be nil).
Re:GoKart: A static analysis tool for securing Go code by drvd on Thursday August 19, 2021 @09:37AM (#28233656)
Programming in Go since 10 years and I do not have to worry about nil pointers. You seem to assume that the possibility of a pointer being nil is something that is complicated, a burden to the programmer and a source of runtime bugs. It's not. At least not in Go. At least not something you have to worry about in practice.
Re:GoKart: A static analysis tool for securing Go code by SquishyPanda23 on Thursday August 19, 2021 @10:04AM (#28233964)
I'm glad you've had a good experience, but yes it is a source of runtime bugs.

One piece of code from a well established tech company would just crashloop if authentication failed. I've seen others just die if the RPC service couldn't make a connection.

So in practice, yes I do have to spend my time tracking down nil pointer runtime bugs both from colleagues and also from other organizations.

Re:GoKart: A static analysis tool for securing Go code by conradludgate on Thursday August 19, 2021 @03:24PM (#28238180)
In my experience I've not had too many issues because of it (due to good testing) but it definitely requires more effort of me. If they didn't exist I'd be much more productive
Re:GoKart: A static analysis tool for securing Go code by dabfiend19 on Wednesday August 18, 2021 @07:09PM (#28228319)
as opposed to?
Re:GoKart: A static analysis tool for securing Go code by streblo on Wednesday August 18, 2021 @07:13PM (#28228360)
Optionals would have been a way to solve this problem
Re:GoKart: A static analysis tool for securing Go code by logical42 on Wednesday August 18, 2021 @07:39PM (#28228605)
You know what's fun?

Getting a nil where you are supposed to have an Optional.

Re:GoKart: A static analysis tool for securing Go code by mattnewton on Wednesday August 18, 2021 @08:42PM (#28229043)
Afaik this is impossible in swift and kotlin, only optional values can contain nill.
Re:GoKart: A static analysis tool for securing Go code by tbarbugli on Wednesday August 18, 2021 @10:16PM (#28229603)
Try Core Data with Swift and you will see that happening. Lazy objects (vaults) are mapped from objc into Swift and will happily crash on something like a = b where both are not optional.
Re:GoKart: A static analysis tool for securing Go code by mattnewton on Thursday August 19, 2021 @11:45AM (#28235299)
This is happening in objc code or in the swift part? I'm not terribly surprised though, my one experience with core data was miserable once we strayed even a little from the happy path and I ended up rolling my own since we didn't need full functionality anyways. And this was for an internal app, at Apple ┐( ∵ )┌
Re:GoKart: A static analysis tool for securing Go code by logical42 on Thursday August 19, 2021 @03:20PM (#28238119)
It's possible in Java.
Re:GoKart: A static analysis tool for securing Go code by mattnewton on Thursday August 19, 2021 @03:48PM (#28238458)
Right, because the language has the "million dollar mistake" of nullable references by default, which you cannot change without breaking code. And the original comment was bemoaning that Go choose to to have nullable references by default too.
Re:GoKart: A static analysis tool for securing Go code by rowanG077 on Wednesday August 18, 2021 @09:04PM (#28229187)
so... Just make that impossible. It's not like this is unprecedented at this point. It's a standard feature even C++ of all languages supports.
Re:GoKart: A static analysis tool for securing Go code by joshdev on Wednesday August 18, 2021 @07:45PM (#28228658)
Somebody has used Scala
Re:GoKart: A static analysis tool for securing Go code by still_grokking on Wednesday August 18, 2021 @08:51PM (#28229097)
More likely a Java lib form Scala than Scala as such.

In "pure" Scala (not in the FP sense, but just without mixing with Java) something like that is almost impossible.

Re:GoKart: A static analysis tool for securing Go code by marwis on Wednesday August 18, 2021 @09:55PM (#28229475)
Unless something drastically changed in Scala 3, there is nothing to protect you from null in Scala. In fact even Java is effectively safer thanks to all the null checking done by IntelliJ
Re:GoKart: A static analysis tool for securing Go code by still_grokking on Wednesday August 18, 2021 @10:11PM (#28229570)
Null is basically non-existent in idiomatic Scala. So technically you're right but besides calling Java libs there is only an infinitesimal small chance to get NPEs form Scala code. (Scala's NPE is the MatchException ;-)).

For Scala 3 there are improvements. It's "null safe" as long as you opt-in (modulo Java libs, and of course doing stupid things like casting a null to some other type).

https://docs.scala-lang.org/scala3/reference/other-new-featu...

Re:GoKart: A static analysis tool for securing Go code by joshdev on Monday August 23, 2021 @08:00AM (#28274550)
Idiomatic scala yes, but while on boarding engineers hit many cases, where they ended up returning null in places they shouldn’t.
Re:GoKart: A static analysis tool for securing Go code by still_grokking on Monday August 23, 2021 @04:31PM (#28281509)
So you actually complaining about people who don't know what they're doing? How is this related to Scala?

When you have people without clue on the team no language will safe you. You can also crash Haskell programs by throwing exceptions or just using List.head…

Re:GoKart: A static analysis tool for securing Go code by marwis on Monday August 30, 2021 @03:30AM (#28353466)
I'd say safety implies safety from incompetent developers. Similar to the use of the word in memory safety.
Re:GoKart: A static analysis tool for securing Go code by magicalhippo on Thursday August 19, 2021 @06:45AM (#28232287)
So you get an Optional which haven't been set instead of a nil pointer. What's better about that?
Re:GoKart: A static analysis tool for securing Go code by SquishyPanda23 on Thursday August 19, 2021 @07:59AM (#28232734)
The type system knows about it and you're forced to check it
Re:GoKart: A static analysis tool for securing Go code by magicalhippo on Thursday August 19, 2021 @08:11AM (#28232804)
Right, but my point is, the code which would raise an error because the pointer is nil now raises an error because the Optional is not set.

Is there really that much of a difference between those cases?

I agree though that in an interface, Optional conveys a more explicit meaning than something pointer-like, which is always a good thing.

Re:GoKart: A static analysis tool for securing Go code by SquishyPanda23 on Thursday August 19, 2021 @09:38AM (#28233672)
In practice it does make a big difference because if the type system knows about it then it can enforce handling of the exception. (Or more generally, it can just force you to pattern match on the optional time and make sure you handle the empty case.)

Go programs crash at runtime. In general, program failures and bugs should surface as soon as possible. Ideally no later than compile time. Instead, Go makes you wait until the app is running.

For an app that has a lot of configuration options, for example, there can be a latent bug that crashes the binary for some options. And that bug may not be detected for months because nobody was using that combination of config options.

The only real defense of this is to pepper your code with a bunch of nil checks. But these nil checks are also hard to test, so Go devs just learn to ignore missing code coverage. In fact, your code coverage metrics look better if you don't check for nil.

I'm sure at some point Go or a library will offer a version of optional types that is well-adopted. But my point is that by the time Go was designed, null references were already widely considered a bad idea and the source of a huge class of computer bugs. Go still deliberately designed them into the type system.

Re:GoKart: A static analysis tool for securing Go code by drvd on Thursday August 19, 2021 @09:43AM (#28233716)
Some people think Optional/Either/etc are the absolute cure to a certain sort of problems and -- I speak from experience -- it is impossible to convince them that it's not.

Well, of course, if your end users, the business users, are okay if you present them an "optional result" which might or might not be a result, then Optionals/Either _are_ the cure. Unfortunately most end users are pissed if you tell them that you optionally shipped their purchase or that the refund will either be credited to their CC or not.

Re:GoKart: A static analysis tool for securing Go code by mattnewton on Thursday August 19, 2021 @03:55PM (#28238556)
It's about handling those error cases or failing the build, rather than getting a null pointer exception at runtime that moves execution to some higher part that has no context and little ability to correct the problem, or just crashes. You can still handle it wrong, but you are forced to handle it rather than just, in your example, charging the card and crashing the thread when updating the database that you charged them.
Re:GoKart: A static analysis tool for securing Go code by mattnewton on Thursday August 19, 2021 @03:51PM (#28238502)
One has to be addressed at compile time and the other is a runtime error. Sure, you can address it wrong, but it's better than any reference anywhere being able to throw and in my experience really does cut down on application crashes.
Re:GoKart: A static analysis tool for securing Go code by magicalhippo on Thursday August 19, 2021 @04:46PM (#28239168)
Interesting. I've been coding in languages which heavily rely on pointers for decades, primarily C, C++, C# and Turbo Pascal/Delphi, and in my experience nil pointer errors are quite rare.

I agree that it's nice to be explicit about optional stuff, but overall it's not been a huge deal in the projects I've been involved with.

Re:GoKart: A static analysis tool for securing Go code by UncleMeat on Friday August 20, 2021 @09:44AM (#28245836)
Optional doesn't give you that automatically. In C++, you can happily dereference a std::optional without checking it.
Re:GoKart: A static analysis tool for securing Go code by still_grokking on Wednesday August 18, 2021 @08:54PM (#28229112)
As opposed to some modern type system feature that would catch such bugs at compile time rather than let them happen at runtime.
Re:GoKart: A static analysis tool for securing Go code by SquishyPanda23 on Wednesday August 18, 2021 @07:39PM (#28228600)
Most modern languages have solved the problem. There are a variety of ways to do it.
Re:GoKart: A static analysis tool for securing Go code by pkaye on Wednesday August 18, 2021 @08:04PM (#28228784)
Can you list the variety of ways?
Re:GoKart: A static analysis tool for securing Go code by SquishyPanda23 on Wednesday August 18, 2021 @08:44PM (#28229058)
Basically if you need to manipulate pointers, use safe pointers and keep track of pointer ownership. This is how modern C++ and Rust work.

If you don't need to manipulate pointers then nil really just represents a degenerate or optional value. For optional values these can be encoded any number of ways depending on the type system. One common pattern is an optional type. Another is to annotate the type to indicate that it might be null.

The idea is that if a programmer doesn't check that an nullable or optional type is missing then the program should fail at compile time instead of crashing at runtime. Golang chose to crash programs at runtime.

So for whatever reason, Go has decided that null pointer dereferences are not a big deal. But God help you if you try to comment out a variable use without assigning it to "_". Then the program fails to compile.

Re:GoKart: A static analysis tool for securing Go code by Zababa on Thursday August 19, 2021 @06:01AM (#28232080)
> So for whatever reason, Go has decided that null pointer dereferences are not a big deal. But God help you if you try to comment out a variable use without assigning it to "_". Then the program fails to compile.

I think that's a good explanation of why people are so frustrated with this. There are lots of features like go fmt, go vet, the compiler checking that you use all variables and all imports that can feel a bit restrictive. But for something like null pointers, there is nothing. It's incoherent.

Re:GoKart: A static analysis tool for securing Go code by masklinn on Thursday August 19, 2021 @02:49AM (#28231103)
I don't know about "variety of ways". You just make it so pointers can't be null, then provide a mechanism for opt-in nullability, requiring an explicit check / conversion to get a non-nullable pointer (which you can dereference) and allowing free / cheap / implicit conversion from non-nullable to nullable. This can be:

* separate pointer types (e.g. C++ pointers v references)

* a built-in sigil / wrapper / suffix e.g. C#'s Nullable / `?` types

* a bog-standard userland sum type e.g. Maybe/Option/Optional

In modern more procedural languages the third option often will have language-level (non-userland) facilities tackled on for better usability but that's not a requirement.

For cases (2) and (3) it can (depending on language and implementation) also provides a mechanism for making other value types optional without necessarily having to heap-allocate them.

Re:GoKart: A static analysis tool for securing Go code by mattnewton on Wednesday August 18, 2021 @08:33PM (#28228989)
Usually it's forcing the programmer to handle the null case statically, by wrapping the underlying value in something like an optional type and defining the operations that access the underlying value. Think of swift's optional unwrapping in "if let" statements
Re:GoKart: A static analysis tool for securing Go code by izgzhen on Wednesday August 18, 2021 @09:49PM (#28229442)
Golang doesn’t have a proper generics support at its beginning. It is too late now.
Re:GoKart: A static analysis tool for securing Go code by masklinn on Thursday August 19, 2021 @02:52AM (#28231119)
You don't even need generics to avoid nullable pointers, you can special-case it as they special-cased slices, maps, channels, etc… e.g. `*int` -> non-nullable pointer to int; `?int` -> nullable pointer to int, and a tiny bit of flow analysis in nil checks so e.g. `if foo != nil` implicitly creates a new non-nullable version of `foo` inside the block body.