News for hackers. Stuff that matters.
Re:Gotchas, Irritants and Warts in Go Web Development by chimeracoder on Thursday March 07, 2013 @12:29PM (#5338849)
> Go does not allow many basic types, such as strings or booleans, to be nil. Instead, when a type is initialized without a value, it defaults to the “zero value” for that type. This is frequently useful, but complicates database interactions, where null values are common.

This sounds like they weren't defining their types properly.

If your value can potentially be null, it should be a pointer to the type, not the type itself. A string can't be null, but a pointer to a string can.

(In fact, there's no magic going on here - 'nil' is simply the zero value of a pointer. So you always get the zero value - you just need to choose the type that has the zero value you want... which is, in this case, a pointer, not a value).

As explained well in this thread[0], this is the most accurate representation of the data itself. You could create your own type that automatically decodes all null values to whatever the zero value is (empty string, etc)., but then you lose that information.

Yes, this forces you to do a check for the null value before using the data for the first time (or to invent your own monad for abstracting this), but at a high level, that's what you have to do in every language.

[0] https://groups.google.com/forum/?fromgroups#!topic/golang-nu...

Re:Gotchas, Irritants and Warts in Go Web Development by pcwalton on Thursday March 07, 2013 @12:53PM (#5339020)
Using a pointer adds an extra heap allocation, though. The right thing would be to use a Nullable<T> type and define a custom marshaller once and for all... but Go doesn't have generics, so you can't do that.
Re:Gotchas, Irritants and Warts in Go Web Development by chimeracoder on Thursday March 07, 2013 @12:58PM (#5339051)
> Using a pointer adds an extra heap allocation, though.

If you want to dereference it immediately (as they seem to want to in the post), this isn't really going to affect you. You probably want to pass around a pointer, anyway, so that you're not copying values over each time.

> The right thing would be to use a Nullable<T> type and define a custom marshaller once and for all... but Go doesn't have generics, so you can't do that.

Sure you can - that's literally what the NullBool, etc. types do

A pointer to a string IS a 'Nullable<String>' - what they need to do is define a way to unwrap this cleanly, which is easy.

Re:Gotchas, Irritants and Warts in Go Web Development by pcwalton on Thursday March 07, 2013 @01:04PM (#5339084)
> If you want to dereference it immediately (as they seem to want to in the post), this isn't really going to affect you. You probably want to pass around a pointer, anyway, so that you're not copying values over each time.

Sure, I'm not saying that the extra allocation will always matter. In most cases it won't. My point is just that this type system workaround does cost some performance. For example, a heap-allocated struct that contains two nullable ints using the pointer trick has 3 heap allocations, not one.

> Sure you can - that's literally what the NullBool, etc. types do

But that has to be done for each type. If you define a custom type Foo and want a nullable version, you have to write the NullFoo boilerplate yourself. This is what generics are for.

Re:Gotchas, Irritants and Warts in Go Web Development by chimeracoder on Thursday March 07, 2013 @01:07PM (#5339095)
> If you define a custom type Foo and want a nullable version, you have to write the NullFoo boilerplate yourself. This is what generics are for.

I don't want to descend down the rabbit hole of 'Go doesn't have generics', but as I said above, in this case, pointers do exactly what is needed.

Re:Gotchas, Irritants and Warts in Go Web Development by kevingadd on Thursday March 07, 2013 @05:45PM (#5340883)
A pointer to T is not the same as Nullable<T>. There are many important differences.

The expressible values for a Nullable<T> are, in theory, the following:

A valid instance of T, or Null

The expressible values for a pointer-to-T are, in theory, the following:

A null pointer (pointer-to-0)

A pointer to a valid instance of T

A pointer to a previously valid, but now invalid because it was freed, instance of T

A pointer to an arbitrary location in memory

In systems programming (the space Go is presumably designed to be most useful for), the distinction between valid and invalid data is pretty important, so it's a little lazy to say 'just use pointers' when there are examples out there of safer, more efficient alternatives.

Re:Gotchas, Irritants and Warts in Go Web Development by eurleif on Thursday March 07, 2013 @05:50PM (#5340904)
>The expressible values for a pointer-to-T are, in theory, the following: A null pointer (pointer-to-0) A pointer to a valid instance of T A pointer to a previously valid, but now invalid because it was freed, instance of T A pointer to an arbitrary location in memory

In C. Go has garbage collection, and no pointer arithmetic, so that's not the case in Go.

Re:Gotchas, Irritants and Warts in Go Web Development by kevingadd on Thursday March 07, 2013 @07:10PM (#5341297)
If you think a garbage collector and the lack of pointer arithmetic protect you from heap corruption bugs and developers deciding they're smart enough to manually manage memory for some extra speed, I've got some bad news for you. ;)
Re:Gotchas, Irritants and Warts in Go Web Development by eurleif on Thursday March 07, 2013 @07:13PM (#5341313)
How do you expect them to do that when the language doesn't allow it? Can you show an example of Go code which creates an invalid pointer?
Re:Gotchas, Irritants and Warts in Go Web Development by Symmetry on Thursday March 07, 2013 @08:32PM (#5341618)
There's a library called "unsafe" explicitely for this. I think it's reasonable to expect people to generally not use it, though.

http://golang.org/pkg/unsafe/

Re:Gotchas, Irritants and Warts in Go Web Development by eurleif on Thursday March 07, 2013 @08:37PM (#5341638)
As far as I know, it can only create a pointer of type A that actually points to a value of type B, not reference unallocated/deallocated memory. Nullable<T> could do that too if the language allowed it.
Re:Gotchas, Irritants and Warts in Go Web Development by lagom on Thursday March 07, 2013 @10:19PM (#5341906)
It's definitely possible to access unallocated memory e.g.,

// p points to address 1000

p := (*int)(unsafe.Pointer(uintptr(1000)))

Of course, this is why use of the unsafe package is heavily discouraged.

Re:Gotchas, Irritants and Warts in Go Web Development by kevingadd on Thursday March 07, 2013 @11:34PM (#5342102)
Any language that lets you call out into third-party libraries not written in that language can end up with heap corruption.

The important distinction is that a 'Nullable' value type need not involve a pointer, which means it can entirely live on the stack, which dramatically limits the damage that can be done by corruption: Worst case, corruption sets the 'hasValue' flag to true, and you read an uninitialized struct off the stack. Much less catastrophic than a double-free or pointer into random memory (Though, of course, a determined attacker could probably make do with either).

Re:Gotchas, Irritants and Warts in Go Web Development by drbawb on Thursday March 07, 2013 @03:02PM (#5339814)
To avoid the nil pointer dereference panic, could you use an idiom borrowed from Go's map implementation?

A library could take advantage of multiple assignment, a la maps:

`i, ok := m["route"]`

If the map has no entry for key "route", i will be the zero type [e.g: an empty string], and OK will be a bool indicating that the key wasn't found in the map.

So in a SQL library it might look something like

`func (s Statement) GetString(columnNum int) string,bool`

Called as: `str,ok := s.GetString(i)`

At least this way your program won't panic() if you forget to check that string != nil and dereference *string, but you still retain the information that the string is nil.

Re:Gotchas, Irritants and Warts in Go Web Development by chimeracoder on Thursday March 07, 2013 @03:11PM (#5339870)
Yes, that's actually more or less what NullString does. It's a struct that contains two fields: String and Valid (a boolean). It also implements a couple of other interfaces, but at its core, it's what you describe.

Functionally, they're basically the same thing - it's only a syntactic difference whether you check whether a pointer is nil vs. checking whether a boolean is false. At some point, you either have to do a check or coerce a null value into a reasonable value. (Or, I guess, just risk it!).

Functional programming purists may cringe at the analogy, but this is is all more or less the same functionality as Scala's 'Option' monad - it just depends on how you want to express it. The only difference is that the Go compiler doesn't force you to check against nil before dereferencing, but if you want to ensure the safety before converting the database value to a String and forgetting about it, then the struct will do that.

Re:Gotchas, Irritants and Warts in Go Web Development by icey on Thursday March 07, 2013 @12:41PM (#5338933)
This is the approach I've taken for dealing with database nulls in go. I tried using NullString / NullTime / etc, but doing that required writing a custom JSON marshaller for every class with nullable data, which was a huge pain in the ass.
Re:Gotchas, Irritants and Warts in Go Web Development by chimeracoder on Thursday March 07, 2013 @01:05PM (#5339090)
I don't know if you're using SQL or NoSQL databases, but if the latter (and your data is stored in JSON), you may find this package I wrote useful: https://www.github.com/ChimeraCoder/gojson

Instead of unmarshalling to map[string]interface{} and then doing awkward type assertions and checks, etc., this will generate structs that you can unmarshal into. That way, if the unmarshalling succeeds, you're guaranteed to know the types of all your values instead of checking them each time.

As for null values, it automatically uses a pointer in the struct definition if it finds a null value - writing that package was how I first learned about this entire topic, actually, which is why I mention it!

Re:Gotchas, Irritants and Warts in Go Web Development by lbarrow on Thursday March 07, 2013 @12:41PM (#5338929)
I think that's fair, and in retrospect we should have started with pointers to strings. If this is the best practice, though, we should change the standard library to reflect that, rather than providing NullStrings.
Re:Gotchas, Irritants and Warts in Go Web Development by chimeracoder on Thursday March 07, 2013 @12:49PM (#5338997)
Well, it seems that they've just chosen the second option (no pun intended) that I've mentioned above - that is, to define a monad to abstract this. There may be some benefit to doing this instead of pointers, but as you've found yourself, using a monad requires you to pass that monadic type up the calling chain (OR to unwrap that and deal with both cases safely).

I haven't used SQL databases with Go much, so I'm just looking at NullFoo types now for the first time, but every time I've thought I could improve on the design of Go in some way, the folks on #go-nuts have (kindly) shown me that I wasn't really thinking about things the right way. So, there's probably some reason that it's been designed this way, but I'm saying that purely based on prior experience with other Go matters, not with package "database/sql".

Re:Gotchas, Irritants and Warts in Go Web Development by abraininavat on Thursday March 07, 2013 @02:10PM (#5339486)
It's extremely generous... hell it's just wrong.. to call Go's NullString/NullBool/etc monads. "Monad" is not a synonym for "Wrapper type"
Re:Gotchas, Irritants and Warts in Go Web Development by Tyr42 on Thursday March 07, 2013 @03:05PM (#5339827)
Well, they do correspond to the "Maybe a" for some fixed type a, and Maybe is a monad, however you are right that it would be clearer to to say Maybe or Nullable or Option type, since that's what is meant, not some general monad.