This was basically Dijkstra's point in his BASIC considered harmful post... I think in 2022 it should be C considered harmful for the same reason. C is not the base truth of computation. It isn't even very good. A language smart enough to analyze taking pointers and notice it can't put something on a stack and simply take care of it is, in my opinion, the one that is not catching you off guard... specifically, the "guard" that one must take in C around what is stack versus heap.
That history makes it feel like "taking the address" is a really trivial operation - returning a numerical value that the compiler had access to at that point anyway. Here it's adding a reference to the object in some sense, and maybe even changing how it's allocated earlier in its lifetime (on the heap rather than the stack). I don't use Go and I agree that using &x for that operation feels a bit wrong as an outsider.
It also occurs in langages of category (2), specifically C++ lambdas where i think it can cause UAF/UB. I assume it also happens in C with the block extension (is that still Apple specific?) though I don’t know the details of that thing so maybe not.
Go does in fact stack allocate variables which it can prove not to outlive their lexical scopes, but this is merely an optimization.
There's not really any such thing as a 'local variable' in Go. A variable has whatever scope it has, but there's nothing special, semantically speaking, about variables defined inside functions or inside loops.
>That history makes it feel like "taking the address" is a really trivial operation - returning a numerical value that the compiler had access to at that point anyway.
It is in fact a trivial operation in Go too, as I hope the above has clarified.
---
† Strictly speaking 'semantically heap allocated' is nonsense, but hopefully you know what I mean. There is no way to declare a variable in Go in such a way as to force it to be deallocated at the end of a particular lexical scope. A variable's lexical scope and its lifetime are entirely divorced (as is typical in a GCed language).I never used the word "special". As you say, adding a reference will mean the garbage collector won't deallocate it (until that reference is removed). In other words... its lifetime is extended. That's exactly what I meant.
> If the use of & in the example code is puzzling, it's probably because you're expecting Go to have some C-like concept of an automatic (i.e. stack allocated) variable ...
Not at all. In C++, you can use & on a reference variable and it will return the address of the object being referred to, regardless of whether it is allocated on the stack or the heap (or even statically allocated). Even in C, you can do &*x on a pointer to any object (which is silly by itself, but useful when combined with pointer arithmetic e.g. &x[3] translates to &*(x+3)).
> It [the & operator in Go] is in fact a trivial operation in Go too, as I hope the above has clarified.
Maybe I should have avoided the word "trivial" as its meaning is subjective, but I was careful to define what I meant by it: "returning a numerical value that the compiler had access to at that point anyway". Your comment just confirms that, as I said, it does more than that – it also adds a reference to the object.
---
To be clear, I'm not saying that it's bad or wrong that Go uses the & operator to mean this. Once you're familiar with the language, you probably get used to it very quickly. My point was just that it's a surprise initially if you're not familiar with the language, that's all.
It simply evaluates to the address of the object, just as it does in C. if you think the & operator is doing something in addition to this, I think that must just be based on a misunderstanding.
I am not quite sure what you mean by 'adding a reference' to the object.
Let's take this function:
func foo() *int {
var x int
return &x
}
All that happens is the following:- An integer is allocated (and initialized to zero).
- The address of this integer is returned.
If we dig into the implementation, we'll see that the integer is allocated on the heap. As far as Go's language semantics are concerned, everything is allocated on the heap and left to the GC to clean up.
As an implementation detail, values that provably don't outlive their containing functions are (sometimes) stack allocated. As x outlives its containing function, it won't be stack allocated. That's it. There is no special operation of 'adding a reference' or 'extending a lifetime'. Nor does the compiler even analyze lifetimes except for the purposes of applying an optional optimisation which has no effect on the semantics of the program. If you turned this optimisation off (which you totally could) then there'd be no need for the compiler to worry about x's lifetime at all.
> - An integer is allocated (and initialized to zero).
> - The address of this integer is returned.
That is not all that happens, at least down at the C/assembler level.
Let me illustrate what I mean. Consider this function, which also does both of these things (cobbled together from Google searches so please excuse incorrect syntax):
func foo() uintptr{
var x int
return uintptr(unsafe.Pointer(&x))
}
All that function does is allocate an integer (and initialise to zero) and return the address of that integer. Exactly the same as your function, right? Except it's obviously not - it doesn't extend the lifetime of the integer variable.So why not? The GC somehow knows to ignore the number returned from my function, even though, under the hood, it's still stored in a register or stack location or whatever in exactly the same way as the address returned from your function. So how does the GC know to ignore it? Is that number somehow marked in a way that says "GC, when you're scanning memory looking for address-like numbers, don't pay attention to this one"? No. It doesn't look at the number in the first place because it hasn't been told to look at it.
In contrast, in your example, the memory address is not just returned from the function (in the C sense that it's put in a register for the caller to receive). It, additionally, somehow registers that memory address with the GC to let it know that there's another reference to that variable location. That is the extra thing that your function does that mine doesn't. And that magic happens (or at least starts) at the moment you use the & operator.
It's not just this bizarre gotcha (the fact that C# had it too doesn't make it OK). It's that Go has so many of these cases where they took very strong positions on things and then later reversed their position only after many, many years:
- Generics
- Only one gc knob
- No backwards incompatible language changes
Also, Go has been around quite a long time now and we've all read quite a few rants about its surprising cases. How comes this one never came up before? The thread provides evidence that it bites people regularly. It suggests to outsiders that you can't easily evaluate Go by reading about it because there will be sharp edges that people aren't talking about simply due to the quantity of things that are even worse.
Personally after using it for 10 years I've been bitten very rarely by weird corners of the language and have enjoyed using it. My complaints are more around things I'd rather see removed (struct tags, panic, nils) and inconsistencies (built-in generics were quite limited, I quite like the design for generics they came up with though so I guess that is resolved once they update the stdlib).
Overall it's still my favourite language compared to others I'm forced to work in, I particularly like the decision to eschew inheritance.
But Go did have closures initially, and worse yet, they already had C# as an example of how closures and loops interact. So they definitely had the opportunity to learn from that mistake, and I don't think it's unreasonable to ask why they did not.
I'm not sure prefixing your comment with "WTF" and your 10-years-ago dismissal helps the discussion here. Yes, as we've learned, this was probably the wrong decision, but it's not hard to see why it was done that way originally (C# made the same decision), and now they're having a reasonable technical discussion to try to solve it. And -- even though I've been bitten by this several times myself -- it's not a terribly common occurrence.