News for hackers. Stuff that matters.
Re:Go(lang): Robust generic functions on slices by jerf on Friday February 23, 2024 @10:45PM (#39488964)
A missing tidbit that may help contextualize this post: One of the things about Go that surprised me is that if you have a slice which does not represent the full capacity of the underlying array, you can go ahead and reslice it up to that full capacity even though it's a panic to access the things you're reslicing directly: https://go.dev/play/p/oThz2bNFwgr

Consequently, the GC has to assume that anything forward of any given slice into the underlying array may become accessible in the future as there is legal Go that can access it. It's still memory safe, but it surprised me.

I had some code that was using my incorrect belief that slices could not be resliced up in size to implement some light security boundaries. Fortunately it was still legal, because the code in question simply didn't slice things larger and it's not like I was allowing arbitrary user-supplied code to run, so it was still correct in what it was doing. But I was expecting the runtime to scream if I did somehow screw it up when in fact it may not, depending on the exact capacity and what happened when.

It's also asymmetric, as far as I know; you can slice forward into the array if there is capacity, but if you've got a slice that starts after index 0 in the backing array you can't use that slice to walk back into the underlying array. That is, with

     s := []int{11, 12, 13, 14, 15}
     s = s[2:]
as far as I know, 11 and 12 are no longer accessible to any legal Go code (not using "unsafe") after that second line executes.

Corrections (again not involving "unsafe", it's obvious those two values are still accessible through "unsafe") welcome. I was wrong once, it's easy to believe I could be wrong again.

Re:Go(lang): Robust generic functions on slices by infogulch on Friday February 23, 2024 @11:37PM (#39489160)
One detail in the latest 1.22 release is a change in the "slices" library [1] (the library is based on generics, introduced in 1.21) that helps avoid such cases:

> Functions that shrink the size of a slice (Delete, DeleteFunc, Compact, CompactFunc, and Replace) now zero the elements between the new length and the old length.

[1]: https://tip.golang.org/doc/go1.22#slices

[2]: https://pkg.go.dev/slices

Re:Go(lang): Robust generic functions on slices by stock_toaster on Friday February 23, 2024 @11:12PM (#39489050)
You can prevent later longer reslicing by add an additional cap() element to the slicing to shrink the capacity.

  s = s[:3:3]
from your first example link.
Re:Go(lang): Robust generic functions on slices by never_inline on Friday February 23, 2024 @11:13PM (#39489054)
It is reinforcing my belief that language design is hard. Go is supposed to be a simple language, one may write it for long time, but there will be some trap you discover once in a while.
Re:Go(lang): Robust generic functions on slices by bobbylarrybobby on Saturday February 24, 2024 @12:45PM (#39493473)
Language design is hard, but... Jesus Christ, Go really just threw away 50 years of CS language research/experience in the name of “simplicity”.
Re:Go(lang): Robust generic functions on slices by bananapub on Saturday February 24, 2024 @01:40PM (#39493984)
it's got all the latest features from the 1980s built right in
Re:Go(lang): Robust generic functions on slices by sitzkrieg on Friday February 23, 2024 @10:59PM (#39489013)
interesting catch, thanks for sharing. my poor memory is already racing thinking about cases where i may have left traps with the same assumption
Re:Go(lang): Robust generic functions on slices by hairyplanter on Saturday February 24, 2024 @02:06AM (#39489755)
In your playground example, if you print the capacity and the length before and after “re-extension”, it becomes clear what happened. In fact, accessing item 5 after reduction gives a size panic, where as accessing item 6 after re-extension gives you a capacity panic.

Understanding rsc’s “Go Slices” blog is very helpful here. Coming from Java or something, this exposure of underlying storage could be jarring, but coming from C, Go slices are basically built in fat arrays, and this behavior doesn’t surprise me. Maybe it was a design mistake to expose so much of the underlying machinery. Ymmv.

Re:Go(lang): Robust generic functions on slices by everybodyknows on Saturday February 24, 2024 @02:58PM (#39494665)
> ... the GC has to assume that anything forward of any given slice into the underlying array may become accessible in the future as there is legal Go that can access it.

This property complicates the semantics of slices.Delete() in an interesting way:

  It is out of scope of the current proposal to write anything to the original tail
https://github.com/golang/go/issues/63393