News for hackers. Stuff that matters.
Re:Down the Golang Nil Rabbit Hole by saghm on Tuesday March 30, 2021 @12:02AM (#26631161)
> It, to me, is Go's third biggest wart.

Out of curiosity, what are the first two?

Re:Down the Golang Nil Rabbit Hole by shp0ngle on Tuesday March 30, 2021 @01:31AM (#26631495)
Not him.

For me it's:

1. context.Context and putting everything under the sun there, untyped (which is a design fault, not just "people using it wrong")

2. reusing the looping variable in for loop

2.5. general awkwardness of go templating (I plan to write whole article about this, but too lazy to actually write it)

Re:Down the Golang Nil Rabbit Hole by pcl on Tuesday March 30, 2021 @04:58AM (#26632446)
I struggle with context.Context as well. It really comes across as a design cop-out.

The language makes a big deal about memory management and data structure ownership, and as a result it’s sorta a pain to send stuff around on the stack, both at initialization and runtime. As a casual golang user, the context mess feels like an escape hatch to help with those problems. I’d prefer some GC compromises built into the language itself instead.

Re:Down the Golang Nil Rabbit Hole by throwaway894345 on Tuesday March 30, 2021 @10:15AM (#26634828)
I'm confused about how a "GC compromise" would help with having to pass things around the stack. It sounds like you just want a request-scoped global or something, but I don't know what that has to do with the GC?

FWIW, I think request-scoped globals are terrible and is among the many things I hated about Flask. I think generics will largely solve this problem by allowing us to write generic middleware with a type-safe context and probably even functions to compose the middleware and automatically propagate the context variables.

Re:Down the Golang Nil Rabbit Hole by pcl on Tuesday March 30, 2021 @04:28PM (#26640004)
context.Context is used in a number of situations, including for a "don't call it a request-scoped global". And whatever, web frameworks always end up with one of those. Would love to see languages do better, but ok.

My concern with its use in golang is that so much seems to get jammed into it. My experience with it is that it ends up being a holding pen for all the things that other systems do with initialization-time dependency injection / resource lookup sorts of things, but replaced with "oh just pass `ctx` as an argument everywhere".

From what I've seen of Go, it looks like that happens because of the state management idiomatic constraints of the language, which in turn seem GC-related.

In a garbage-collected language, I'd put a bunch of context-y things into some sort of request executor datastructure, the web framework would manage the initialization and retention of that, and request handlers would operate against those intialized data structures. That doesn't line up all that well with idiomatic Go, and so we end up needing to hide those sorts of things somewhere else. Enter context-abuse.

Of course, there are plenty of use cases where those sorts of patterns aren't relevant. Right tool for the job and whatnot.

Again, I'm a casual golang user at best -- I bet I'm missing some newer / better approaches, and would love to learn more about what they are.

Re:Down the Golang Nil Rabbit Hole by shp0ngle on Wednesday March 31, 2021 @12:31AM (#26643544)
Go has GC? I don't understand your message that much :)
Re:Down the Golang Nil Rabbit Hole by pcl on Wednesday March 31, 2021 @06:20AM (#26645481)
Yeah but my experience with idiomatic go has been heavily value-biased, and a bunch of useful patterns for state management tend to lend themselves more towards a reference-oriented system. In my experience, this is outside the happy path, but necessary, and so people end up jamming all sorts of things into context.Context to get around the idiomatic paths.
Re:Down the Golang Nil Rabbit Hole by saturn_vk on Tuesday March 30, 2021 @06:33AM (#26632999)
My belief is that a) context should never have had a dual role of being able to hold arbitrary data (just add a map[interface{}]interface{} to the *http.Request ...) and b) it should not have existed as a means of work cancellation. Right now, one is forced to have a context argument just so that it can be passed from some upper level to a lower level that actually needs it, even if your function itself doesn't really need it. Since goroutines are built into the language, a means of cancelling them should've been built in as well.
Re:Down the Golang Nil Rabbit Hole by kortex on Tuesday March 30, 2021 @08:42AM (#26633897)
But how do you cancel something without a handle to that something?

What (I think) you are suggesting implies globally scoped, mutable variables that affect control flow dramatically. Sounds really hard to reason about and very munch anti-Go-explicitness. Context is the handle by which an outer context holds on to the nested ones.

The only non-bonkers way to have a handle like that would be to give every function a getContext function to query some kind of context tree and signal parent/children. This is totally against Go explicitness as well.

Maybe something like Scala's implicits to sugar it, but that's another not-go-looking pattern (although it would be nice)

Re:Down the Golang Nil Rabbit Hole by bbatha on Tuesday March 30, 2021 @09:10AM (#26634148)
The go statement could return a handle to the go routine. There can be a cancel magic function that cancels the go routine. You can regain the task local storage by using this handle as a key into a map of task data. This would have been a breaking change particularly in CGo threads but it’s a vastly simpler model for the user.
Re:Down the Golang Nil Rabbit Hole by saturn_vk on Tuesday March 30, 2021 @03:31PM (#26639242)
This is just hypothetical, but the go statement could be an expression that returns a cancel function. From there, the sky is the limit. And in the other side, have a runtime.Done function that returns a channel that will need closed when the said cancel function is called. There are a bunch of other runtime functions that are goroutine specific already, so there is precedent
Re:Down the Golang Nil Rabbit Hole by skrtskrt on Tuesday March 30, 2021 @10:37AM (#26635077)
I hated Go templating until I really dug into it and started loving how flexible and not-OOP it is.

It makes me want to port it to Rust, since so many of the major Rust templating libraries are Jinja copies or Handlebars copies.

Re:Down the Golang Nil Rabbit Hole by specialist on Tuesday March 30, 2021 @08:18AM (#26633726)
What would you call fixing #2? "Loop's vars are const within loop's scope?"
Re:Down the Golang Nil Rabbit Hole by shp0ngle on Tuesday March 30, 2021 @08:50AM (#26633962)
No, it can just use a different variable each loop turn?

Go authors themselves admit it might have been a design mistake, but is unfixable without breaking compatibility.

https://golang.org/doc/faq#closures_and_goroutines

> This behavior of the language, not defining a new variable for each iteration, may have been a mistake in retrospect. It may be addressed in a later version but, for compatibility, cannot change in Go version 1.

Re:Down the Golang Nil Rabbit Hole by throwaway894345 on Tuesday March 30, 2021 @10:18AM (#26634864)
> No, it can just use a different variable each loop turn?

How does that work? The call-frame needs to be of fixed-size, but it's rarely knowable at compile time how many iterations a loop will perform. Unless we begin heap-allocating loop variables or treat each loop iteration as a function call (both of which will kill performance, I think), I don't see how this could work?

Re:Down the Golang Nil Rabbit Hole by morelisp on Tuesday March 30, 2021 @12:36PM (#26636791)
You seem to have some fundamental misunderstanding about what costs memory. As the documentation says, the following works fine:

    for _, v := range values {
        v := v // create a new 'v'.
        go func() {
            fmt.Println(v)
            done <- true
        }()
    }
The scope of the variable has no effect on how many values may or may not be allocated.
Re:Down the Golang Nil Rabbit Hole by throwaway894345 on Tuesday March 30, 2021 @12:55PM (#26637076)
> You seem to have some fundamental misunderstanding about what costs memory.

Probably, hence my "How does that work?" question.

I don't doubt your example--I know it works, but I'm struggling to understand how it manages to work, since (as I understand it) the closure presumably captures the environment, which presumably means taking the address to the new `v` which presumably is the same address for each loop iteration?

Re:Down the Golang Nil Rabbit Hole by morelisp on Tuesday March 30, 2021 @01:32PM (#26637595)
https://en.wikipedia.org/wiki/Funarg_problem

Whether a loop is involved is irrelevant, it comes up any time a value referenced a closure might outlast the environment which created the closure. That can happen in loops but also if conditions or normal function calls. The value might be promoted to the heap, but in trivial cases like the above it could be copied directly on the new goroutine's stack. Or if escape analysis says the closure can't outlast the environment, it will use the same stack.

Re:Down the Golang Nil Rabbit Hole by throwaway894345 on Tuesday March 30, 2021 @01:48PM (#26637835)
That makes sense, so most likely Go's compiler is promoting the new variable to the heap (I doubt it's smart enough to recognize that the new `v` is only used by the closure which in turn is only used by the new goroutine and thus safe to allocate `v` in the new goroutine's stack). In any case, thanks for the link, I didn't know there was a formal name for the problem.
Re:Down the Golang Nil Rabbit Hole by morelisp on Tuesday March 30, 2021 @02:12PM (#26638158)
So again: Allocations of regions of memory representing values, not variables, are what does or does not get promoted. If `values` is an array of strings, do you think the following makes a new string on the heap (that is, a new 16 byte heap allocation, internally pointing to the same variable-sized region of memory)?

   for _, v := range values {
        go func(v string) {
            fmt.Println(v)
            done <- true
        }(v)
    }
No, the 16 bytes will be copied onto the goroutine's stack.

Do you think it's smart enough to do the trivial transformation of the first into the second?

Re:Down the Golang Nil Rabbit Hole by throwaway894345 on Tuesday March 30, 2021 @05:39PM (#26640895)
> So again: Allocations of regions of memory representing values, not variables, are what does or does not get promoted.

Right, I understand the difference between a variable and a region of memory.

> If `values` is an array of strings, do you think the following makes a new string on the heap (that is, a new 16 byte heap allocation, internally pointing to the same variable-sized region of memory)?

No.

> Do you think it's smart enough to do the trivial transformation of the first into the second?

I guess I'm surprised that the semantics for the `v := v` in a loop thing are "this will get allocated somewhere else such that it's guaranteed not to be stomped on by another loop iteration". For example:

    for _, v := range []int{2, 4, 6} {
        v := v
        mut := sync.Mutex{}
        go func() {
            mut.Lock()
            fmt.Println(v)
            mut.Unlock()
            done <- true
        }()
        go func() {
            mut.Lock()
            v = 1
            mut.Unlock()
            done <- true
        }()
    }
If it allocates `v` on the goroutines' stacks, then it won't catch the mutation, so presumably it allocates on the heap at least in that case. In whatever case, the semantics are surprising to me.
Re:Down the Golang Nil Rabbit Hole by nemothekid on Tuesday March 30, 2021 @02:10AM (#26631643)
First, I consider warts of the language to be things that are side effects of the way the language is designed. This means that I would have loved for Go to have non-nullable types, Pike didn't bless Go with that feature. Same for generics and the other popular criticisms. That said the other 2 are

1. Error handling. Not only is `if err == nil` (or my preferred `if _, err := foo(); err == nil`) a lot of noise, errors are completely opaque and might as well be strings.

2. Dependency management. It was a mess for a long time, go modules fixed a lot of issues in it's own way but still has problems (e.g. the "v2" workaround). My theory is they expected everyone, in some way, to "self-host" their modules (like how Java would have com.google.foo.bar, eventually everyone else would have "http://go.company/package", but GitHub became central repository of software for everyone and linking directly to packages was no longer sound).

Re:Down the Golang Nil Rabbit Hole by tptacek on Tuesday March 30, 2021 @03:38AM (#26632063)
Errors in Go have been queryable chains since 1.13; it's pretty seamless with `%w`. Rust has the `?` shorthand, which is very convenient, but also the annoyance of type matching errors, particularly across different libraries, to the point where third-party error handling libraries like `anyhow` are pretty important. Leaving nil itself out of the picture (obviously, I'd rather Go had Option types), I don't know that either language has a definitive edge here.

Errors in modern Go are definitely not just strings.

Before wrapped errors in Go, this would have been an easier case to make against the language.

Re:Down the Golang Nil Rabbit Hole by iudqnolq on Tuesday March 30, 2021 @04:05PM (#26639697)
Anyhow's tagline is literally "a better Box<dyn Error>", which gets you nearly the same result modulo some pretty printing.
Re:Down the Golang Nil Rabbit Hole by tptacek on Tuesday March 30, 2021 @05:05PM (#26640493)
Yes, Anyhow is almost as easy to use as Go's current error handling.
Re:Down the Golang Nil Rabbit Hole by iudqnolq on Tuesday March 30, 2021 @05:10PM (#26640562)
You're significantly more experienced than me, so I'll defer to you here. What you're saying doesn't make sense to me though.
Re:Down the Golang Nil Rabbit Hole by tptacek on Wednesday March 31, 2021 @03:29AM (#26644434)
You're probably more experienced than I am with Rust!
Re:Down the Golang Nil Rabbit Hole by iudqnolq on Wednesday March 31, 2021 @06:14AM (#26645446)
I might be misunderstanding go, but I think the big difference is you use anyhow to get looses error handling at the edges of binaries. It'd be weird to see in a published library, and unusual to see deep in an internal library that supports a binary. Whereas in go you do that sort of error handling everywhere.
Re:Down the Golang Nil Rabbit Hole by tptacek on Wednesday March 31, 2021 @02:18PM (#26650932)
I think %w makes sense in libraries, too. They're structured errors that behave like strings. It's a neat trick.
Re:Down the Golang Nil Rabbit Hole by tsimionescu on Tuesday March 30, 2021 @05:30AM (#26632640)
Most errors in Go are still just strings, potentially with some comparison possible witht sentinel values like in the "io" package. As and Is give some nice tools, but its still up to documentation to decide which one to use, and different packages have different conventions for returning error types vs error values.
Re:Down the Golang Nil Rabbit Hole by saturn_vk on Tuesday March 30, 2021 @06:36AM (#26633008)
Most errors are probably strings since there probably is nothing you can customize as a handling for such particular error types. Now, if one could do something with a certain error class, then such error are usually not strings.
Re:Down the Golang Nil Rabbit Hole by tgv on Tuesday March 30, 2021 @03:02AM (#26631876)
#1 has improved a bit lately, but I did get some mileage out of making my own type which carries an optional HTTP status, a severity level, the core error and an optional payload; and it implements Error(). It allows for easy logging, and can be used to return to the http client with a variety of messages.
Re:Down the Golang Nil Rabbit Hole by jrockway on Tuesday March 30, 2021 @05:10AM (#26632516)
How are errors opaque? If you want a FooError:

   foo := &FooError{}
   if errors.As(err, &foo) {
       fmt.Println("foo error!")
   }
Re:Down the Golang Nil Rabbit Hole by j-james on Tuesday March 30, 2021 @12:34AM (#26631286)
Lack of generics and clumsy error handling would be my guess.
Re:Down the Golang Nil Rabbit Hole by BobbyJo on Tuesday March 30, 2021 @12:47AM (#26631346)
In practice, those have honestly been the two largest assets to me. No generics means not worrying about a coworker going off the rails and creating a library where it's not needed (me being that coworker often in my past), and clumsy error handling means factoring said error handling into your APIs, making you much more deliberate about where and how errors are eaten. Having used Java extensively, those two 'warts' have honestly saved me a lot of heartache.
Re:Down the Golang Nil Rabbit Hole by somezero on Tuesday March 30, 2021 @02:22AM (#26631686)
How much do your colleagues (ab)use goto statements that Go provides? :)
Re:Down the Golang Nil Rabbit Hole by tirrex on Tuesday March 30, 2021 @03:04AM (#26631889)
C++ has goto, Java has “break <label>“, nobody uses it but you see a lot of generics abuse.
Re:Down the Golang Nil Rabbit Hole by calessian on Tuesday March 30, 2021 @06:45PM (#26641475)
I’ve seen break label a handful of times for nested loops. Whether that was the best choice is of course always up for debate. It’s definitively possible to replace and not a necessary language feature.

Maybe it’s a good thing it’s not widely known, generics are trivial to discover.

Re:Down the Golang Nil Rabbit Hole by saturn_vk on Tuesday March 30, 2021 @06:37AM (#26633017)
What can be classified as generics abuse in Java?
Re:Down the Golang Nil Rabbit Hole by BobbyJo on Tuesday March 30, 2021 @11:53AM (#26636122)
Using generics anywhere only a single type is ever used. The ambiguity really just hurts readability and creates a landmine for future engineers that think other types can be used willy-nilly and things will behave as they should. Especially if some optimizations get done underneath that assume or asserts some set of types, but doesn't enforce them at the API level.
Re:Down the Golang Nil Rabbit Hole by masklinn on Tuesday March 30, 2021 @09:05AM (#26634110)
Using java is abusive towards generics, they didn't deserve that?
Re:Down the Golang Nil Rabbit Hole by BobbyJo on Tuesday March 30, 2021 @11:35AM (#26635833)
I've literally never seen them. Not even once.
Re:Down the Golang Nil Rabbit Hole by TheDong on Tuesday March 30, 2021 @01:22AM (#26631466)
One of them has to be the for loop reusing value thing.

    func processItem(item *string) {
      if item == nil {
        fmt.Println("nil item")
        return
      }
      fmt.Println(*item)
    }
    
    stuffToProcess := []string{"one", "two"}
    wg := sync.WaitGroup{}
    for _, item := range stuffToProcess {
      wg.Add(1)
      go func(s *string) {
        time.Sleep(1 * time.Millisecond)
        processItem(s)
        wg.Done()
      }(&item)
    }
    wg.Wait()
    // What do I print?
Playground here: https://play.golang.org/p/mouvT1BkpNJ
Re:Down the Golang Nil Rabbit Hole by zimpenfish on Tuesday March 30, 2021 @03:17AM (#26631960)
gosec[1] will warn you about this, at least.

    [reuse.go:26] - G601 (CWE-118): Implicit memory aliasing in for loop. (Confidence: MEDIUM, Severity: MEDIUM)
    25:    wg.Done()
  > 26:   }(&item)
    27:  }
[1] https://github.com/securego/gosec
Re:Down the Golang Nil Rabbit Hole by JulianMorrison on Tuesday March 30, 2021 @06:33PM (#26641399)
Easily enough fixed by item := item (that is, shadow the variable with something new that won't be overwritten)
Re:Down the Golang Nil Rabbit Hole by mimischi on Tuesday March 30, 2021 @02:41AM (#26631766)
Can‘t this be fixed by changing bits to be passed by value?

https://play.golang.org/p/srAyKZN2NTH

Re:Down the Golang Nil Rabbit Hole by thwarted on Tuesday March 30, 2021 @02:55AM (#26631836)
Yes, it is odd to pass the address of a local variable from the enclosing scope like this to a go routine.
Re:Down the Golang Nil Rabbit Hole by masklinn on Tuesday March 30, 2021 @03:52AM (#26632139)
Well that's true in the sense that usually you'd just close over it, which is the classic broken case of closing over a mutable binding: https://play.golang.org/p/40FRmvhifBl

I assume GGP's contrived example is to show the underlying mechanics of the issue.

Re:Down the Golang Nil Rabbit Hole by latch on Tuesday March 30, 2021 @12:57AM (#26631389)
I think these two rightfully get highlighted, but I've found it difficult to organize complex Go code (though clearly some people manage), which I think is now my biggest issue with the language.

It's inability to deal with cyclic references, the lack of overloading, the lack of package-level visibility and the lack of static functions all contribute to a lot of ceremony.

You end up with functions like newUserWithPassword and NewRole (casing intentional), instead of User::new() and Role::new()

You could just have a NewUser that takes a NewUserOpts which exposes a fluent interface. But again, a lot of ceremony.

You can use a package per type, but still no overloading, and you'll need a 3rd package to bridge the two if the reference each other.

Re:Down the Golang Nil Rabbit Hole by kodah on Tuesday March 30, 2021 @01:02AM (#26631406)
> It's inability to deal with cyclic references, the lack of overloading, the lack of package-level visibility and the lack of static functions all contribute to a lot of ceremony.

After 4 years of Go, this is usually a design thing. Once I figured out how to separate and isolate domains of interest better, I stopped running into these issues entirely. My code, in and outside of Go, is much better for it. I also learned that much of "thread-safe" programming is taught this way, which makes sense.

Re:Down the Golang Nil Rabbit Hole by christophilus on Tuesday March 30, 2021 @06:45AM (#26633070)
F# has similar constraints, and gets similar complaints, and yet advocates (like I) think it’s a feature, not a bug. Most problems can be decomplected, and I like that Go enforces this particular constraint.
Re:Down the Golang Nil Rabbit Hole by wizhi on Tuesday March 30, 2021 @04:30AM (#26632304)
Do you possibly have any example projects you would be willing to share?

I've been getting into Go for the past couple of months, but I still struggle with this.