News for hackers. Stuff that matters.
Re:Fourteen Years of Go by beltsazar on Saturday November 11, 2023 @06:29AM (#38229366)
There are a few surprises of Go, but one thing that surprised me the most is the fact that an unrecovered panic in any goroutine will crash the whole program. To make things worse, you can only recover a panic in a goroutine that you spawn. So, you can't do anything to prevent a panic in a goroutine spawned by a third-party library crashing your whole program. The best you can do is to set up a supervisor to restart your program after crash.
Re:Fourteen Years of Go by galangalalgol on Saturday November 11, 2023 @06:51AM (#38229504)
Rust made the same call. Panic isn't supposed to be "I'm not sure what to do now, maybe my caller will know?". That is an error not a panic amd you return it. A panic is for "I just realized I or someone else did something really bad and the invariants of everything are now in question, hit the breakers, lets try this again". If your dependency library, like so many, panicked rather than take the time and effort to check the conditions that would cause an out of bounds access, then they are bad and they should feel bad, fork and merge the fix and don't say anything, they will feel the shame.
Re:Fourteen Years of Go by beltsazar on Saturday November 11, 2023 @07:17AM (#38229677)
> Rust made the same call.

Completely different. In Rust when you spawn a thread [1] or an async task [2], you'll get a handle from which you can check if a panic happens inside the thread/task. Even if you ignore the handle, a panic inside a thread/task won't crash the entire program.

---

[1] https://doc.rust-lang.org/std/thread/fn.spawn.html

[2] https://docs.rs/tokio/latest/tokio/task/fn.spawn.html

Re:Fourteen Years of Go by dinosaurdynasty on Saturday November 11, 2023 @11:34AM (#38231741)
Yeah I've been hit by this, I've setup a panic handler to purposefully kill the entire process because otherwise long-running threads die and don't restart (and it was easier to do it that way).
Re:Fourteen Years of Go by quickthrower2 on Saturday November 11, 2023 @06:55AM (#38229527)
Running off forks is no fun though.
Re:Fourteen Years of Go by hackerbrother on Saturday November 11, 2023 @07:14AM (#38229663)
So the question is: should there exist a method that means “bring down the whole ship”. (Or, should that method be something more severe than “panic”, and panic should be redefined with a more limited scope.)
Re:Fourteen Years of Go by quickthrower2 on Saturday November 11, 2023 @06:19PM (#38235599)
I agree. I like C# where you can expect libraries to play nice. In theory it is possible for C# library to crash your app but it is definitely not on the trodden path.

But I do prefer return values over exceptions! I have not used Go and I think the err != nil boilerplate might drive me a little insane, but Haskells option types seem a nice fit.

Re:Fourteen Years of Go by zoky on Saturday November 11, 2023 @07:48AM (#38229869)
> Panic isn't supposed to be "I'm not sure what to do now, maybe my caller will know?".

So panic is supposed to be “I’m not sure what to do now, I don’t care if my caller knows”?

No process running outside of kernel space ever has any business panicking. Throw an error and move on with your life (or don’t, that’s fine too) but don’t punish your caller by forcing it to crash because you failed to foresee X, Y, or Z circumstances.

Re:Fourteen Years of Go by usrbinbash on Saturday November 11, 2023 @08:04AM (#38229973)
> Throw an error and move on with your life

No, absolutely not!

Panics are the result of conditions that absolutely should crash the program! They are not errors, they are not similar to exceptions, they are conditions that should not occur, and thus if they do occur, either the program itself, or the platform it is running on, has a problem that needs to be solved before the program runs again.

If a cars fuel is running low, it should turn on a warning light and inform the driver that he should find a gas station. If a piece of the road is suddenly missing, the car needs to stop.

If a panic occurs, I don't WANT the program to continue executing! That would lead me right back into C land, where an out-of-bounds memory access would silently start to poison my program and crash it days after the actual error ocurred, if i'm lucky! If I'm unlucky, and the process needs to be sufficiently privileged, it would kill the server, again several days (or weeks) later, with slim-to-no chance for me to figure out what even went wrong.

Re:Fourteen Years of Go by iainmerrick on Saturday November 11, 2023 @08:16AM (#38230051)
Why would there be a "recover" function, then?

A quick google turns up https://go.dev/blog/defer-panic-and-recover, which says:

For a real-world example of panic and recover, see the json package from the Go standard library. It encodes an interface with a set of recursive functions. If an error occurs when traversing the value, panic is called to unwind the stack to the top-level function call, which recovers from the panic and returns an appropriate error value

That's panic and recover used for control flow, just like might use exceptions in other languages.

Looks like panic has at least two purposes, depending on whether you recover or not?

Re:Fourteen Years of Go by usrbinbash on Saturday November 11, 2023 @08:27AM (#38230110)
> A quick google turns up https://go.dev/blog/defer-panic-and-recover, which says:

And another quick googling turns up this:

https://github.com/golang/go/issues/26799

which says:

    However, this is a poor example and a misuse of panic and recover as exceptions.
    See https://golang.org/doc/effective_go.html#panic

    The example will be invalid soon with the release of go 1.11 as well. The
    panic/recover was removed from the unmarshal code. See master's
So no, panic and recover are not meant to be used as control flow. Yes, they CAN be used that way. I also CAN use pythons `eval` a lot, use a scripting language as my command line interpreter, or make an apple-pie with onions and garlic.

As for why `recover` exists in the first place: Packages should have the ability to stop internal panics. There are various reasons for that, some worse than other. For example it is probably okay if a package doing sensor readouts recovers from a division by zero and replaces it with an error saying that the sensor value makes no sense.

Re:Fourteen Years of Go by iainmerrick on Saturday November 11, 2023 @11:00AM (#38231444)
Aha, good find, thanks! Maybe they need to work on their SEO. :)
Re:Fourteen Years of Go by zoky on Saturday November 11, 2023 @08:10AM (#38230018)
And why exactly do you trust a called method to determine whether or not you should be able to keep running? Like… do your job, you object. I didn’t load your library and call a method on you to tell me what the state of my system is, I just want you to either do what I asked you to do or fail gracefully, and then I can decide whether we crash or not.

> If a piece of the road is suddenly missing, the car needs to stop.

Uh, no. If a piece of the road is missing, it’s up to the driver to stop, not the car. The car has no business whatsoever determining whether the road exists or not, except inasmuch as it communicates to the driver “hey, whatever the heck this is is difficult or impossible to drive on, your call what we do though.”

Re:Fourteen Years of Go by zozbot234 on Saturday November 11, 2023 @08:20AM (#38230071)
> I just want you to either do what I asked you to do or fail gracefully, and then I can decide whether we crash or not.

Then you're looking for resumable conditions, which just so happen to be effectively isomorphic to async-await. (The called function is effectively a piece of async code which 'awaits' upon a failure condition, after which it can either be resumed or canceled as with an ordinary panic.)

Re:Fourteen Years of Go by usrbinbash on Saturday November 11, 2023 @09:05AM (#38230369)
> just want you to either do what I asked you to do or fail gracefully

And most of the time, this is EXACTLY what happens. In fact most libraries that do use panics, recover them internally and return errors to the callers.

Again: panics are not errors, panics are not exceptions. A library should never panic in a way that it's caller notices, UNLESS it met a condition that simply is not recoverable, not by the library itself, and not by it's caller.

Yes, this requires care by the libraries authors not to overuse panics. Yes, there are libraries that do overuse them. No, this doesn't change anything about what I wrote. If a library I use would do that, I would either fix the library, or find a better one.

> Uh, no. If a piece of the road is missing

Okay, let me try again with a better analogy:

If a cars engine breaks down, the car will stop. It doesn't matter if the driver wants the car to go for another 100 kilometers. It stops. Period. Not because anyone or anything "wants" the car to stop, but because its engine "package" met a condition, under which the further running of whatever context it is used in (in this case, the car), is no longer feasible and/or possible.

That is exactly what panicking in a way that hits the runtime communicates: Oh noes, the engine just broke down, and now the machine can no longer run.

Re:Fourteen Years of Go by zoky on Saturday November 11, 2023 @07:42PM (#38236106)
I’m not sure how valuable these analogies actually are, but your example of the broken engine really seems to go towards my point. If the car breaks down, the driver still has several options: coast downhill, switch to a different car, pull a bicycle out of the trunk, or just get out and continue on foot. But the car enforcing the end of the trip by simply exploding and killing the driver isn’t really an acceptable response to engine failure.

A library that unrecoverably crashes your program for any reason is defective by design, and a programming language that purposely allows a library to do so as a design choice is also defective by design in my opinion.

Re:Fourteen Years of Go by usrbinbash on Sunday November 12, 2023 @07:47AM (#38239649)
> If the car breaks down, the driver still has several options:

The one option he doesn't have though: Continue running the car as-is. And that's the point I am making.

> and a programming language that purposely allows a library to do so as a design choice is also defective by design in my opinion.

You are entitled to your opinion of course, but I'm not going to agree, and it seems a lot of people are completely fine with that design decision: https://madnight.github.io/githut/#/pull_requests/2023/3

Again, the library isn't "crashing the program". The library meets a condition under which it can no longer function, no matter what the caller does, and can no longer function in a way that makes running the entire program infeasible. That is the point of panicking and not recovering it within the library.

And yes that is the libraries call to make. Yes, it needs to have a very good reason to make that call. Very few libraries panic in that way. And the ones that do, and are widely used, have very good reasons to. If libraries do so without a good reason, then people stop using them.

Re:Fourteen Years of Go by zoky on Sunday November 12, 2023 @09:26AM (#38240318)
> The one option he doesn't have though: Continue running the car as-is. And that's the point I am making.

Exactly! So raise an EngineFailure exception and let the driver decide how to proceed. But don’t kill the driver because you can’t figure out how anyone could possibly recover from what seems to you like the end of the world, when it might just be a speed bump as far as the driver is concerned.

> You are entitled to your opinion of course, but I'm not going to agree, and it seems a lot of people are completely fine with that design decision: https://madnight.github.io/githut/#/pull_requests/2023/3

I mean, a lot of people are fine with programming in PHP, that doesn’t make it a good choice.

> And yes that is the libraries call to make.

No. Absolutely, 100%, beyond question not. The library can’t possibly know every conceivable circumstance in which it is called, and therefore it is a guarantee that there is some conceivable set of circumstances from which what appears to be a fatal error from the library’s perspective is nothing more than a blip, or even a complete non-issue, from the perspective of the caller. Panicking the calling program so it can’t possibly continue is an abuse of trust, and any library that does so shouldn’t be admitted to any repo, and any language that allows it to do so is broken by design and should be considered harmful.

Re:Fourteen Years of Go by usrbinbash on Sunday November 12, 2023 @12:09PM (#38242036)
> Exactly! So raise an EngineFailure exception and let the driver decide how to proceed.

In the analogy, the car is the program. The ENTIRE program. It is the limit of the analogy. The car driving is the program running. The car no longer being able to drive, is the program not running. Everything outside of that is out of the scope of the analogy.

> I mean, a lot of people are fine with programming in PHP, that doesn’t make it a good choice.

That is absolutely true, but it doesn't make PHP "defective". If someone is unhappy with how Go's `panic` works, they can use something else, same as I am not going to use PHP.

> No. Absolutely, 100%, beyond question not.

Yes, absolutely, 100%, beyond question it is.

The library is an implementation hidden behind a public interface. Same as I don't have to worry about how it performs it's functions, I don't have to worry how it determines what it considers to be an irrecoverable failure condition.

If I disagree with a libraries decision on that, I have the same options I have if I disagree with how it's interface works (assuming it is open source): 1) I can fork it and roll my own implementation. 2) I can vendor-copy it and adapt it locally for the scope of that project. 3) I can use another lib (including writing my own from scratch).

> The library can’t possibly know every conceivable circumstance in which it is called

It doesn't have to. If the engine breaks down, it does't matter if the car is yellow, the sun is up, or the driver was born on a Tuesday. The engine is broken, and cannot run. End of story.

If there are conceivable circumstances under which the lib should continue running instead of panicking, it is the libs job to implement it, same as it is its decison what these circumstances are, and when they apply.

> and any language that allows it to do so is broken by design and should be considered harmful.

Well, I think we are gonna have to agree to disagree on this. Considering Go's success and continuous growth, it is pretty safe to say that many people and industries don't consider it broken, nor harmful.

Re:Fourteen Years of Go by xyzzy_plugh on Saturday November 11, 2023 @08:02AM (#38229956)
If it's reasonable for the caller to continue then return an error. If the state of the world is irreconcilable, panic is the right call.
Re:Fourteen Years of Go by zoky on Saturday November 11, 2023 @08:05AM (#38229984)
And how is the called method supposed to be able to determine whether the state of the world is irreconcilable?

For that matter, how is the called method supposed to determine what the state of the world even is? And why on Earth would we trust it to do so? Is encapsulation not even a thing anymore? I really just don’t understand.

Re:Fourteen Years of Go by bogeholm on Saturday November 11, 2023 @08:30AM (#38230134)
A classic example is:

- Accessing array out of bounds: panic

- File not found: error

Re:Fourteen Years of Go by nishs on Saturday November 11, 2023 @09:18AM (#38230474)
There are two "kinds" of panics: those initiated by the Go runtime, and those initiated by the program author, though technically there isn't a distinction.

When you access an array out of bounds, the Go runtime initiates the panic. On the other hand, for a condition that the program should hold but doesn't, the program author should initiate the panic. For example, in the exhaustive switch below, reaching the default case means that either you have somehow incorrectly allowed the construction of an invalid token value earlier in your program, or you have missed to account for a valid token value in this function.

  func (t token) symbol() byte {
          switch token {
                  case add: return '+'
                  case sub: return '-'
                  case mul: return '*'
                  case quo: return '/'
                  default:  panic("unknown token")
          }
  }
 
Errors are different.

For example, url.Parse returns an error if its input string is an invalidly-formatted URL.

  package url
  func Parse(rawURL string) (*URL, error)
Re:Fourteen Years of Go by galangalalgol on Saturday November 11, 2023 @09:41AM (#38230659)
That is a great example of a reason to panic. There is nothing you can do at that point but return an error or panic, and something has really gone wrong if you are pulling bad tokens out of a hashmap you made yourself, tonthe point I think you should panic. Sadly, almost all the panics I see coming out of rust crates are because someone called unwrap instead of handling the error case.
Re:Fourteen Years of Go by zoky on Sunday November 12, 2023 @09:53AM (#38240550)
That is a terrible example of a reason to panic. If the programmer believes the list to be exhaustive and it isn’t, then the programmer is wrong. So raise an exception and let the caller decide what to do with it. But don’t punish the caller by forcing it to crash just because you made a mistake as a programmer.
Re:Fourteen Years of Go by nishs on Monday November 13, 2023 @10:30AM (#38251307)
(in go, "raise an exception" means "return an error.") error values are meant to represent expected unsuccessful conditions, internal properties of the program, during the program's execution. errors in go---and exceptions in other languages---are not meant to represent mistakes by the package author. in go these are panics. but perhaps other language reuse exceptions for both.

put another way: how is the non-exhaustiveness of the switch statement qualitatively different from an out-of-bounds array access? they both are equivalent package author mistakes.

the java runtime raises IndexOutOfBounds exceptions for bad array accesses, but do each function call or your program with an IndexOutOfBounds exception handler?

Re:Fourteen Years of Go by masklinn on Saturday November 11, 2023 @07:56AM (#38229908)
> Rust made the same call.

It did not though?

- threads implicitly catch_panic at their boundary, and convert that to a `Result` which you can receive when you join on the thread: https://doc.rust-lang.org/std/thread/type.Result.html

- in theory an async runtime could shut down on panic, but tokio at least does not, much like `std::thread::spawn` it'll catch_panic and return a JoinError to however is await-ing the joinhandle: https://docs.rs/tokio/latest/tokio/task/struct.JoinError.htm...

If you drop the handles, the error just is not signaled:

- sync: https://play.rust-lang.org/?version=stable&mode=debug&editio...

- async (tokio): https://play.rust-lang.org/?version=stable&mode=debug&editio...

Re:Fourteen Years of Go by galangalalgol on Saturday November 11, 2023 @09:57AM (#38230825)
Yeah I think I remember that being a controversial decision when it was made for threads? One of the valid reasons for a panic is that you've encountered something that makes you think your adress space is in a bad state. Threads share an address space, so the reasoning was that if they panic, so should the process, but that isn't what happened.
Re:Fourteen Years of Go by DangitBobby on Saturday November 11, 2023 @10:06AM (#38230902)
Then why does recover() even exist? Programs seem to continue just fine after a recover.
Re:Fourteen Years of Go by usrbinbash on Saturday November 11, 2023 @07:59AM (#38229931)
> To make things worse,

How is a panic crashing a program a bad thing?

Panics are not Exceptions. They usually are the result of something happening that should crash the program.

Re:Fourteen Years of Go by beltsazar on Saturday November 11, 2023 @09:12AM (#38230418)
Yes, "usually", but not always. That's why Go has a mechanism to handle a panic with the `recover` function. The issue is more on the inability of handling a panic in a "foreign" goroutine.
Re:Fourteen Years of Go by eloisant on Saturday November 11, 2023 @08:13AM (#38230030)
But you should have a supervisor anyway, if you want your service to be always up.

Unless you're working on a CLI tool that is short lived anyway (and represents a lot of projects using Go).

Re:Fourteen Years of Go by goldinfra on Saturday November 11, 2023 @08:57AM (#38230308)
Sounds like you may be confused thinking that a panic in Go is like throw in other languages? Third-party libraries should virtually never panic. Go has panic, and it's useful for a few things, but does not use them as exceptions are used in other languages.

This is a bit like worrying that your programming language won't prevent a third-party library from sending a SIGKILL to your program or that it doesn't prevent third-party libraries from delete your entire file system.

No general purpose language protects against malicious or stupid code.

Re:Fourteen Years of Go by mbivert on Saturday November 11, 2023 @09:40AM (#38230654)
> but does not use them as exceptions are used in other languages

IIRC, the Go lexer/parser itself relies on panic/recover to handle errors[0], in an exception-like fashion. Granted, it's one unusual case, where the code is split in a multitude of mutually recursive functions.

[0]: https://github.com/golang/go/blob/master/src/go/parser/parse...

Re:Fourteen Years of Go by beltsazar on Saturday November 11, 2023 @09:32AM (#38230586)
> Sounds like you may be confused thinking that a panic in Go is like throw in other languages?

Not at all. I knew that panics in Go shouldn't be used for error handling in general.

> Third-party libraries should virtually never panic.

True, as true as 3rd-party libraries shouldn't have any security vulnerabilities. But we don't live in a perfect world. Bugs happen.

> This is a bit like worrying that your programming language won't prevent a third-party library from sending a SIGKILL to your program or that it doesn't prevent third-party libraries from delete your entire file system.

Very different. Your examples are intentional and malicious actions. What usually happens is due to bugs.

> No general purpose language protects against malicious or stupid code.

Are you implying that all bugs are caused by either malicious code or stupid code? Panic-related bugs are often due to the billion dollar mistake [1] that Go's team refused to solve.

---

[1] https://www.infoq.com/presentations/Null-References-The-Bill...

Re:Fourteen Years of Go by jamescostian on Saturday November 11, 2023 @09:31AM (#38230573)
I've seen apps break in prod where third party libraries had array accesses on indices that didn't exist, or dereferenced nil pointers, in goroutines they spawned. And those aren't the only ways to cause a panic without ever calling the panic function.

Third party libraries have bugs sometimes, and other times our usage of them can create bugs. Nothing like a SIGKILL or anything malicious. That's just the way things are and will be. And breaking down like this in dev can be fantastic!

But when a once in a blue moon issue crops up in prod and all other connections are dropped, the fact that a simple defer+recover couldn't have been preemptively placed on all goroutines is a huge nightmare. I'd even take a "you can add defer+recover for any goroutines created further in this callstack" - it's rare that libraries spawn goroutines, I'll already know which ones to trigger that behavior on

Re:Fourteen Years of Go by beltsazar on Saturday November 11, 2023 @09:42AM (#38230684)
> But when a once in a blue moon issue crops up in prod and all other connections are dropped, the fact that a simple defer+recover couldn't have been preemptively placed on their goroutines is a huge nightmare

Yeah, this is one of the weird things, to say it mildly, that Go does that is hardly mentioned anywhere. Most people don't know about it until they themselves first encounter the nightmare bug similar to what you said.

Re:Fourteen Years of Go by goldinfra on Monday November 13, 2023 @11:20AM (#38251936)
Wrapping every third-party library in defensive code doesn't make a lot of sense since this isn't the most dangerous kind of bug. To make a one example of a million: a third-party database library that corrupts your database is infinitely more dangerous and that there's no form of wrapping that will save you.

Bad code is bad code. Bugs are bugs. This issue is not a major source of problems in Go code.

Re:Fourteen Years of Go by Mawr on Saturday November 11, 2023 @11:45AM (#38231874)
A panic means your program has entered an invalid state. Continuing to run an arbitrarily broken program would be reckless, so crashing is the only option.

Even if you could somehow prevent your program from crashing internally, it could be killed at any time by the environment it's running in, up to the cleaning lady tripping on the power cable.

That's why it doesn't make sense to try to prevent crashes. Instead, programs should be capable of resumption irrespective of when they were killed. Resumption should be facilitated by supervisors layered on top of the program, as needed.

Re:Fourteen Years of Go by Too on Saturday November 11, 2023 @02:02PM (#38233373)
Panics are definitely one of the bigger design mistake of go. They should have either removed panics entirely and made error as return values the only way to do signal errors or they should have made defer+recover more mainstream like exceptions.

Current state of affairs is that third party libraries panic where they shouldn’t, programs don’t defer where they should, recovers cause deadlock due to non deferred mutex releases, that assumed panics are panics and not exceptions. On top of that we have to live with the verbose error handling that still isn’t water tight.