News for hackers. Stuff that matters.
Re:Conc: Better Structured Concurrency for Go by preseinger on Wednesday January 11, 2023 @06:51PM (#34346850)
> In practice, panics happen.

I guess this is the crux of the issue. I don't think this is true, or needs to be true. It certainly hasn't been my experience. I think assuming panics are normal will take you down some paths that make it basically impossible to write reliable software. But, to each their own.

> I'm accepting the risk that my application is left in an inconsistent state,

Inconsistent state makes it impossible to reason about your program's execution or outcomes. An account value that previously had balance = 0 may now have balance = 1000. Is this acceptable risk?

Re:Conc: Better Structured Concurrency for Go by Groxx on Wednesday January 11, 2023 @07:11PM (#34347076)
Since defers run during panics for exactly this reason, no. You can in fact guarantee that is not the case.

Runtime-safety "panics" in Go, like concurrently modifying and iterating a map that can lead to other memory being corrupted, tend to abort the whole process immediately and not be suppress-able panics.

Re:Conc: Better Structured Concurrency for Go by preseinger on Thursday January 12, 2023 @06:24PM (#34361704)
> Runtime-safety "panics" in Go, like concurrently modifying and iterating a map that can lead to other memory being corrupted, tend to abort the whole process immediately and not be suppress-able panics.

https://go.dev/doc/effective_go#panic

> The usual way to report an error to a caller is to return an error as an extra return value. . . . But what if the error is unrecoverable? Sometimes the program simply cannot continue. For this purpose, there is a built-in function panic that in effect creates a run-time error that will stop the program

Panics express unrecoverable failures. This is plainly stated in the language documentation. There are exceptions to this rule, but they are exceptional.

Re:Conc: Better Structured Concurrency for Go by Groxx on Thursday January 12, 2023 @08:34PM (#34362860)
That's a style decision, not a correctness issue. You are claiming it is a correctness issue.
Re:Conc: Better Structured Concurrency for Go by preseinger on Thursday January 12, 2023 @09:02PM (#34363067)
It is absolutely a correctness issue. Panics do not provide safety guarantees that generalize enough that it is safe to arbitrary recover from them. The statement in the previous sentence is not a subjective opinion, it's a statement of fact. I'm not sure how else to convey this information.
Re:Conc: Better Structured Concurrency for Go by Groxx on Friday January 13, 2023 @02:46AM (#34365011)
Panics do not violate any runtime guarantees, and defers run in the presence of panics.

All safety guarantees possible if there were no panics are possible with.

Re:Conc: Better Structured Concurrency for Go by preseinger on Friday January 20, 2023 @01:01PM (#34457022)
When some bit of code invokes `panic` it is saying that there is an error which is unrecoverable, and the default expectation is that the process will terminate. There is no way to assert that panics do not violate runtime or memory model expectations. They can.
Re:Conc: Better Structured Concurrency for Go by groestl on Wednesday January 11, 2023 @07:18PM (#34347143)
> assuming panics are normal will take you down some paths that make it basically impossible to write reliable software

Na, citation needed. Assuming "panics are normal" is just extrapolating from "errors are normal". It makes reliable software more reliable.

Re:Conc: Better Structured Concurrency for Go by bsaul on Thursday January 12, 2023 @05:20AM (#34351544)
it's pretty obvious that it could influence new developers into the wrong direction though. Saying things like "ha, let's not bother checking this, at worst it'll just panic and i'll simply abort the request".

Which would definitely impact the quality of the software overall in a bad way.

Re:Conc: Better Structured Concurrency for Go by groestl on Thursday January 12, 2023 @05:37AM (#34351649)
I'd not be so sure. Accepting that everything that can fail will fail shaped me as a young developer, and "Exceptional C++" had a huge influence on me. Now my approach for new code I review is this:

* Make sure you support properly unrolling the stack

* Keep a clean failure boundary, probably somewhere on top of your loop

* Fastidiously check your preconditions

* Fail brutally if they're not met

* Improve from there

Re:Conc: Better Structured Concurrency for Go by preseinger on Thursday January 12, 2023 @05:51PM (#34361388)
Right, all of these are good points, but the problem is that the "failure boundary" of a panic is the entire process. You can't constrain it, or assume that it's scoped to a single goroutine. Errors do not have this property.
Re:Conc: Better Structured Concurrency for Go by groestl on Sunday January 15, 2023 @03:04AM (#34387658)
> the "failure boundary" of a panic is the entire process.

This is trivially falsifiable by panicking yourself and immediately recovering. Neither failure domain nor failure boundary need to align with the entire process.

Re:Conc: Better Structured Concurrency for Go by preseinger on Tuesday January 17, 2023 @04:59PM (#34419984)
The impact of a specific panic does not extrapolate to the impact of all panics, and my claim is not falsified by such an example. Panics are defined by the language to represent unrecoverable errors.

    func (x *Thing) Method() {
        x.somethingThatPanics()
        x.somethingThatAssumesTheAboveDidntPanic()
    }
Recovering from a panic thrown by Method invalidates the state of the Thing which threw that panic. If that Thing is shared among concurrent actors, the entire program state is invalidated.
Re:Conc: Better Structured Concurrency for Go by groestl on Tuesday January 17, 2023 @06:45PM (#34421175)
> If that Thing is shared among concurrent actors

You're adding preconditions to your claim.

> the entire program state is invalidated.

No, the state represented by a connected graph of variables accessible by the concurrent actors is tainted. This is hardly "the entire program state". Often it's just a few cache entries.

Also, see my first, most important, bullet point:

"* Make sure you support properly unrolling the stack."

Which means a request to Thing errored out, but it never enters an invalid state. If you fail at that, all bets are off. But then you're dealing with a mediocre codebase anyway.

And finally, let me rewrite your example to something, that I see much more often in real life code, which problematic _even without concurrent actors_ because somethingThatAssumesTheAboveDidntReturnAnError might do horrible things all by themself:

func (x *Thing) Method() {

x.somethingThatReturnsAnError()

x.somethingThatAssumesTheAboveDidntReturnAnError()

}

Re:Conc: Better Structured Concurrency for Go by preseinger on Wednesday January 18, 2023 @07:03AM (#34425728)
I'm not sure how to respond to this. You seem to believe that panics express problems which are constrained to the call stack which instantiated the panic. This isn't true. But I'm not sure how to express this to you in a way that will convince you. So I guess we're at a stalemate.
Re:Conc: Better Structured Concurrency for Go by groestl on Wednesday January 18, 2023 @09:07AM (#34426709)
> You seem to believe that panics express problems which are constrained to the call stack which instantiated the panic.

Not inherently, but it's your job as a developer to make sure this is the case, that's what:

"* Make sure you support properly unrolling the stack."

means.

In case you're dealing with unknown code it's your job to find out what the connected graph of potentially tainted objects is and discard them. That's what "keep a clean failure boundary" means.

If you can't, because you don't want to (short lived process, prototypes) or are unable to (hairy ball of code), tearing down the process is indeed the only option and a sane fallback choice made by the language designers. But it's not necessarily a hallmark of robust software.

I hope I had a final shot to clear up what I meant, thanks for the discussion anyway.

Re:Conc: Better Structured Concurrency for Go by preseinger on Friday January 20, 2023 @12:58PM (#34456978)
In Go when some code writes `panic` it is expressing an error condition which should not be intercepted by callers and is expected to terminate the process. A panic is not an error, and panics should not be recovered as if they were errors.
Re:Conc: Better Structured Concurrency for Go by preseinger on Thursday January 12, 2023 @05:47PM (#34361352)
Panics are categorically different than errors. Errors are normal, panics are not normal.
Re:Conc: Better Structured Concurrency for Go by jasonhansel on Wednesday January 11, 2023 @07:49PM (#34347496)
> An account value that previously had balance = 0 may now have balance = 1000. Is this acceptable risk?

Your entire web app process crashes due to a panic every time a request triggers an extremely rare edge case. A hacker discovers this and uses it to conduct a DoS attack. Is this acceptable risk?

Re:Conc: Better Structured Concurrency for Go by ikiris on Wednesday January 11, 2023 @08:56PM (#34348042)
Why the heck are you writing web apps that panic?
Re:Conc: Better Structured Concurrency for Go by aetherane on Wednesday January 11, 2023 @09:36PM (#34348439)
It is pretty easy to have accidental panics in Go, for instance due to a runtime assertion that unexpectedly failed
Re:Conc: Better Structured Concurrency for Go by preseinger on Thursday January 12, 2023 @06:27PM (#34361737)
Runtime assertions without defensive checks are programmer errors that are not difficult to spot in code review and should not be expected to make it to deployed code.

    // RED FLAG
    x := y.(type)

    // good
    x, ok := y.(type)
    if !ok { return an error }
Re:Conc: Better Structured Concurrency for Go by jasonhansel on Wednesday January 11, 2023 @09:00PM (#34348092)
Because people make mistakes?
Re:Conc: Better Structured Concurrency for Go by philosopher1234 on Thursday January 12, 2023 @01:00AM (#34349762)
Classic Go programmer. This is why I use rust B)

(joke)

Re:Conc: Better Structured Concurrency for Go by unscaled on Thursday January 12, 2023 @04:04AM (#34350934)
Joking aside, you could clearly plot the probability of running into a runtime error by programming language.

Of course, a language with less runtime errors is a far cry from being a panacea. Avoiding runtime errors is not the same as avoiding all categories of bugs. And while I personally prefer stronger type systems - they definitely come with increasing levels of cognitive cost.

But I still feel that the type-safety vs. runtime trade-off is more often ignored, underestimated or undersold than it is being hyped. Yes, certain languages (cough Rust cough) are being hyped, but not the conscious choice of balancing programmer learning curve with runtime type-safety.

And while on the topic of Rust, it's probably not the best choice for a language that sees less runtime panics. Especially since unwrapping an error is always the easiest way to handle an error, and thus quite common. But lazy error unwrapping aside, Rust does avoid null dereference exceptions, type casting exceptions and most types of race conditions that can be quite prevalent with go[1].

[1]: https://songlh.github.io/paper/go-study.pdf

Re:Conc: Better Structured Concurrency for Go by unscaled on Thursday January 12, 2023 @03:45AM (#34350798)
This is equivalent to asking "Why the heck are you writing code with bugs?"

Sure, if we could write code without bugs, we wouldn't need to suppress panics. But since we do tend to write code bugs and some of them are bugs that can be detected by the runtime - we get panics.

If you hate panics, you can do better than Go and go for a language with a stronger type system, where you won't get nil pointer panics or interface conversion panics, but even an almost onerously-tyepesafe language like Haskell still panics on some bogus operations such as division by zero or trying to read the head of an empty list. Perhaps Idris really have no runtime errors but they are quite niche.

Re:Conc: Better Structured Concurrency for Go by preseinger on Thursday January 12, 2023 @05:52PM (#34361394)
Yes, definitely preferable! Denial of service is definitely better than invalid state, right?
Re:Conc: Better Structured Concurrency for Go by yakaccount4 on Thursday January 12, 2023 @01:39AM (#34349992)
> I guess this is the crux of the issue. I don't think this is true, or needs to be true. It certainly hasn't been my experience. I think assuming panics are normal will take you down some paths that make it basically impossible to write reliable software. But, to each their own.

I'd rather have the control to log the panic on a service rather than it forcibly dying and taking down any other connections with it. Kube will just spin up a new one anyway, which just introduces a downtime gap that doesn't need to exist.

Re:Conc: Better Structured Concurrency for Go by preseinger on Thursday January 12, 2023 @05:50PM (#34361378)
I don't think I'm effectively communicating the impact of handling a panic and continuing program execution. A panic that comes from a memory model violation (as one example) can change the value of anything in the memory space of the program. If the program continues, that change will go undetected, and can have results that make the program completely nondeterministic. This isn't a doom and gloom, sky-is-falling prognostication, it's literally what is defined by the spec and memory model of the language.
Re:Conc: Better Structured Concurrency for Go by yakaccount4 on Thursday January 12, 2023 @09:18PM (#34363165)
> A panic that comes from a memory model violation (as one example) can change the value of anything in the memory space of the program ... This isn't a doom and gloom, sky-is-falling prognostication, it's literally what is defined by the spec and memory model of the language.

I do not think you are correct. Go has a class of unrecoverable panics for this specific reason. Go also runs deferred functions after a recoverable panic, so the notion that it's unsafe to handle it, or continue executiona after doesn't hold at all - it is literally a first-class feature of the language.

I have not seen an instance of a recoverable panic that is raised _after_ such a fatal operation. If you have an example of such, I would love to see it.

Re:Conc: Better Structured Concurrency for Go by preseinger on Thursday January 12, 2023 @09:29PM (#34363245)
What are unrecoverable panics vs. recoverable panics? Where is that distinction defined?
Re:Conc: Better Structured Concurrency for Go by yakaccount4 on Thursday January 12, 2023 @10:00PM (#34363461)
There seems to not be any standard list of unrecoverable panics/aborts, but this Stackoverflow post [1] has a list of a few.

As far as the user/developers are concerned, it doesn't matter too much, since you have no option to recover them, but it would be nice if it was explained if defers are still ran. I'm assuming they are not.

1. https://stackoverflow.com/questions/57486620/are-all-runtime...

Re:Conc: Better Structured Concurrency for Go by preseinger on Thursday January 12, 2023 @10:14PM (#34363545)
If there is no way for callers to reliably distinguish recoverable panics from unrecoverable panics, then this distinction doesn't really exist, does it? Panics are panics.
Re:Conc: Better Structured Concurrency for Go by yakaccount4 on Thursday January 12, 2023 @10:32PM (#34363639)
I'm not sure what point you are trying to make anymore.

Of course you cannot distinguish between unrecoverable and recoverable panics, because by definition an unrecoverable panic is not recoverable. There is no caller to distinguish between it - it is killed.

Re:Conc: Better Structured Concurrency for Go by preseinger on Friday January 13, 2023 @12:43AM (#34364358)
Oh. You're using the word panic to describe a superset of actual panics and other even more serious errors. Those things you call unrecoverable panics are not actually panics.

The point I'm trying to make is that panics are not errors by another name, and they are not safe to recover from in general.