News for hackers. Stuff that matters.
Re:Speeding Up the Rust Compiler by pornel on Wednesday December 11, 2019 @07:12AM (#21761716)
`unwrap()` is "handling" errors by crashing the program, so it shouldn't really be used. It's common in examples, because it's usually the shortest/laziest code possible. However, all practical uses of `.unwrap()` have better alternatives, e.g. instead of

    if foo.is_some() {
       let unwrapped = foo.unwrap();
    }
you write:

    if let Some(unwrapped) = foo {
    }

There are plenty of combinators like .map_or(), .ok_or()?, .filter_map() that deal with optional values gracefully.
Re:Speeding Up the Rust Compiler by burntsushi on Wednesday December 11, 2019 @08:36AM (#21762169)
> `unwrap()` is "handling" errors by crashing the program, so it shouldn't really be used

This isn't the right advice to give, and is only going to confuse beginners. Panicking via unwrap/expect is perfectly fine, whether in a library or an application, only when the panic represents a bug. For example, accessing an element of a Vec with an incorrect index because of a logic error. If a panic occurs, then it should reflect a bug that is intended to be fixed.

Stated differently, it should be impossible for an end user to cause a Rust application to panic. If they can, then it's a bug.

This advice permits use of unwrap/expect anywhere, so long as its occurrence is never expected.

Re:Speeding Up the Rust Compiler by bluejekyll on Wednesday December 11, 2019 @09:42AM (#21762749)
I mostly agree with you, but in practice I think using unwrap or panicking in libraries is usually wrong.

As an example it’s quite easy to fall astray of taking your perspective to say read a packet off the network, and panic on malformed data. I know you’d agree that would be a inappropriate time to use panic, as that would crash any upstream program with trivial DOS exploits. But this is easy to do if you say have an API that takes a value, translates it to something else and panics on invalid inputs.

My advice is generally in line with the GP, pornel—use the tools at hand for avoiding panics where it’s easy to do so, such as ?, map, etc. Only panic/unwrap on bad library API usage (which I think is what you’re suggesting). Feel free to panic anywhere in main.

People often translate “ use of unwrap/expect anywhere, so long as its occurrence is never expected” in ways that can create major bugs as the software goes into more use.

Re:Speeding Up the Rust Compiler by burntsushi on Wednesday December 11, 2019 @09:58AM (#21762917)
I understand that perspective to an extent, but I really want to double down here because I think it's important. :-)

> but in practice I think using unwrap or panicking in libraries is usually wrong

Very strongly disagree. I've created dozens of Rust libraries, and probably all of them have dozens of code paths that can panic. Other core libraries do the same. Just a simple slice access, e.g., `slice[i]`, is a line of code that could panic. (Since it's just a shorter way of writing `slice.get(i).unwrap()`.) I don't think we should be giving advice that runs contrary to how important libraries actually work. A lot of people learn to code by reading others' code, and when we give advice like "don't use unwrap/expect," they get confused when they see that virtually every widely used piece of Rust code violates it.

The key here is really that one shouldn't panic unless the panic itself is indicative of a bug somewhere. Blanket advice like "don't use unwrap/panic in libraries" is bad because---as I argued in the Clippy issue I posted in a sibling comment---it's effectively a prohibition against runtime invariants themselves. As a programmer, you insert a panic when you've failed (for any number of reasons) to capture the invariant in the type system. Blanket advice saying that one shouldn't use unwrap/expect in these circumstances leads one toward a path of much more complex APIs with error types that are never constructed unless there is a bug in the code. (It's likely that actually adding all of those error types will be so annoying that it's plausible the programmer will give up on Rust.)

> As an example it’s quite easy to fall astray of taking your perspective to say read a packet off the network, and panic on malformed data. I know you’d agree that would be a inappropriate time to use panic, as that would crash any upstream program with trivial DOS exploits. But this is easy to do if you say have an API that takes a value, translates it to something else and panics on invalid inputs.

I think this is a reflection of error handling being hard. It's just as easy to code a program with the mistaken assumption that a particular file path will always point to a valid and readable file. It takes a bit of learning to understand which things you can rely on never happening and which you can't. In the mean time, I don't think we should be giving advice that is very easy to misinterpret into a conclusion that just isn't tenable. Personally, I think it's much easier to talk about end user behavior. If there's a panic, then it's a bug that ought to be fixed. If you follow that, it's hard to go wrong.

You could rephrase it with, "If there's a reachable code path that panics, then the code path should be removed." You could then talk about what "reachable" means, i.e., code paths that are determine based on data that the program doesn't control vs code paths that are enforced for all inputs to the program.

Re:Speeding Up the Rust Compiler by Tainnor on Wednesday December 11, 2019 @02:10PM (#21765708)
I unfortunately know very little about Rust, but is it possible to recover from panics in Rust?

Because the problem that I'm seeing in Swift code (which also has a "crash on logic errors" approach) is that bugs can bring your whole system down, which is especially bad on server-side apps with multiple threads. Yes, you can use supervisord and/or load balancers, but you still lose in-flight requests.

By contrast, in a language with a runtime like Java or Ruby, you can catch almost everything at the top level, so you could just have some logic that generates a "whoops something went wrong" response in case of a serious error.

The point here is resiliency. I agree that you want to catch bugs early, and you should immediately abort execution once bad things happen; I also fundamentally agree that you don't want to litter your code with error types or other constructs that are supposed to never actually be used. But faults do happen in practice, because we write buggy code, and in such a case, it's good if you can isolate the fault and recover at a higher level. Languages like Erlang take this idea to an extreme.

This is currently a real problem for us with Swift, so I was wondering whether Rust has a better solution here.

Re:Speeding Up the Rust Compiler by burntsushi on Wednesday December 11, 2019 @03:02PM (#21766195)
> I unfortunately know very little about Rust, but is it possible to recover from panics in Rust?

Yes: https://doc.rust-lang.org/std/panic/fn.catch_unwind.html

However, this comes with a critical caveat: unwinding can be disabled when compiling an application, which means one cannot guarantee that catch_unwind will actually work. (If unwinding is disabled, then panics turn into unrecoverable aborts.)

For this reason, and because the standard error handling mechanism is done through return values, panicking is not an acceptable way to do robust error handling in Rust. Recovering from panics is useful in niche scenarios, like keeping a web server running even if a request causes a panic or in tests for ensuring that all tests run even if one panics. Which basically matches your key concern here.

(To be clear, I think this is mostly orthogonal to my original comment in this thread. :-))

Re:Speeding Up the Rust Compiler by Tainnor on Thursday December 12, 2019 @11:03AM (#21772983)
Yep, that sounds like kind of what I would need. :)

But in general, exception handling is hard. There is value in having locally-unrecoverable, but globally-recoverable faults, stuff like that is probably where languages like Erlang shine.

Re:Speeding Up the Rust Compiler by bluejekyll on Wednesday December 11, 2019 @11:36AM (#21764084)
I can’t tell if you and I are agreeing or disagreeing at this point. And yes, I would not disagree with the quality of your code and the libraries you’ve published, they’re some of the best and most widely used in the community.

You’ve said don’t panic for code paths that the program doesn’t control. That is nuanced, and often is not always obvious and easy to reason about, but the language makes those exceptional cases obvious, so why not use that to your advantage?

edit: and by the way, I didn't say that on indexes, I totally agree with you. Though, sometimes it is valuable to go to the extreme and prevent malicious index values from untrusted sources crashing your program...

Re:Speeding Up the Rust Compiler by burntsushi on Wednesday December 11, 2019 @01:23PM (#21765233)
>I can’t tell if you and I are agreeing or disagreeing at this point.

I think we are disagreeing mostly over pedagogy.

> That is nuanced, and often is not always obvious and easy to reason about

Right. I think this is what I meant by "error handling is hard." But this really boils down to understanding what a runtime invariant is, and that takes time to learn. It's hard to do good error handling without internalizing that.

> but the language makes those exceptional cases obvious, so why not use that to your advantage?

I think you are, specifically by using unwrap/expect. In many such cases, the unwrap/expect has a comment explaining why it's impossible to panic.

> edit: and by the way, I didn't say that on indexes, I totally agree with you. Though, sometimes it is valuable to go to the extreme and prevent malicious index values from untrusted sources crashing your program...

Well, `&slice[i]` and `slice.get(i).unwrap()` are equivalent. So if we give blanket advice like "don't use unwrap/expect in libraries" then the latter gets caught up in that advice while the former doesn't, even though they are exactly equivalent. To me, this reveals the problem in that pedagogy because it focuses too much on one particularly common manifestation rather than the thing that actually matters: a panic visible to an end user is always a bug.

To be clear, the thing I am disagreeing with is the advice to "not use unwrap/expect; use case analysis instead." I can appreciate a pedagogy that simplifies things upfront with the cost of getting some corner cases wrong. But unwrap/expect are used too much in too many valid cases IMO for that type of strategy to be effective.

Re:Speeding Up the Rust Compiler by jayflux on Wednesday December 11, 2019 @08:54AM (#21762285)
I’m pretty sure clippy “encourages” you to use .get() for Vectors returning an Option<> for you to deal with so accessing a bad index is avoidable in idiomatic rust.

I’m not sure that was the best example but I agree with your overall point.

Re:Speeding Up the Rust Compiler by burntsushi on Wednesday December 11, 2019 @09:20AM (#21762524)
It does not. I know because I advocated against it. :-) https://github.com/rust-lang/rust-clippy/issues/1300

Also, Clippy's defaults are developed to be quite opinionated. It makes many suggestions that I (and many others) disagree with. So Clippy's defaults should not be used as a bludgeon for what's idiomatic and correct.

Re:Speeding Up the Rust Compiler by steveklabnik on Wednesday December 11, 2019 @10:14AM (#21763095)
(I also do not use clippy very aggressively due to disagreeing with a lot of lints)
Re:Speeding Up the Rust Compiler by GrayShade on Wednesday December 11, 2019 @09:12AM (#21762473)
Re:Speeding Up the Rust Compiler by cesarb on Wednesday December 11, 2019 @07:31AM (#21761810)
One of the commits in one of the first MRs in this article shows an example of a legitimate use of `unwrap()`...

    pub fn new(mut streams: Vec<TokenStream>) -> TokenStream {
        match streams.len() {
            0 => TokenStream::empty(),
            1 => streams.pop().unwrap(),
            _ => TokenStream::Stream(Lrc::new(streams)),
        }
    }
Since `streams.pop()` will never return `None` when `streams.len()` is `1`.

But I agree, most of the time using `match` (or its simplified form `if let`) or one of these combinators instead of `unwrap()` is better.

Re:Speeding Up the Rust Compiler by asdkhadsj on Wednesday December 11, 2019 @08:39AM (#21762189)
As a sidenote, I've been thinking about instituting an internal styling guide to discourage `unwrap()` in your scenario, in favor of `expect("impossibly missing")` or etc.

The message helps a bit more, and I also want to avoid debug unwraps that were forgotten about.

Though, the more I think about it the more a macro makes sense to me, purely for code search. If we used a macro, something like `impossible!(streams.pop())` then I can code search for `.unwrap` and `.expect`.

Hmm

Re:Speeding Up the Rust Compiler by rrobukef on Wednesday December 11, 2019 @12:25PM (#21764630)
A compiler error if the code cannot be eliminated during optimization. Kind-of hacky, but it could be valuable as a new function.
Re:Speeding Up the Rust Compiler by jashmatthews on Wednesday December 11, 2019 @07:50AM (#21761898)
> `unwrap()` is "handling" errors by crashing the program,

IIUC, panicking via unwrap() or expect() IS graceful error handling. It unwinds the stack and releases resources. We should be clearer because it's confusing to people less familiar with Rust who will confuse this with actually crashing and immediately terminating.

Personally, I think if there's nothing you can actually do to handle this error then unwrap() or expect() are the correct choices. There's no point writing more code just to bubble an error that you can't handle.

Re:Speeding Up the Rust Compiler by cesarb on Wednesday December 11, 2019 @08:37AM (#21762170)
> IIUC, panicking via unwrap() or expect() IS graceful error handling. It unwinds the stack and releases resources.

AFAIK, only with the default panic=unwind mode. With the panic=abort mode, it aborts the whole process instead of unwinding.

Re:Speeding Up the Rust Compiler by jashmatthews on Wednesday December 11, 2019 @09:37AM (#21762709)
panic=abort is only intended for embedded stuff, I think.
Re:Speeding Up the Rust Compiler by steveklabnik on Wednesday December 11, 2019 @10:17AM (#21763140)
Nope, it's intended whenever you want this semantic, and do not want to pay for landing pads. Embedded is a common case where this is true, but it's not exclusively for that.
Re:Speeding Up the Rust Compiler by galangalalgol on Wednesday December 11, 2019 @08:10AM (#21762004)
If the filesystem you are trying to open a log file on fails, only your logging thread needs to give up, not your whole program. If there is anything you can still accomplish, don't panic, carry on. Tesla let the logging exception prevent charging of the vehicle.
Re:Speeding Up the Rust Compiler by lmm on Wednesday December 11, 2019 @08:43AM (#21762217)
> If there is anything you can still accomplish, don't panic, carry on.

This is often poor advice, because continuing in a corrupt state can often be worse (and harder to debug) than a clean panic. E.g. it's better to crash than to overwrite a save file with corrupt data, or transfer money to the wrong account, or show one user data belonging to another user.

Re:Speeding Up the Rust Compiler by galangalalgol on Wednesday December 11, 2019 @09:24AM (#21762567)
If we are talking microservices I agree, take down the service. I have written a lot of safety critical stuff, and the advice for a given address space (virtual or physical) has been throw on inputs that are out of range or otherwise bad. This makes you fail fast and notice stuff, but i makes the conglomerate of address spaces as a whole be horribly unreliable, all coming down all the time. As we have bevome less monolithic with little persistent data that is shared between concerns, it has become possible to kill and restart only the offending thread, service, hardware etc. The result is a degraded functionality with an alert that it has become so. Tesla shouldn't have just stopped, they should have alerte the user that maintenance was necessary and given up on logging those errors to the now failed flash.
Re:Speeding Up the Rust Compiler by EugeneOZ on Wednesday December 11, 2019 @08:15AM (#21762041)
Let's say you have server or daemon and you call some library function - if it was written with such careless approach, whole server will fail just because user record had invalid email address, or because some entered data was not in expected format.

Let's say you have CLI tool - instead of message "XX is a directory - file expected" your tool will just fail silently.

There are much more examples. Graceful shutdown is what we have to do, even laziest of us.

Re:Speeding Up the Rust Compiler by jashmatthews on Wednesday December 11, 2019 @11:01AM (#21763677)
Validation would be an expected Error and handled by returning an Err() rather than using unwrap() or expect().

.expect(format!("{} is a directory - file expected")) gives exactly the behaviour you've described for a CLI tool.

Graceful shutdown means releasing locks, deleting PID files etc when encountering an unrecoverable error. Not just any error.

Re:Speeding Up the Rust Compiler by EugeneOZ on Wednesday December 11, 2019 @12:12PM (#21764476)
> Validation would be an expected Error and handled by returning an Err() rather than using unwrap() or expect().

When language allows you to make all errors "expected", it's really stupid to don't use it. Only reason is laziness.

Re:Speeding Up the Rust Compiler by jashmatthews on Wednesday December 11, 2019 @01:28PM (#21765294)
I don't understand what you're trying to say here. Laziness has nothing to do with it. Returning a Result vs using .expect("this should never be missing or an error") is about whether your program can continue after this error.
Re:Speeding Up the Rust Compiler by EugeneOZ on Wednesday December 11, 2019 @02:57PM (#21766145)
No, panicking is not a correct way to handle programm execution, it's just "I don't want to care about this possible error case right now", nothing more. I gave you 2 examples and you preferred to ignore them or say "it's ok". Let's agree to disagree, I will never understand people who are too lazy to allow panicking in their code and say "works as intended".

Many languages don't have Result types to return so there you can't do much sometimes to prevent data corruption or something worse. But in languages where you can return more than just "false", it's not acceptable.

Re:Speeding Up the Rust Compiler by gdxhyrd on Wednesday December 11, 2019 @07:23AM (#21761771)
In libraries, yes. In applications, no. A lot of errors are fatal.
Re:Speeding Up the Rust Compiler by devit on Wednesday December 11, 2019 @07:54AM (#21761921)
It should be used when you can prove that the error never happens, although unfortunately the compiler can't check that.