News for hackers. Stuff that matters.
Re:Rav1e: An experimental AV1 video encoder, designed to be ... by kristofferR on Monday July 16, 2018 @04:13AM (#17539573)
What does "safest" mean in this context?
Re:Rav1e: An experimental AV1 video encoder, designed to be ... by jabl on Monday July 16, 2018 @04:17AM (#17539587)
Presumably that it's implemented in Rust, and thus does not suffer from many of the usual bugs that C code bases suffer from?

There seems to be some asm code as well, which obviously does not enjoy the safety advantages of Rust.

Re:Rav1e: An experimental AV1 video encoder, designed to be ... by arez on Monday July 16, 2018 @04:59AM (#17539743)
I always wondered if in practice it really checks out, is rust code really safer? Did the bugs just shift to different ones? Is there anyone who wrote about that already?
Re:Rav1e: An experimental AV1 video encoder, designed to be ... by masklinn on Monday July 16, 2018 @05:06AM (#17539770)
It's probably not quite equivalent but I believe Frederico & al found & fixed a number of issues when they converted librsvg to rust. You may want to check the archives on their blogs (IIRC it's split between Frederico's own and the librsvg one) for more, or specifics.

But "safe" rust at least intrinsically protects from use-after-free, double free, dangling pointers, null-pointer dereferences, out-of-bounds accesses, … You may still have logic bugs of course (though the richer type system expressivity also allows better static encoding of application & domain logic), but these baseline memory-safety issues will only be a problem in specific and tagged `unsafe` blocks rather than throughout the application.

Re:Rav1e: An experimental AV1 video encoder, designed to be ... by kevincox on Monday July 16, 2018 @03:23PM (#17544208)
While I do believe that rust will prevent many memory safety issues. I think you would probably catch a bunch of safety issues rewriting any code.
Re:Rav1e: An experimental AV1 video encoder, designed to be ... by ryanmarsh on Monday July 16, 2018 @05:16PM (#17545211)
I think you would probably catch a bunch of safety issues rewriting any code.

As well as introduce new ones.

Re:Rav1e: An experimental AV1 video encoder, designed to be ... by derefr on Tuesday July 17, 2018 @02:15PM (#17551805)
That's only if you rewrite "from the design" in a fresh repo.

If, on the other hand, you rewrite the code line by line from C to Rust (which Rust is actually quite amenable to!), faithfully translating the semantics of the C code into the Rust code (and thereby having to use lots of unsafe{} blocks) then you can avoid most of the problems—what you'll end up with will essentially be the same as what a hypothetical C-to-Rust transpiler would output.

Importantly, you can also translate the test suite in this same way.

After that step is done, you can just refactor the resulting code, replacing unsafe{} blocks with Rust idioms, and then rerunning the tests (which you can leave un-idiomatized) as regression tests.

Re:Rav1e: An experimental AV1 video encoder, designed to be ... by kevincox on Monday July 16, 2018 @05:57PM (#17545517)
Of course. I would suspect that with Rust you would introduce less new ones but I just wanted to point out that finding bugs isn't evidence that the language helped you out.

Even though I am a believer I would love to see some solid evidence of Rust making code safer.

Re:Rav1e: An experimental AV1 video encoder, designed to be ... by brian-armstrong on Monday July 16, 2018 @03:30PM (#17544267)
All of the pointer issues you mentioned have already been solved in C++ by unique_ptr.
Re:Rav1e: An experimental AV1 video encoder, designed to be ... by pjmlp on Monday July 16, 2018 @04:08PM (#17544640)
Good, now try to enforce developers to actually use it on their code and all third party libraries they link to.

Ah, and not passing it around by address or reference, instead of actually moving ownership.

C++17 is a great improvement, but for it to work out in this context, developers need to actually write C++ instead of "C with C++ compiler".

Re:Rav1e: An experimental AV1 video encoder, designed to be ... by brian-armstrong on Monday July 16, 2018 @06:57PM (#17545888)
This is a silly argument. When evaluating whether to use a tool, you should consider how /you/ would use the tool, not how someone else does.
Re:Rav1e: An experimental AV1 video encoder, designed to be ... by pjmlp on Tuesday July 17, 2018 @12:25AM (#17547308)
Yeah, it kind of works out in the ideal world where one works alone, writing 100% of the source code.
Re:Rav1e: An experimental AV1 video encoder, designed to be ... by stouset on Monday July 16, 2018 @07:19PM (#17546012)
Right up until the moment you have third-party dependencies.
Re:Rav1e: An experimental AV1 video encoder, designed to be ... by dbaupp on Monday July 16, 2018 @05:23PM (#17545264)
Pretty much only use-after-free and double-free of that list is truly solved by unique_ptr, which is great, but nothing like what you say:

- std::move out of a unique_ptr x and "*x" is a null pointer dereference,

- take a reference into a unique_ptr, and it becomes dangling if it is held after the pointer is deallocated,

- an T[N] array doesn't get bounds-checked whether or not it is stored in a std::unique_ptr (it is nice that it reduces the number of raw pointers flying around, which likely does reduce the number of out-of-bounds accesses, but it doesn't solve them)

Re:Rav1e: An experimental AV1 video encoder, designed to be ... by brian-armstrong on Monday July 16, 2018 @07:01PM (#17545911)
Clang does a good number of warnings/errors on unique_ptr, and I'm not sure your first point is actually right - it tends to be pretty hard to use after move.

If you want to pay the cost of runtime array checking every time, an impl in operator[] is just a few lines long. Thankfully it is becoming more common to run programs under ASAN in dev mode.

Re:Rav1e: An experimental AV1 video encoder, designed to be ... by dbaupp on Monday July 16, 2018 @08:15PM (#17546281)
It is trivial to use-after-move. The following compiles completely without warnings with the clang on my system (even with -Wall -Weverything), and segfaults:

  #include <memory>
  
  int main() {
    std::unique_ptr<int> p = std::make_unique<int>(0);
  
    std::unique_ptr<int> q(std::move(p));
  
    return *p;
  }
Maybe you mean it doesn't happen much in practice, which might be true (although, we need to be comparing use-after-move with use-after-free etc., which also don't happen that much, per line-of-code), but is a different point. The fact remains that unique_ptr doesn't solve use-after-move.

> If you want to pay the cost of runtime array checking every time, an impl in operator[] is just a few lines long

This is also a different point, and moving the goal posts.

In any case, which operator[] exactly? AFAIK, neither T[N] nor T* can have a custom operator[] (and getting the bounds of a raw pointer is essentially impossible), and none of the operator[]s of std::array nor std::vector nor std::span do bounds checking (sure, you can use ...::at for the first two, but you have to remember to do that everywhere, going against the default that pretty much every programming language uses).

> Thankfully it is becoming more common to run programs under ASAN in dev mode.

Yes, this is great! However, this is yet another different point, and it is orthogonal to modern C++ and its fancy new features like std::unique_ptr: C++98 code, and even C code benefit from ASan.

Re:Rav1e: An experimental AV1 video encoder, designed to be ... by pcwalton on Monday July 16, 2018 @08:51PM (#17546452)
> I'm not sure your first point is actually right - it tends to be pretty hard to use after move.

    std::unique_ptr<int> foo(new int(10));
    
    void bar(int& x);
    
    int main() {
        bar(*foo);
        return 0;
    }
    
    void bar(int& x) {
        foo = std::unique_ptr<int>(new int(20));
        std::cout << x; // use after free
    }
No warnings on any warning level.
Re:Rav1e: An experimental AV1 video encoder, designed to be ... by pcwalton on Monday July 16, 2018 @08:46PM (#17546429)
unique_ptr helps with one thing and one thing only: memory leaks.

The list you were responding to didn't include memory leaks.

Re:Rav1e: An experimental AV1 video encoder, designed to be ... by derf_ on Monday July 16, 2018 @07:48AM (#17540506)
The "safest" comment is mostly tongue-in-cheek, but the advantages are real. I enjoyed this comment from someone contributing to the project for the first time:

< barrbrain> atomnuker: I felt good when my patch was done and I had exactly what I set out to write.

< barrbrain> I didn't enjoy the compiler forcing me to fix all my bugs upfront.

There are also advantages other than safety to using a modern systems language that has learned something from the last 30 years of programming language design but doesn't come with all of the baggage of C++.

Re:Rav1e: An experimental AV1 video encoder, designed to be ... by manmal on Monday July 16, 2018 @08:43AM (#17540835)
Having coded full-time in Swift for 2 years now, I can attest that strongly-typed languages really do a lot for code quality and especially reliability. Sure, you still have semantic bugs all over (that's what tests are for), but whole classes of bugs (null-pointer refs, dangling pointers) are at least greatly reduced. As long as I stay away from the no-nos (like implicitly unwrapped optionals) and use the goodies (immutable models; enums with attached values; generics), my apps hardly ever crash now, once out in the store.
Re:Rav1e: An experimental AV1 video encoder, designed to be ... by coldtea on Monday July 16, 2018 @10:15AM (#17541382)
>I always wondered if in practice it really checks out, is rust code really safer? Did the bugs just shift to different ones?

Bugs don't magically "shift". If you eliminate a class of bugs, it's gone (e.g. memory bugs).

Whether you can still have other bugs (e.g. logic bugs) that's irrelevant, you could still have those in C as well.

Re:Rav1e: An experimental AV1 video encoder, designed to be ... by davrosthedalek on Monday July 16, 2018 @11:33AM (#17542083)
Or, the way you eliminated one class opens up or makes more likely a different class. Maybe avoiding pointers makes it more likely to introduce logic bugs. It could also make some algorithm too slow, which one could call a performance bug.

I don't think it's particular likely, but there are always trade-offs.

Re:Rav1e: An experimental AV1 video encoder, designed to be ... by pjmlp on Monday July 16, 2018 @11:39AM (#17542148)
As proven by Ada, Modula-2 and a few other systems languages, you don't need to scatter pointer arithmetic all over your code to achieve the same machine language output.
Re:Rav1e: An experimental AV1 video encoder, designed to be ... by tzahola on Monday July 16, 2018 @01:17PM (#17542987)
>Bugs don't magically "shift".

It depends... You can’t have dangling pointers in Java - instead, you can “leak” memory by holding onto your objects even when they’re not needed anymore.

Re:Rav1e: An experimental AV1 video encoder, designed to be ... by elteto on Monday July 16, 2018 @01:34PM (#17543127)
I don't think you are using "instead" correctly in that sentence.

You are implying that not being able to have dangling pointers in Java (and thus eliminating an entire category of bugs) inevitably leads to the new category of bugs of leaking memory by holding on to references. But that is the case in any language that allows dynamic memory allocation. You can leak memory in Rust, C++ and VB if you don't dealloc.

Re:Rav1e: An experimental AV1 video encoder, designed to be ... by Dylan16807 on Monday July 16, 2018 @06:42PM (#17545804)
A program with more memory controlled by GC is generally going to have more leaks like that.

The switch reduces bugs some, and definitely reduces the severity, but I would largely put the bugs in the same category and say it hasn't been eliminated.

Re:Rav1e: An experimental AV1 video encoder, designed to be ... by coldtea on Monday July 16, 2018 @03:12PM (#17544080)
Part of my point would be that you can have that in C as well.
Re:Rav1e: An experimental AV1 video encoder, designed to be ... by pjmlp on Monday July 16, 2018 @07:36AM (#17540451)
Yes, but not only Rust, rather any memory safe system programming language all the way back to ESPOL in 1961.

In those languages you have "Logic errors" to debug, in C and its direct descendants you have "Logic errors" + "UB errors" + "Memory corruption errors" to debug.

Also having the use of unsafe explicit in the type system, means that it is relatively easy to track down where such issues might occur, whereas in C every line of code can be a source of problems, depending on which compiler and flags are being used.

Again, this applies to all memory safe system programming language, not just Rust.

EDIT: Fixed the NEWP reference, as ESPOL came first and NEWP only replaced it in 1976.

Re:Rav1e: An experimental AV1 video encoder, designed to be ... by coldtea on Monday July 16, 2018 @10:17AM (#17541394)
>Yes, but not only Rust, rather any memory safe system programming language all the way back to ESPOL in 1961.

Yes, but now we're discussing only those that practically matter to more than 5 people today.

Re:Rav1e: An experimental AV1 video encoder, designed to be ... by pjmlp on Monday July 16, 2018 @11:15AM (#17541923)
I bet Unisys MCP deployments still matter to more than 5 people.

Plus the point was to talk about safe systems programming languages in general.

Or should we ignore history just because UNIX won on the server?

Re:Rav1e: An experimental AV1 video encoder, designed to be ... by tzahola on Monday July 16, 2018 @01:20PM (#17543010)
What’s this “UNIX”? A Linux distribution?
Re:Rav1e: An experimental AV1 video encoder, designed to be ... by pjmlp on Monday July 16, 2018 @01:45PM (#17543262)
Re:Rav1e: An experimental AV1 video encoder, designed to be ... by coldtea on Monday July 16, 2018 @03:12PM (#17544074)
His point is probably UNIX didn't won -- Linux did.
Re:Rav1e: An experimental AV1 video encoder, designed to be ... by pjmlp on Monday July 16, 2018 @03:23PM (#17544202)
Linux is non-certified copy of UNIX, without POSIX it wouldn't matter in the marketplace, ergo UNIX won.
Re:Rav1e: An experimental AV1 video encoder, designed to be ... by vetinari on Monday July 16, 2018 @04:52PM (#17544995)
As you can see in the wild, nobody really cares about UNIX certification. Ergo, UNIX didn't win.
Re:Rav1e: An experimental AV1 video encoder, designed to be ... by pjmlp on Monday July 16, 2018 @05:16PM (#17545207)
The fact is that the Linux kernel without the APIs copied from POSIX/UNIX and the userspace copied from UNIX is useless on the server space.

Of course we can play semantic games about the use of UNIX word, and GNU/Linux not sharing any code from either BSD or AT&T linage, it doesn't make it any less UNIX.

Had Linux not copied UNIX, and provided a playground for UNIX vendors kind of outsource their development costs, it would have turned out to yet another hobby OS.

Re:Rav1e: An experimental AV1 video encoder, designed to be ... by vetinari on Monday July 16, 2018 @06:26PM (#17545707)
NT had POSIX subsystem too.

What Linux (and NT POSIX to certain degree) allowed, was to move the existing investment in software from hodge-podge of mutually incompatible, but expensive UNIX systems, to somewhere else.

Re:Rav1e: An experimental AV1 video encoder, designed to be ... by pjmlp on Tuesday July 17, 2018 @12:33AM (#17547348)
NT isn't a copy of UNIX, just because it had support for the first edition of POSIX, it goes beyond than that.

Starting by wanting to copy Minix, an educational OS that copied UNIX, getting the GNU tools on board (which goal was to copy UNIX), kernel architecture, device drivers subsystems, userland, culture, ....

My old stack of Linux Journal editions at home are pretty clear what the ongoing story was.

If NT was a copy of anything, it was VMS given Dave Cutler's contribution to its design.

Re:Rav1e: An experimental AV1 video encoder, designed to be ... by AndrewDucker on Monday July 16, 2018 @04:16AM (#17539584)
"Written in Rust"
Re:Rav1e: An experimental AV1 video encoder, designed to be ... by blitmap on Monday July 16, 2018 @06:03AM (#17540004)
I'm trying to get my insurance discounted on my car. I should slap a "written in Rust" sticker on it. Safe driver discount?
Re:Rav1e: An experimental AV1 video encoder, designed to be ... by pleasecalllater on Monday July 16, 2018 @06:55AM (#17540251)
It makes sense. If your car is covered with rust, you should be almost sure that no one will even try to steal it. :)
Re:Rav1e: An experimental AV1 video encoder, designed to be ... by Crespyl on Monday July 16, 2018 @03:03PM (#17543998)
Given that automotive control software is apparently a huge mess right now, and a rapidly growing attack surface, a more disciplined rewrite in Rust might actually be enough for an insurance company to consider...
Re:Rav1e: An experimental AV1 video encoder, designed to be ... by CUViper on Monday July 16, 2018 @04:07PM (#17544630)
PDXRust just had a related talk, "Writing Software That's Safe Enough To Drive A Car": https://www.meetup.com/PDXRust/events/252160920/