I don't know if its the case with rav1e but in many video codecs most of the assembly is just a flat unrolled circuit with zero or minimal control flow, minimal address calculation-- essentially just a static circuit that you might otherwise implement as a fixed block of logic in an asic. As a result they aren't too much of an unsafety vector.

This is also often true for cryptographic software, particularly fixed-parameter optimized code.

The performance gain usually comes from some mixture of good scheduling or register allocation that the compiler fails at and the use of instructions that the compiler won't (reliably) emit.

For these kinds of straight-line codes, I haven't found mechanically validating them to be significantly more complicated than verifying C code... and as C code these functions tend to be the lowest risk type.

So, without any direct experience, I'd expect there to be more risk in rav1e from the unsafe-rust than the asm.

That said, already the 'attack surface' of an encoder is pretty small. I'd personally expect rav1e's gain from rust's safety would be less security and more in avoiding wasting time on blind alleys caused by memory corruption in the codebase. I've seen more than one poor design decision made in a multimedia codec which was ultimately due to a bug that a better language might have prevented.