I understand that the patch is correct. The point that I am trying to make is that the correctness of this code relies on non-local reasoning (the allocation/reallocation of the table before it is used). I believe that non-local reasoning raises the chance of error (and is likely one of the reasons why this bug existed in the first place).

By adding length counts and explicit bounds checks locally you provide local reasoning to back up this non-local reasoning which is much *easier* to make and maintain correct. I think that would result in code that is less likely to have similar bugs in the future.

So what exactly do you propose?

ye gads man, did you even read the comment you're replying to?

I did - and I'm not sure how adding those checks here would help. Are you proposing a full library reaudit?

$ ./pedant.sh

If you read what they proposed and disagreed with it, then don't ask "What do you propose?" - Argue/debate why you think their proposal, which you read, is not useful.

The best solution to this sort of bug is not using languages that are susceptible to this sort of bug. At the very least, I think it's time to retire the fallacy that we're generally capable of producing sound programs in memory unsafe languages. Just like we don't write code manually checking raw SQL to protect against injections and we don't roll our own crypto when we need to encrypt something or do a key exchange, we shouldn't treat writing safe code in C as something anyone can do by just being vaguely diligent.

I feel like we need for C and C++ what Typescript is for Javascript: Not a language from scratch but something which is as close as possible to the thing everyone is familiar with while doing the thing Rust does. A standard library where everything has the same names, the same kind of C++ objects and templates and RAII etc., change only this and nothing else.

Because otherwise you make people learn 100 other things at once to start using it effectively, and then they don't.

Indeed, I would kill for a proof assistant that integrates with C

Check out the tooling from the seL4 project. I'm aware they used intermediate levels between raw C and the proof assistant, but I don't see why that should be prohibitive.

Something like this? http://cyclone.thelanguage.org/wiki/Cyclone%20for%20C%20Prog...

That isn't the thing Rust does, and is slower, so then people won't use it.

The advantage of Rust is that you can produce efficient safe code. The disadvantage is that you have to learn a different language first, which deters adoption.

I'm asking this into the comment section, since I'm curious if anyone has checked:

Does Zig fill this space in any way? It still allows manual memory management but offers more protections, the specifics I'm unclear on.

Zig is something like this for C, but not C++. There are a number of ongoing attempts to do something like this for C++, but it is very hard and they are all way behind Rust in terms of mindshare.

You do understand that's not a practical solution here, do you?

Implementing webp and similar libraries in another language may not be quick or easy, but having these sorts of issues continue to crop up over and over doesn't feel particularly practical to me; it's just easier to act like every time a critical vulnerability comes up due to memory safety that it's an isolated issue than confront the reality that its a systemic issue and there aren't any easy solutions.

This response is becoming so tiresome

> This response is becoming so tiresome

I'd argue the constant stream of CVEs due to memory unsafety is even more tiresome.

Agreed, I'll stop saying it when we stop having these issues. I don't think people want to confront the fact that there isn't any easier solution, but we're going to keep running into the same thing over and over until we do.

The lack of response is way more tiresome. We keep having the same catastrophic failures decade after decade. Let's do nothing.

Over a library used by billions of people a lot of times a day, those bounds checks add up to a lot of wasted time and energy if you don't strictly need them.

If this is a common function in the webp library, this function is probably called something like a trillion times a day.

They aren't wasted if they are preventing security vulnerabilities.

I also wouldn't be surprised if patching, rebuilding and distributing this fixed version of libwebp to all of these devices would be comparable to the extra cost of decoding.

"Preventing security vulnerabilities" is not an unadulturated good (the most secure computer does nothing but checking to make sure it hasn't been breached), and any bounds check that does not actively prevent an intrusion is wasted from a value perspective.

Your intuition about the cost of a patch vs the extra instructions in the hot path to check bounds everywhere is also very much wrong: the cost of a patch (which is also amortized among many fixes) is not even close. The hot path adds up when you call it a lot.

The most secure computer does nothing, because checking if it has been breached is attack surface.

I think your intuition on the cost of not-taken bounds check branches is off, especially in the context of devices that spend most of their resources browsing the web, running radios, powering screen backlights. I bet any holistic measurement could not tell the difference.

> … resources browsing the web, running radios

That's the whole point of webp, though.

Not-taken branches and filling your superscalar slots with extra instructions costs 0 time, but they still cost energy.

The past 40+ years of CS research has created a huge number of tools for reducing the costs of memory safety. Some of these made it into Rust, but others are sitting in (often abandoned) academic projects. It is an abject failure of industry that we are still parsing untrusted input in languages like C.

WebP gets a slight pass because the code is so old; WebP is at least partly based on code from On2 that extends back at least until the early '00s. Rewriting working code is always going to be lower priority than writing new code (and it's not always a safety win either; being deployed for 10 years gets you an awful lot of real-world testing!).

But look at implementations for newer image formats and you see more of the same; C and C++ with the token java implementation that won't get used outside of the JVM.

C and C++ are often used because at the optimization level that these sorts of libraries use, Rust makes tradeoffs that do not work out to 0 overhead. I would personally suggest that Google et al invest in figuring out how to make "formally-verified C" a more widely adopted language for libraries that are truly security-critical.

C safe enough to use for parsing untrusted input like this needs to be either formally verified to be safe (not to be correct, at least it's it's the canonical implementation), or written in an intrinsically memory-safe way which humans can fully reason about the safety of. And at that latter point it's slower than Rust. (Not too mention that Rust's awareness of memory lifetimes enables substantial optimizations beyond plain C code that quite often result in faster code at the same level of effort.)

People should learn to use profilers, not even C and C++ do zero overhead versus writing in pure Assembly.

Maybe WEBP should have been written in Assembly, all the speed and I bet quite safe. /s

Apple has actually measured the impact of adding bounds checking to commonly used libraries including media libraries and found no impact on overall power use. Audio and video encoding libraries took around a 1% performance impact.

https://llvm.org/devmtg/2023-05/slides/TechnicalTalks-May11/...

There is no technological reason to not use bounds checking as the default mode of operation since ALGOL exists.

My favourite quote, C.A.R Hoare's "The 1980 ACM Turing Award Lecture"

"A consequence of this principle is that every occurrence of every subscript of every subscripted variable was on every occasion checked at run time against both the upper and the lower declared bounds of the array. Many years later we asked our customers whether they wished us to provide an option to switch off these checks in the interests of efficiency on production runs. Unanimously, they urged us not to--they already knew how frequently subscript errors occur on production runs where failure to detect them could be disastrous. I note with fear and horror that even in 1980 language designers and users have not learned this lesson. In any respectable branch of engineering, failure to observe such elementary precautions would have long been against the law."

-- https://www.labouseur.com/projects/codeReckon/papers/The-Emp...