bool silly_mistake = false;
//... lots of lines of code
free(x);
//... lots of lines of code
if (silly_mistake) { // silly_mistake shown to be false at this point in the program in all testing, so far
free(x);
}
A bug like above would still be something that would be patched, even if a way to exploit it has not yet been found, so I think it's fair to call out (perhaps with less sensationalism).FWIW there's a whole boutique industry around finding these. People have built whole careers around farming bug bounties for bugs like this. I think they will be among the first set of software engineers really in trouble from AI.
Imagine “silly mistake” is a parameter, and rename it “error_code” (pass by reference), put a label named “cleanup” right before the if statement, and throw in a ton of “goto cleanup” statements to the point the control flow of the function is hard to follow if you want it to model real code ever so slightly more.
It will be interesting to see the bugs it’s actually finding.
It sounds like they will fall into the lower CVE scores - real problems but not critical.
Those really evil bugs are the ones that exist in code paths that only trigger 0.001% of the time.
Often, the code path is not triggerable at all with regular input. But with malicious input, it is, so you can only find it through fuzzing or human analysis.
That is precisely what a static analyser can determine. E.g. if you are reading a 4-byte length from a file, and using that to allocate memory which involves adding that length to some other constant, it will assume (unless told otherwise) that the length can be all 4G values and complain about the range of values which will overflow.
Why is everyone here trying to invent conspiracies to explain the obvious? Actually I do know why - cope.
Everyone here doing mental gymnastics to imagine Anthropic playing 5-D chess because they're in denial of what is happening in front of their faces. AI is getting more capable/dangerous - it's not surprising to anyone. The trendlines have pointed in this direction for years now and we're right on schedule.
As an example, I have taken a formally verified bit of code from [1] and stripped out all the assertions, which are only used to prove the code is valid. I then gave this code to Claude with some prompting towards there being a buffer overflow and it told me there's a buffer overflow. I don't have access to Opus right now, but I'm sure it would do the same thing if you push it in that direction.
For anyone wondering about this alleged vulnerability: Natural is defined by the standard as a subtype of Integer, so what Claude is saying is simply nonsense. Even if a compiler is allowed to use a different representation here (which I think is disallowed), Ada guarantees that the base type for a non-modular integer includes negative numbers IIRC.
[1]: https://github.com/AdaCore/program_proofs_in_spark/blob/fsf/...
[2]: https://claude.ai/share/88d5973a-1fab-4adf-8d29-8a922c5ac93a
> Running LLMs over projects that I'm very familiar with will almost always have the LLM report hundreds of "vulnerabilities" that are only valid if you look at a tiny snippet of code in isolation because the program can simply never be in the state that would make those vulnerabilities exploitable.
Their OpenBSD bug shows why this is not so simple. (We should note of course that this is an example they've specifically chosen to present as their first deep dive, and so it may be non-representative.)
> Mythos Preview then found a second bug. If a single SACK block simultaneously deletes the only hole in the list and also triggers the append-a-new-hole path, the append writes through a pointer that is now NULL—the walk just freed the only node and left nothing behind to link onto. This codepath is normally unreachable, because hitting it requires a SACK block whose start is simultaneously at or below the hole's start (so the hole gets deleted) and strictly above the highest byte previously acknowledged (so the append check fires).
Do you think you would be able to identify, in a routine code review or vulnerability analysis with nothing to prompt your focus on this particular paragraph, how this normally unreachable codepath enables a DoS exploit?
There are notably no SHA-3 sums for all their out-of-bound write Linux vulnerabilities, which would be the most interesting ones.
They're not publishing SHAs for things that aren't confirmed vulnerabilities. They're doing exactly the thing you'd want them to do: they claim to have vulnerabilities when they have actual vulnerabilities.
They don’t explicitly rule out, I suppose, that these were only limited partial scans they did to find the vulnerabilities. But I don’t know why they’d do it that way, it’s not like they don’t have the resources to scan the entire Linux kernel.
(A "security scanner" is a one-and-done proposition because it's deterministic and is going to find what it finds the first time you run and nothing more. But a software security assessment project you run every year on the same target with different teams will turn up different stuff every year. I'm at pains to remind people how totally lame source code security scanners are. People keep saying "static analyzers already do this" and like, nobody in security takes those tools seriously.)
What were they actually able to do and not do? I got confused by this when reading the article as well.
Do they really need to include this garbage which is seemingly just designed for people to take the first sentence out of context? If there's no way to trigger a vulnerability then how is it a vulnerability? Is the following code vulnerable according to Mythos?
Is it really so difficult for them to talk about what they've actually achieved without smearing a layer of nonsense over every single blog post?Edit: See my reply below for why I think Claude is likely to have generated nonsensical bug reports here: https://news.ycombinator.com/item?id=47683336