From what I understand, the copy fail bug was found by researcher who noticed something weird and then using AI to scan the codebase for instances where that becomes a problem.

I bet that with a slightly looser prompt/harness, the LLM could have found these twin bugs too.

Yet at the same time, I also think that if the human researcher had manually scanned the code, he'd have noticed these bugs too.

FWIW I do think LLMs are great tools for finding vulnerabilities in general. Just that they were visibly not optimally applied in this case.

They could also have found all these things at the same time - and are slow-rolling the disclosures.

I don't think the copy.fail people understood the issue they found, as is evident by the heavy focus on AF_ALG/aead_algif, which is essentially "innocent" as we're seeing here.

I think LLMs are great for vulnerability discovery, but you need to not skimp on the legwork and understanding what even you just found there.

Right but without the LLM the bug doesn't get found at all.

That's not necessarily true. Who's to say the security researchers wouldn't have found it if they'd searched the code manually?

It's an AI security firm! You might just as productively ask "why did all the other engineers who ever looked at this code not find it, and why was Theori the one to actually surface it?".

I would have taken a LOT longer but often this kind of manual search is so tedious people just don't do it. LLMs don't get bored.

> LLMs don't get bored

They do not get bored like a human but they are trained on human language and replicate the same traits, such as laziness, and expressing boredom or annoyance (even if obviously they do not experience anything at all). It’s actually a lot of effort to get them to engage with things at a deeper level without skipping corners

I’m hardly going to simp for LLM tools but the fact that the bug existed and no one had reported it seems proof positive no one was about to find it without them

Yes, I agree. I'm not the GP poster.

Safer to assume at least one of NSA, Mosad and a few others were sitting on it for years.

Am I missing something? Where does it say that the researcher that found Dirty Frag used LLM to find it? Have you read the original report from the researcher?

No, they did not. Careful of falling for the psychosis.

> This finding was AI-assisted, but began with an insight from Theori researcher Taeyang Lee, who was studying how the Linux crypto subsystem interacts with page-cache-backed data.

https://xint.io/blog/copy-fail-linux-distributions

Theori is an AI security research firm.

You appear to want to die on the hill of "This vulnerability would never have been found if we lived in a world without LLM AI" which is a very strange hill to die on.

There's no question that we live in the world where LLM AI was involved in finding the copy fail vulnerability at this specific time, and it's completely normal for people to see a vulnerability and then look closer and find related vulnerabilities or a deeper root cause, but there's no need to adopt an extreme "without AI LLM we don't find these vulnerabilities" position.

It's weird to say I want to "die on this hill" because that's not even something I believe. There was nothing especially difficult about this particular vulnerability. My only observation that nobody did find it before, then an LLM security firm went out looking for Linux LPEs, and thus it was discovered.

That is a very difficult fact pattern to which to attach the conclusion "LLMs have sabotaged security research" (my paraphrase).

The finding started with human intuition and was assisted by an LLM. You can yell "AI sec firm" 1000 times. A human got it started. You shouldn't die on that hill.

Of the MANY things I've completed in the last year that I would never have done without an LLM, a human got 100% of them started. The ideas were mine in every case.

But it is still a fact that I have been taking on all sorts of tasks I would never have taken on if I didn't have power tools.

My comment was solely about the correct attribution who made the initial finding. It's not a comment about the value of AI. I think we can get facts right and still argue for or against AI.

That's a pet peeve of mine as well, the inability to discuss facts / correct mistaken things on the internet, if the fact/mistake is on the "wrong" side of an argument.

I don't think I'm doing that though.

The context is a discussion about whether it would have been found without the assistance of an LLM. I agree that further upthread there maybe some misattribution but it is not present in the post you were directly replying to and it is not really the argument being made.

His whole sentiment was yelling several times "LLMs did this". He wanted to smuggle his pro AI attribution in, one way or another. In that way I could also argue "without humans, we wouldn't have LLMs." But it doesn't have value, right? I don't know why some try so hard to play down any human impact in this context. LLMs can help to find bugs. Without broader context it's a good and interesting thing. There is no need to trample over everything left and right just to overhype it.

Well.. every new vulnerability is one nobody did find it before.

Otherwise, it won't be classified as "new"

--

Edit:

I think LLM is very useful here.

When a researcher spot something funny, instead of spending two days on reading and testing, he can fire up a LLM and have it read all the code lead to there in ~30 minutes.

It seems as though this issue occurred to him, then he used their tool ("Xint Code") to analyze the codebase for instances of it.

I don’t think that’s what the OP is saying at all, just that using LLMs needs to be a cooperative research process.

Also I see you jumping around a lot to the defense of LLMs when I don’t think anyone is really attacking them. Maybe cool it a bit.

From the thread that ensued I feel comfortable that my interpretation of the comment (or rather, my confusion about it) was in fact germane.

Germane or not the knee-jerk reactions related to LLMs are getting ridiculous and it seems like it’s the same people throwing down at a moments notice and then chalking it up to a misunderstanding.

So like I said, just chill out.

It’s incredible humans spot stuff like this. I guess even more incredible that LLMs can do it!

Right. Finding the bug is in itself a win. It seems we’re jumping from that spend-electricity-to-find-bugs win to arguing about how some things around it are not quite good or comfy.