Is "with a sufficiently smart LLM" the new "with a sufficiently smart compiler?"

"imagine feeding this into an LLM/ChatGPT" is the new "imagine a Beowulf cluster of these"

grendellm?

GPT-5, maybe not. But somebody somewhere is building something that can do that. And if they can't do it _now_ they have a plan that tells them what's missing. TLDR; it's coming, soon.

Writing exploits is a bit of an art-form. Current incarnations of GPT have trouble writing code at a level more advanced than a junior developer.

and lots of people are spending lots of time and money on AI Coding Assitants... which is more or less the knowledge base you need.

If they could use that structural training to answer queries like "Is there any code path where some_dangerous_func() is called without it's return value being checked"...

You can do this today by querying the AST output by a compiler. Regardless, the parent comment was talking about exploits, not vulnerabilities/bugs. Vulns are a dime-a-dozen compared to even PoC exploits let alone shippable exploits.

Ok, so add "and generate a C program to exploit it" to the prompt.

You're either being sarcastic or wildly underestimating how hard it is to write an exploit. I haven't written about exploit dev publicly for a _long_ time, but I invite you to read https://fail0verflow.com/blog/2014/hubcap-chromecast-root-pt... for what I consider to be a pretty trivial exploit of a very "squishy" (industry term) target.

XNU isn't the hardest target to pop but it is far from the easiest.

There's nobody more confident in the world, than HN poster wiring about a topic they have no experience with.

There is a huge gap (in the binary exploitation world) between identifying a problematic code pattern and having a workable bug (a reproduction), and even larger one between a reproducible crash and a working exploit (because we're not in the 90s anymore and complier/hardware mitigations are literally always enabled). Current LLMs can cross neither gap, and are not even close to bridging the second one.

Have you used GPT-5?

If you aren’t using GPT-6a then you are years behind.

GPT-69 is already far ahead of 6a.

Been using 73 for months now.

you need to wake up at 4am and have a cold shower!

12 before 12, 12 cold showers before 12 allowing GPT-7 to take care of my daily needs.

Apparently GPT-4 has some capacity to conduct exploits this by "reading" CVE reports. I don't know if it can autonomously create exploits though:

GPT-4 can exploit vulnerabilities by reading CVEs (theregister.com) 81 points by ignoramous 60 days ago | hide | past | favorite | 29 comments

https://news.ycombinator.com/item?id=40101846

which links to a Register Article[0], which links to a paper[1]:

"In this work, we show that LLM agents can autonomously exploit one-day vulnerabilities in real-world systems. To show this, we collected a dataset of 15 one-day vulnerabilities that include ones categorized as critical severity in the CVE description. When given the CVE description, GPT-4 is capable of exploiting 87% of these vulnerabilities compared to 0% for every other model we test (GPT-3.5, open-source LLMs) and open-source vulnerability scanners (ZAP and Metasploit). Fortunately, our GPT-4 agent requires the CVE description for high performance: without the description, GPT-4 can exploit only 7% of the vulnerabilities."[1]

[0] https://www.theregister.com/2024/04/17/gpt4_can_exploit_real...

[1] https://arxiv.org/pdf/2404.08144

Yes, that sounds about right. LLMs aren’t quite good enough to find novel bugs and exploit them like a human would.

Yeah, that works for web vulns where the vuln description is practically the exploit anyway. I could write a perl script that parses out variable names and writes sql injections for it.

For comparison, in the native world program is considered vulnerable when someone finds arbitrary write primitive (even without leak), use after free, and even double free. There is a huge gap between these and actually having a working RCE exploit. Most CVEs in this space are given without a working exploit ever written.