Re:Over 100k Infected Repos Found on GitHub
by dns_snek on Thursday February 29, 2024 @08:29PM
(#39557449)
I agree that it's more likely to be inspected but I think the vast majority of developers aren't inspecting the code rigorously enough (including me, but I don't use LLMs for development) to catch non-obvious bugs, see for example the "Underhanded C contest" [0]
As you've pointed out, this vector would give them near surgical precision and insight into their target's code & systems, rather than casting a wide net with a vulnerable library on Github. They could use a model trained on "underhanded" code or even selectively overwrite parts of the responses with hand-crafted vulnerabilities while only targeting select organizations.
It makes me wonder what the business model of OpenAI and their peers is going to be over the long term. I can't imagine large corporations using "LLM as a service" indefinitely with the risk of IP theft and "bug injection".
As you've pointed out, this vector would give them near surgical precision and insight into their target's code & systems, rather than casting a wide net with a vulnerable library on Github. They could use a model trained on "underhanded" code or even selectively overwrite parts of the responses with hand-crafted vulnerabilities while only targeting select organizations.
It makes me wonder what the business model of OpenAI and their peers is going to be over the long term. I can't imagine large corporations using "LLM as a service" indefinitely with the risk of IP theft and "bug injection".
[0] https://en.wikipedia.org/wiki/Underhanded_C_Contest