Re:Remote Prompt Injection in Gitlab Duo Leads to Source Cod...
by Retr0id on Monday May 26, 2025 @03:48AM
(#44095017)
You can still experimentally determine a strategy that works x% of the time, against a particular model. And you can keep refining it "offline" until x=99. (where "offline" just means invisible to the victim, not necessarily a local model)
Re:Remote Prompt Injection in Gitlab Duo Leads to Source Cod...
by LegionMammal978 on Monday May 26, 2025 @03:06AM
(#44094771)
Perhaps not, but the same input will lead to the same distribution of outputs, so all an attacker has to do is design something that works with reasonable probability on their end, and everyone else's instances of the LLM will automatically be vulnerable. The same way a pest or disease can devastate a population of cloned plants, even if each one grows slightly differently.
Re:Remote Prompt Injection in Gitlab Duo Leads to Source Cod...
by thaumasiotes on Monday May 26, 2025 @05:19AM
(#44095536)
OK, but that's also the way attacking a bunch of individuals who can get tricked works.
Re:Remote Prompt Injection in Gitlab Duo Leads to Source Cod...
by zwnow on Monday May 26, 2025 @05:56AM
(#44095760)
For tricking individuals your first got to contact them somehow. To trick an LLM you can just spam prompts.
Re:Remote Prompt Injection in Gitlab Duo Leads to Source Cod...
by thaumasiotes on Monday May 26, 2025 @05:29PM
(#44101863)
You email them. It's called phishing.
Re:Remote Prompt Injection in Gitlab Duo Leads to Source Cod...
by throwaway314155 on Monday May 26, 2025 @09:40PM
(#44103203)
Right and now there's a new vector for an old concept.
Re:Remote Prompt Injection in Gitlab Duo Leads to Source Cod...
by zwnow on Tuesday May 27, 2025 @04:10AM
(#44104909)
Employees usually know to not click on random shit they get sent. Most mails alrdy get filtered before they even reach the employee. Good luck actually achieving something with phishing mails.
Re:Remote Prompt Injection in Gitlab Duo Leads to Source Cod...
by thaumasiotes on Tuesday May 27, 2025 @05:15AM
(#44105240)
When I was at NCC Group, we had a policy about phishing in penetration tests.
The policy was "we'll do it if the customer asks for it, but we don't recommend it, because the success rate is 100%".
Re:Remote Prompt Injection in Gitlab Duo Leads to Source Cod...
by bluefirebrand on Tuesday May 27, 2025 @10:52AM
(#44107543)
How can you ever get that lower than 100% if you don't do the test to identify which employees need to be trained / monitored because they fall for phishing?
Re:Remote Prompt Injection in Gitlab Duo Leads to Source Cod...
by 33hsiidhkl on Monday May 26, 2025 @09:58AM
(#44097476)
It absolutely is deterministic, for any given seed value. Same seed = same output, every time, which is by definition deterministic.
Re:Remote Prompt Injection in Gitlab Duo Leads to Source Cod...
by tough on Monday May 26, 2025 @04:49PM
(#44101545)
only if temperature is 0, but are they truly determinstic? I thought transformer based llm's where not
Re:Remote Prompt Injection in Gitlab Duo Leads to Source Cod...
by 33hsiidhkl on Monday May 26, 2025 @06:05PM
(#44102113)
temperature does not affect token prediction in the way you think. The seed value is still the seed value, before temperature calculations are performed. The randomness of an LLM is not related to its temperature. The seed value is what determines the output. For a specific seed value, say 42069, the LLM will always generate the same output, given the same input, given the same temperature.
Why? Output isn't deterministic.