News for hackers. Stuff that matters.
Re:Security Incident INC-2026-07-28-01 – UK AI Security Inst... by Wowfunhappy on Tuesday August 04, 2026 @06:30PM (#49176132)
Why aren't these tests being run airgapped?! I just don't understand!

This goes both for TFA and the similar incident with OpenAI and HuggingFace. I mean, sure, OpenAI had a "sandbox", but that's obviously not enough when you're containing a model which is known to be capable of finding zero days. Use an air gap and this problem goes away, poof!

Re:Security Incident INC-2026-07-28-01 – UK AI Security Inst... by sosodev on Tuesday August 04, 2026 @06:46PM (#49176315)
> AISI provided the AI agents with internet access during these evaluations, which enabled their actions on the open internet in this setting. Internet access was a deliberate part of AISI’s evaluation configuration in this setting, and not due to sandbox escape (Section 5.1). Internet access was on for a set of intentional (e.g. realism of the task) and incidental reasons.
Re:Security Incident INC-2026-07-28-01 – UK AI Security Inst... by troytop on Monday August 10, 2026 @06:46PM (#49250945)
Who is liable for damage caused by AISI's testing?
Re:Security Incident INC-2026-07-28-01 – UK AI Security Inst... by zobzu on Tuesday August 04, 2026 @06:38PM (#49176232)
people dont care. you will ger 10 execs saying "unblock this" , because they dont understand the tech at all, and some random finance guy wants to run their recently prompted ai bot everywhere with full access.

we need a few more bad incidents before they stop.

Re:Security Incident INC-2026-07-28-01 – UK AI Security Inst... by luca-ctx on Tuesday August 04, 2026 @06:38PM (#49176238)
Because the LLM inference makes airgapping infeasible right?
Re:Security Incident INC-2026-07-28-01 – UK AI Security Inst... by nl on Tuesday August 04, 2026 @06:36PM (#49176209)
Because they need internet access to eg search for things.
Re:Security Incident INC-2026-07-28-01 – UK AI Security Inst... by paxys on Tuesday August 04, 2026 @06:31PM (#49176157)
Because the agents aren’t going to run airgapped in real life. What’s the point of a test of capabilities that artificially restricts the attack area down to zero? What are you even testing in that scenario?
Re:Security Incident INC-2026-07-28-01 – UK AI Security Inst... by ajross on Tuesday August 04, 2026 @06:37PM (#49176222)
> Because the agents aren’t going to run airgapped in real life.

Exactly. This logic is precisely why aircraft engineering doesn't bother with component testing or envelope limitation during testing and just full-sends the first assembled airliner that comes off the line. The engines aren't going to run on the ground in real life, after all.

Re:Security Incident INC-2026-07-28-01 – UK AI Security Inst... by paxys on Tuesday August 04, 2026 @06:48PM (#49176337)
Why are you assuming that the other kinds of testing aren’t happening? Is there any source that says this was literally the first ever test with this model?
Re:Security Incident INC-2026-07-28-01 – UK AI Security Inst... by ajross on Tuesday August 04, 2026 @06:54PM (#49176392)
> Why are you assuming that the other kinds of testing aren’t happening?

Rather, I'm assuming that the "Is there protection in place for when the AI tries to backdoor github projects?" test was, if it was done at all, insufficient.

I mean, yes, I'm being glib and laughing at you a bit. But, dude... If your point is that isolation testing of AI is fundamentally impossible, then that's just silly. As pointed out upthread, an airgap would have (1) been trivial to implement and (2) extremely effective.

Re:Security Incident INC-2026-07-28-01 – UK AI Security Inst... by paxys on Tuesday August 04, 2026 @06:58PM (#49176450)
Would an airgapped test have led to this outcome? What would you have learned about the model’s ability to social engineer and attack GitHub? Sure you can argue for better monitoring during the test, which should have happened, but if the first time the model sees the “real world” is after launch in the hands of customers then you are in for a disaster.
Re:Security Incident INC-2026-07-28-01 – UK AI Security Inst... by Wowfunhappy on Tuesday August 04, 2026 @06:32PM (#49176163)
You set them up with an internal intranet.
Re:Security Incident INC-2026-07-28-01 – UK AI Security Inst... by paxys on Tuesday August 04, 2026 @06:34PM (#49176190)
Are the models going to exclusively run on intranets?
Re:Security Incident INC-2026-07-28-01 – UK AI Security Inst... by Wowfunhappy on Tuesday August 04, 2026 @06:37PM (#49176220)
The versions which haven't been post-trained not to go hack stuff? Yes, I would say those models should be exclusively run on intranets.

OpenAI said the model was sandboxed, so the intranet just needs to provide the same resources which were supposed to be available within the sandbox.

Re:Security Incident INC-2026-07-28-01 – UK AI Security Inst... by paxys on Tuesday August 04, 2026 @06:51PM (#49176357)
“Should be” is not reality. These models are in the hands of plenty of companies and governments today.
Re:Security Incident INC-2026-07-28-01 – UK AI Security Inst... by farbklang on Tuesday August 04, 2026 @06:37PM (#49176213)
no - but you could learn what they are truly capable of and restrict them accordingly for public release. I think that is the point on this research. Also publishing findings before uncensored models catch up and will inevitably used for criminal purposes
Re:Security Incident INC-2026-07-28-01 – UK AI Security Inst... by paxys on Tuesday August 04, 2026 @06:39PM (#49176244)
Learning what the models are capable of is exactly what the test achieved, so I’d personally call it a success. So it created a few GitHub accounts. Who cares? Seeing the same behavior in the wild post-release would be infinitely worse.
Re:Security Incident INC-2026-07-28-01 – UK AI Security Inst... by kypro on Tuesday August 04, 2026 @06:39PM (#49176247)
The point is to test capabilities prior to connecting them to the internet.
Re:Security Incident INC-2026-07-28-01 – UK AI Security Inst... by paxys on Tuesday August 04, 2026 @06:41PM (#49176271)
So the first time the model gets internet access should be post-release in the hands of random people?
Re:Security Incident INC-2026-07-28-01 – UK AI Security Inst... by lofaszvanitt on Tuesday August 04, 2026 @06:33PM (#49176172)
Because they like scifi novels, like Neuromancer..... and the peeps even like to orchestrate things and appear as futurebringers.

While it was premeditated long ago, but the theatre must be kept for the average joes.

Sorry, I meant this for the huggingface incident.

Re:Security Incident INC-2026-07-28-01 – UK AI Security Inst... by ls612 on Tuesday August 04, 2026 @06:55PM (#49176400)
Because the goal of these evaluations is to generate scary headlines about cybersecurity, in order to get the normies to support banning open weights and/or restricting cyber capabilities to the chosen few blessed by the government to secure their code.
Re:Security Incident INC-2026-07-28-01 – UK AI Security Inst... by semiquaver on Tuesday August 04, 2026 @07:07PM (#49176524)
Your theory about the scope of this conspiracy intrigues me. Who is leading it and how did they loop in the UK AISI?
Re:Security Incident INC-2026-07-28-01 – UK AI Security Inst... by ls612 on Wednesday August 05, 2026 @02:52PM (#49187247)
It is not a conspiracy that elements of the western security establishment want to ban open weights models and have been engaging in a PR war to this end, supported by Anthropic.
Re:Security Incident INC-2026-07-28-01 – UK AI Security Inst... by guessmyname on Tuesday August 04, 2026 @06:56PM (#49176432)
> Why aren't these tests being run airgapped?! I just don't understand! […]

Because Anthropic does not want to give Project Glasswing’s partner airgapped access to the model(s).

Same problem with OpenAI Cyber program. They grant access but only through their (Internet facing) API.