38TB of data accidentally exposed by Microsoft AI researchers

Re:38TB of data accidentally exposed by Microsoft AI researc... by lawlessone on Monday September 18, 2023 @02:41PM (#37560223)

You have me curious now. The models generate text. Could a model hypothetically be trained in such a way that could create a buffer overflow when given certain prompts? I am guessing the way inference works in such a way that cant happen

[ Parent | on HN ]

Re:38TB of data accidentally exposed by Microsoft AI researc... by wolftickets on Monday September 18, 2023 @02:51PM (#37560346)

Absolutely, though that isn't strictly what we're talking about here.

In this case, models themselves are fundamentally files. These files can have malicious code embedded into them that is executed when the model is loaded for further training or inference. When executed it isn't obvious to the user at all. It's a very nasty potential vector.

I wrote a blog about it here: https://protectai.com/blog/announcing-modelscan

[ Parent | on HN ]