Re:Agentic backdoors via pretraining poisoning
by NotAnOtter on Monday November 02, 2026 @05:34PM
(#50344979)
I think open source models could be vulnerable to these attacks. So long as I sandbox everything then I am as secure as my sandbox which is air gapped from my personal stuff i.e. no models on the same machine I log into my bank.
I know closed source models will have nefarious shit injected into it. It will start modest at first, of course, they need to get past the hyper scrutiny phase. Then it will simply be telemetry, then it will simply be feeding my behaviors into ad targeting, then it will simply be computer vampirism, and so on. One step at a time.
I prefer a gremlin I know I need to keep chained than a seductress I think I can probably trust.
I know closed source models will have nefarious shit injected into it. It will start modest at first, of course, they need to get past the hyper scrutiny phase. Then it will simply be telemetry, then it will simply be feeding my behaviors into ad targeting, then it will simply be computer vampirism, and so on. One step at a time.
I prefer a gremlin I know I need to keep chained than a seductress I think I can probably trust.