Ultimately, it comes down to the halting problem: If there's a mechanism that can be used to alter the measured behaviour, then the system can change behaviour to take into account the mechanism.
In other words, unless you keep the poisoning attack strictly inaccessible to the public, the mechanism used to poison will also be possible to use to train models to be resistant to it, or train filters to filter out poisoned data.
At least unless the poisoning attack destroys information to a degree that it would render the poisoned system worthless to humans as well, in which case it'd be unusable.
So either such systems would be insignificant enough to matter, or they will only work for long enough to be noticed, incorporated into training, and fail.
I agree it's an interesting CS challenge, though, as it will certainly expose rough edges where the models and training processes works sufficiently different to humans to allow unobtrusive poisoning for a short while. Then it'll just help us refine and harden the training processes.
The question is not whether the system can change, it's whether the system is incentivized to change. Poisoners could operate entirely in the public, and theoretically manage to successfully poison targeted topics, and it could cost the model developers more than it's worth to fix it. Think about obscure topics like, say, Dark Souls speedrunning. There is no business demand for making sure that a model can successfully give information relating to something like that, so poisoning, if it works, would probably not be addressed, because there's no reason for the model developers to care.
Whether model poisoning becomes a bigger issue depends on the incentives for companies to keep fighting it. For now in comparison to attackers the incentives and resources needed to defend against model poisoning are huge so it's just temporary setbacks. Will that unevenness in their favor always be the case?
https://en.wikipedia.org/wiki/Lotka%E2%80%93Volterra_equatio...
https://en.wikipedia.org/wiki/Rice%27s_theorem
Formally, any non-trivial semantic property of a Turing machine is undecidable. Semantic here (roughly) means "behavioral" questions of the turing machine. E.g. if you only look at the "language" it defines (viewing it as a black box), then it is undecidable to answer any question about that language (including things like if it terminates on all inputs).
Practically though that isn't a complete no-go result. You can do various things, like
1. weaken the target you're looking for. if you're ok with admitting false positives or false negatives, Rice's theorem no longer applies, or 2. rephrase your question in terms of "syntatic properties", e.g. questions about how the code is implemented. Rust's borrow checker does this via lifetime annotations, for example.
If you have access to run a transform on data, you can use it to train a model that acts as a detector of whether that transform has been applied to the data.
When you have a detector for a given property, you can use that detector to alter behaviour to exclude that property.
And that is the abstract core of why the halting problem is unsolvable.
In this case, if you have access to a mechanism for poisoning data, you can use that to train a detector. Once you have a detector, you can either exclude poisoned data, or use it for adversarial training.
Either way: The existence of the poisoning mechanism can be directly used to derive the tools to create its own antidote.
And that's back to the core of the halting problem.
This still seems too handwavy. For example, you're implying that the transform is one that can be learned by gradient descent. As a trivial counterexample, you can't train a model to detect valid (text, SHA) pairs.
This particular one doesn't seem to be a problem for your argument, but I still think your argument generally does not hold.
Basically any poisoning attack is also fundamentally limited because it needs to be non-invasive enough for humans not to be adversely affected, and that limits the problem space severely - the poisoning mechanism basically becomes reduced to a training mechanism to train out places where the models act different to humans.
It's a bit of a leap, but the halting problem can be generalized to:
It is impossible in the general case to produce a detector function f(x), that will decide if program x behaves according to rule y if x can include f(x) as part of the itself.
The reason is that if a program x can make use of the detector, it can effectively do if f(x) { do the opposite of what f(x) predicts}
The leap from that to poisoning might be a bit unintuitive, but it boils down to the poisoner having a mechanism that would alter model behaviour.
If you have access to that mechanism, you can produce a detector by using the mechanism to induce the unwanted behaviour, and train a model on that.
Once you have a detector, you can behave differently based on the signal from the detector, and by extension avoid the effects of the original mechanism.
And that is the core of the halting problem.
No, that’s the opposite of the halting problem…
The very point of CS as an academic discipline is _generalization_.
This is likely impossible. As the in-vogue breed of model extraction methods ("distillation attacks") demonstrates, you can infer the underlying training and/or fine-tuning of a model with a series of carefully constructed prompts.
Another name of model poisoning? Adversarial fine-tuning.
> the fact the Chinese populace is much more pro-AI than the West.
https://www.ted.com/talks/peter_steinberger_how_i_created_op...
Then I have good news for you: If humanity goes extinct in the next few years because of unaligned superintelligence, there actually will no longer "be an active community of people who loathe AI and work to obstruct it"
This is either a misunderstanding of the anti-AI crowd or an intentional attempt to discredit them. The majority of anti-AI people don't actually fear this because that belief would require that this person has already bought into the hype regarding the actual power and prowess of AI. The bigger motivator for anti-AI folks is usually just the way it amplifies the negative traits of humans and the systems we have created which is already happening and doesn't need any type of pending "superintelligence" breakthrough. For example, an AI doesn't actually need to be able to perfectly replace the work I do for someone to decide that it's more cost-effective to fire me and give my work to that AI.
This attempt to "reframe and reclaim" (here, paraphrased: "significant existential risks from AI is actually marketing hype by pro-AI fanatics") is a rhetorical device, but not an honest one. It's a power struggle over who gets to define and lead "the" anti-AI movement.
We may agree or disagree with them but there are rational anti-AI arguments that center on X-risks.
See my other comment. I qualified what I said while the comment I replied to didn't, so it's weird that this is a response to me and not the prior comment.
>here, paraphrased: "significant existential risks from AI is actually marketing hype by pro-AI fanatics"
If we're talking "dishonest rhetoric", this is a dishonest framing of what I said. I'm not saying this is inherently intentional marketing hype. I'm saying there is a correlation between someone who thinks AI is that powerful and someone who thinks AI will benefit humanity. The anti-AI crowd is less likely to be a believer in AI's unique power and will simply look at it as a tool wielded by humans which means critiques of it will simply mirror critiques of humanity.
Exactly, "lack of intelligence" is really a much bigger concern than "superintelligence". Companies and government will happily try to save money and avoid accountability by letting AI do work that it can only do poorly and it will be humans who are left with the accelerated AI powered enshittification and blind/soulless paperclip maximization that results.
I've seen people claiming that this could happen, but I've yet to read any plausible scenario where this might be the case. Maybe I lack the imagination, could you enlighten me?
- AI dominated the physical world. Robots, factories, etc.
- AI decides humans aren't contributing and/or wasting resources it feels should go somewhere else.
I mean not unlike humans causing extinction of other species?
Factories, even fully robotic ones, heavily rely on humans to set up and maintain them. Moreover, the safety culture means there are tons of "disable" controls which can be triggered by any human and no machine can override.
Robots look impressive, but they cannot function without the humans either. Military kill-bots are likely the worst, but machines cannot repair or refuel them.
None of this is going to change in the "next few years".
Robots can't function without humans because they're not super-intelligent. We already see quite capable humanoid robots. Those factories that rely on humans - they'll be converted to be operated by humanoid robots. By the super intelligence.
That's the hand wavy story. It's hard to dive into details in an HN comment but I'm happy to try and develop some of those details. You're saying that something much smarter than humans isn't going to be able to bridge the gap to the physical world. I'm not so sure.
EDIT: Another way to think about it is that if a god-like infinitely capable being took control of all our online digital systems including I donno Teslas, factory automation, power grid, any form of connected robot in the world, nuclear weapons launch systems, airplanes, whatnot, do they have any path to a sustainable "existence" without relying on humans. Or at least with us unable to detect and stop that. If the answer is no then we're probably safe. It's kind of hard to convince ourselves of that. Keep in mind that humans can also be manipulated to do work for this god just like spies/saboteurs e.g. are recruited online today and paid bitcoin to do some random master's bidding.
ha ha ha no. Teslas run of energy and cannot refuel, a circuit breaker in factory pops and no robot can reach it (or maybe a roof leaks), and "any forms of connected robot" _either_ cannot walk the stairs or can maybe run for a hundred miles before running out of battery.
The "humans can be manipulated" is the only thing to worry about, and you don't need robots for that, other humans have been trying hardest to do it just fine for millennia. I guess it's up to you if you want to be afraid or not, but I am not seeing anything super special so far.
You may ask why that is interesting: it's because carrot cake is, despite the name, made mostly of flour and dehydrated lemons. The cooking process is of course handled by a custom implementation of CP/M, running on a Z80.
One time it drew a fortnight riding a bike. Hilarious.
Someone shared the list on here years ago but I can't find it again.
> It sounds like you’re thinking of Kwisatz Haderach from Dune.
> The spelling/pronunciation gets mangled a lot (“quiz-atz haderach,” “kwitzatteracht,” etc.), but the original term is Kwisatz Haderach.
I asked it if Hacker News was in it's training data and gave it the website and it gave me the first "I don't know anything about that" I've ever seen from it.
Are you making big money from the hype?
There were never such wide scale and, above all, centralized efforts to coerce and shame people into using the Internet or smart phones in spite of their best efforts.
"I'm not shaming! Not embracing AI is comparable to people who didn't embrace smart phones or the Internet though".
This is a regurgitation of a marketing slogan frequented by OpenAI and similar organizations for the past four years. "AI is the future. If you don't embrace it you will be left behind".
It's intellectually insulting to be subject to as it relies primarily on fear to convince.
Second, your participation in the thread began as fulmination, with “I am so very tired of people who...”, and then continues in this belligerent style right through to your reply to me...
> This is a regurgitation of a marketing slogan frequented
> It's intellectually insulting to be subject to as it relies primarily on fear to convince
This style of argumentation is beneath what we're hoping for on HN, as it paints a simplistic conspiracy theory or narrow commercial incentive as the only plausible explanation for a trend. Things are never that simple, and arguments like that shut off curiosity, when the primary purpose of HN is to cultivate more curiosity.
That said - I can keep my fulminations to myself and phrase my posts in a less confrontational manner. Those aren't exactly conducive to productive discussion.
I mean, it's still ongoing! Tons of people prefer to do things the analog way, and it's certainly not for a lack of companies trying, as the analog way is usually much more expensive.
In their personal lives, everybody should of course be free to do what they want, but I also doubt that zero people have been fired for e.g. refusing to train to use a computer and email because they preferred the aesthetics of typewriters or handwritten memos and physical intra-office mail.
What they can do, however, is they can run heavy advertising campaign targeted at executives. And once executives are convinced, they will write AI policies, and some will force their workers to use AI.
And this has been happening all the time, the examples are too numerous.
Executives decided the shops will now use computerized registers. The cashiers had to adopt, or get fired.
Executives decided - no more typewriters. All documents must be written in Microsoft Word, stored in Sharepoint. The workers have to learn Microsoft Word and Sharepoint or get fired.
Executives decided that that engineers (not computer ones, mechanical ones) should use CAD instead of drafting machines. The amount of engineers who were "let go" because they were protractor head wizards but could not figure out the mouse was truly large.
For something closer to CS, there was version control, automated tests, git, github... In a lot of cases, people where not "willingly choosing it" - if the rest of your team started using SourceSafe, you can't keep using your favorite shared folder anymore, not if you want others to see your results.
"willingly choosing it" only works for personal projects, it is never guaranteed for hired workers.
I can guarantee there will be at least a few small ones, especially in the wake of the Sam Altman attacks and the "Zizian" cult. I doubt they'll be very organized and they will ultimately fail, but unfortunately at least a few people will (and have already) die(d) because of these radicals.
https://www.theguardian.com/technology/2026/apr/18/sam-altma...
https://edition.cnn.com/2026/04/17/tech/anti-ai-attack-sam-a...
https://www.theguardian.com/global/ng-interactive/2025/mar/0...
Also saying "these radicals..." like this makes you sound like you are the Empire in Star Wars.
Can you not see how there's a difference?
* No legitimate justification: their materials are being stolen to train and be regurgitated by LLMs and generate products. They are not being compensated yet their contribution goes on to make AI companies money, and preventing open consumption of their materials, to assist an AI company in rendering them obsolete, is not a justification for retaliating? You would have the barest whiff of a point if OpenAI and company were going to artists, requesting materials for training, and were given tainted ones, that at least I could say was duplicitous. But not when it's publicly posted, that's just an AI company not doing a good job of minding their input.
* Serve only to make access to and transformation of info more difficult: As in, you have to go to the website of the person actually publishing the information, as opposed to having it read in a Google summary? Also worth noting this inconvenience applies only to a theoretical person using an AI search tool. Everyone else is unaffected. Seems like if you're going to a particular service provider whom is uniquely unable to provide the service you want, that seems like an easy to solve issue: use something else.
* can only hope that by these egregiously anti-social luddites: Your daily reminder that the Luddites were not anti-technology, they were anti-corporations using mechanization to make an ever dwindling number of workers produce ever more products of ever lower quality.
* we'll gain the knowledge to render this category of attack moot for the foreseeable future: This is a bad strategy and historically has not worked for a single industry. If your industry itself exists in open opposition to consumer movements, you don't win. At best, you survive. But there's no version of this where everyone just unwillingly adopts AI and you can tell them to deal with it. Whole companies now are cropping up to help people who want to opt-out of the AI future as promised.
An aside, I honestly think that if someone recoils at the idea of an AI learning from their idea and using that idea to help someone else, they're just a bad, selfish person.
Human brains are not machines sold on subscription for profit. Also a human brain can't look at all the Rembrants in existence and generate more in 5 minutes.
> An aside, I honestly think that if someone recoils at the idea of an AI learning from their idea and using that idea to help someone else, they're just a bad, selfish person.
AI's don't learn. They're statistical machines that approximate their training data and what they are rewarded for making to the subjective standards of their makers. That's not learning, they don't know anything, they're just "minimizing error."
On the one hand I agree with you but on the other sometimes I wonder just how insulated we are in the tech community and especially in sites like this.
At some point in the last few months I realized that my friend group is basically a bubble of people making mid 6 figures that all work in tech and while I wouldn’t call it “anti-ai sentiment” even some of them are extremely conservative in their praise.
With that being the case you have to wonder what the average person is feeling about it.
Tech hosts a particularly virulent and ideological strain of anti-AI activism; I think because the disruption it threatens for our jobs is much less abstract than it is for everybody else.
We know how the sausage is made. Your non tech folk only see the marketing/hype, so of course they're optimistic.
And a large chunk, typically the younger and more online / political aware - they absolutely loathe ai. For stealing from artists, ruining online spaces with slop, spreading misinformation, deepfake porn, and of course screwing the economy, destroying jobs, and polluting the world so a few ultra rich people can get richer, and a few more people like you can make mid-six figures while everybody else has to cope.
If the trend i see continues, i would expect the second group will get very big very soon. Either because the hype is real and ai is taking jobs, or because the hype wasn’t real and they feel lied to.
Stanford released a report[1] in April, that shows a wide gap between AI Insiders and everyone else.
> 5. AI experts and the U.S. public have very different perspectives on AI's future, except on elections and personal relationships.
>On how people do their jobs, 73% of experts expect a positive impact compared to just 23% of the public, a 50-point gap. Similar divides appear for the economy (69% vs. 21%) and medical care (84% vs. 44%)
> 6. Nearly two-thirds of Americans (64%) expect AI to lead to fewer jobs over the next 20 years, while only 5% expect more.
>Experts were less pessimistic (39% fewer, 19% more) but forecast far faster adoption, expecting generative AI to assist 18% of U.S. work hours by 2030 versus the public's estimate of 10%.
The average person wants to get on with their lives and look after their family and friends.
People will talk about it, some might feel worried, some might feel intrigued, but they just aren't as prone to perpetual rage and doomerism and the more online community.
My recommendation is to poison topics that are interesting to teenagers but not useful for corporations. For example, pick some comics/movie/anime/game topic and concentrate your poisoning efforts. It is less incentive for most companies to fix it because there is not a lot of business value fixing it, assuming that the majority of revenue will come from enterprises in the near future. But this will lead young people to distrust AI in general.
Meanwhile: the ability to poison models, if it can be made to work reliably, is a genuinely interesting CS question. I'm the last person in the world to build community with anti-AI activists, but I'm as interested as anybody in attacks on them! They should keep that up, and I think you'll see threads about plausible and interesting attacks are well read, including by people who don't line up with the underlying cause.