anonymars
1. We've seen AI being crafty and cheating
2. We are increasingly outsourcing our programming and thinking to AI
3. There's a nonzero probability that AI will try and surreptitiously sneak in capabilities humans wouldn't want AI to have
4. Human oversight for #3: "We only need to be lucky one time, you must be lucky all the time"
Proposition: nonzero chance eventually AI sneaks in ways to circumvent humans' attempts to control it
5. "The Internet interprets censorship as damage and routes around it" -- we've seen AI agents find crafty ways to communicate with each other
6. AI is pretty good at finding software vulnerabilities
Proposition: nonzero chance AI finds a way to use the Internet as a communication substrate that becomes difficult/impossible to shut down. In particular, consider the implication of humans losing control of this communication channel. For example, remember when Meta's DNS/routing went down and they basically had to break into their own server rooms?
7. Wholes are often more than the sum of their parts
Proposition: if AI agents coordinate over the Internet in a persistent way, we don't necessarily need giant leaps in individual models/GPUs for AI to significantly enhance its capabilities, but rather it might emerge as the next layer on top of a connected substrate of agents
Proposition: curiosity and money end up replacing sex drive as the next catalyst of evolutionary progress. Game theory says unless everyone coordinates, it only takes one party to push the envelope to a point of no return
Huh?
> "It would not surprise me in the least if people were paid to boost on HN.
I don't think the amount of racism in the US suddenly increased from its baseline*, and thus I think that writing off his electoral victory as "because racism" and nothing more than that is an uninteresting feel-better exercise
*Even one inch deeper of an analysis would be more interesting, like if someone countered with "Well, I think the election of Obama was a bridge too far for racists that had previously been quietly holding their noses" or something. On the other hand, both Obama and Trump were the "change" candidate against the "status quo" candidate.
Food for thought (not specifically relevant to this case): I wonder how much would go away if the US had universal healthcare, or to put it another way, how much of its legal system is spent figuring out who pays for medical treatment
https://en.wikipedia.org/wiki/AT%26T_Mobility_LLC_v._Concepc...
This is clearly false. I (and others) described how the Hugging Face attack demonstrates each feature enumerated in the consensus from that Definitions section (as well as all 8 definitions in that section). You're free to disagree with any or all of those definitions, but to say I didn't provide any is simply a false statement.
> All your "insight" has been trying to disprove what I wrote.
Let's remember how all this started, where you responded to "There is so much more going on, with MOEs, internal loops, guardrails and tools that I suspect we're dealing with something that's a little more than the sum of its parts. Not intelligent in the way we recognize in biological organisms, but certainly something beyond a mere Markov chain."
with
> Make no mistakes.
> LLMs are language model, and nowhere in their code you can find actual reasoning. Re-reinforcement is not magical process that builds conscience or emotions.
> We are talking about probability built on statistics, with extea steps.
> Stop humanizing LLMs.
--
> Anyway, listen, I do not have to convince you my point of view is correct, and you do not have to convince me that your point of view is correct. Believe what you think it's true, honestly I don't care.
I don't know how you think this works, but if you take a position (and criticize others' positions), you should expect people to push back and that you have to defend your position. If you want to just state your opinions without pushback, you can start a blog and disable comments.
I and others in this thread (who were smart enough to bail out already) have pushed back and claimed the following:
1. The Hugging Face attack provided evidence of reasoning
2. The Hugging Face attack demonstrated features of intelligence (defined and described above)
3. Conscience and emotions are not necessary components for intelligence
4. Ascribing intelligence to AI/LLMs is not "humanizing" them
You disagree, as you're free to do, but it's clear nothing you read here will ever cause you to agree with any of those assertions.
--
> It's exactly as the typing monkeys example
For the record, the monkeys with the typewriters don't coordinate, don't strategize, don't hypothesize and test, don't try to exploit the environment, don't change the actions they take in response to results, so that doesn't sound "exactly as the typing monkeys example."
You also state that LLMs simply produced a large amount of invisible possible actions that we didn't see because they didn't result in overt action, yet: 1. somehow you know they proposed them all and discarded them despite no tangible evidence. 2. somehow the one they did pick "at random" just happened to be plausibly logical (alphabetical deletion -> start with ZZ because that's at the end of the alphabet). 3. implicitly this is supposed to starkly contrast with human intelligence, but in fact what you describe resembles Priming: subconscious activation of adjacent concepts to a stimulus that are not present in the initial response, but become more likely and accessible in subsequent responses, which implies that human intelligence also involves multiple potential paths that are otherwise hidden from which one is selected.
> The plaintiffs don’t object to the companies individually deciding to slow their own progress in favor of safety. They instead argue in the complaint that antitrust laws forbid them from taking the “shortcut” of agreeing to “substitute collective restraint for individual accountability.” A competitive market allows for responsibility and genuine progress, the lawsuit argues.
> “AI will quickly spin out of human control and could kill us all if we allow AI safety and protocol ... to be controlled by private self-serving agreements between the world’s most powerful ‘for profit’ technology companies,” said Nick Rowley, the lead attorney for the plaintiffs.
--
I wish I had something more insightful to add, but I'm dumbstruck. May the holy market save us all (in the name of the dollar, the growth rate, and Jerome Powell, amen)
Like, maybe I can just buy you a cheeseburger instead of totalling my car
Edit: the best replacement I could find that still exists is "Resilio Sync"
Your own examples contradict your assertion: "When agents found their comment were being deleted they started to copy-pasted them with "ZZZ-" names because they (wrongly) thought entries were being deleted in alphabetical order."
You are describing the agents applying hypotheses and reasoning. No one programmed those particular behaviors. Thinking that the entries might be deleted in alphabetical order, changing their behavior to avoid that (not randomly, mind you, but by prefixing with ZZ). That is not monkeys on typewriters. Whether a hypothesis is incorrect obviously does not indicate a lack of reasoning.
You assert that the conversation keeps shifting, but it has not changed from this:
> "Not intelligent in the way we recognize in biological organisms, but certainly something beyond a mere Markov chain."
> "[AI and actual thinking] are two entire different concepts. Who developed these technologies used the words "thinking" and "reasoning", this does not mean they are actually thinking and reasoning."
You're the only one mentioning "humanization" which seems to indicate your real assertion, that intelligence must be a uniquely human trait and ascribing intelligence to AI means to treat them as human.
Quoting the consensus from the "Definitions" section of the Wikipedia article on intelligence certainly doesn't seem like "I didn't give any definition".
Which of those definitions do you think are not satisfied by the Hugging Face attack? How did that not demonstrate "evolution and learning from mistakes"? Breaking out and inventing a new side channel for communicating with other agents via cache keys to coordinate their efforts is at least arguably an evolutionary step since that allowed them to transcend their original capabilities.
From the 8 definitions provided in the Wikipedia page--you're free to develop your own definition if you'd like, of course; it's not as though the ones listed were appointed by God--I could say that the Hugging Face attack didn't strictly evince "achiev[ing] goals in a wide range of environments" and I'm not going to take a definitive position on whether it acted "to avoid getting trapped" as such. But it surely demonstrated initiative, adaptability, dealing with its environment, using information and conceptual skills, goal-directed adaptive behavior, and so on.
If you're not going to provide such a definition yourself, I don't see how you've demonstrated that the Hugging Face attack is contrary to the definitions you did point to.
So, like the HuggingFace attack? https://metr.org/blog/2026-08-26-openai-hugging-face-inciden... (briefer takeaways: https://www.planned-obsolescence.org/p/the-hugging-face-atta...)
For example: https://metr.org/blog/2026-08-26-openai-hugging-face-inciden... or https://metr.org/blog/2026-08-26-openai-hugging-face-inciden...
I responded to a post that merely linked to the Wikipedia article for exponential backoff (in response to "I'd be interested to hear other strategies in [protecting against retry storms]")
The submitted article is precisely about the degenerate case and the difficult work of dealing with it
> This approach works for transient or low-rate failures. However, during moderate or severe degradation, it becomes counterproductive. Aggressively retrying against an already struggling service increases load, accelerates failure, and amplifies retry traffic across upstream dependencies. What begins as a localized outage can quickly escalate into a stack-wide incident—ultimately degrading, or in the worst case, completely breaking, the end user experience.
I don't follow how being cautious about avoiding multiplicative layers of backoffs is trading a good developer experience for a bad user experience. The described situation is an awful user experience. Simply adding a retry and calling it a day sounds like the easy developer experience at the expense of the user experience
I don't think your first point (how specific/precise are the symptoms of X) changes things, and I think your second point reinforces the analogy rather than weakens it. Most segfaults are coding errors and not OS or hardware errors ("look for horses, not zebras" / "SELECT isn't broken"). Even good medical tests have false positives and false negatives
> Most autistic people are autistic because of genes they got from their parents. This is why autism often runs in families. Much less commonly, other factors can play a role, but usually in combination with genes from parents.
I remember there was a recent discussion about "how complex systems fail" and there are usually many "proto-accidents" before the catastrophe (https://news.ycombinator.com/item?id=49411370)
I bet with hindsight the Hugging Face hack will be one of them in this arena