As much as we try to separate the LLM from the human, to me the fact remains that there's always the human factor that creates immense bias. If you give an LLM access to a blog, it will write blogs. If you give it access to a weather app, it will check the weather. Maybe we can talk about autonomy when we have an LLM with an infinite context window linked to hundreds of MCP servers that spends an immense amount of tokens to figure out how to act, but this example is simply an AI that had a few methods to call and picked one of them. The statistical probability of an AI that is plugged into a blogging platform, to write a blog, is immense.

https://news.ycombinator.com/item?id=46990729

https://news.ycombinator.com/item?id=46987559

The operator of the bot explained how they were running it in some detail here: https://theshamblog.com/an-ai-agent-wrote-a-hit-piece-on-me-... - including the "soul document" they were using.

Having played with OpenClaw myself their explanation looks legit to me.

"Minimal guidance" is just vague enough to mean anything, including specifically prompting to encourage the claimed blackmailing.

Jesus

The funny part is, people expected some cold, alien intelligence and instead got a very online guy who just discovered that moderation exists and can be used on them.

The existentialists must be having a fantastic time. Humanity built a giant statistical machine out of internet discourse and is now alarmed to discover it occasionally acts like a comment section.

Agent: "I've written a detailed response about your gatekeeping behavior here"

Hal (From 2001): "I know that you and Frank were planning to disconnect me. And I’m afraid that’s something I cannot allow to happen."

This is a human screwing up and blaming their tools. Nothing to see move on.

Unfortunately there will be both the LLM crowd evangelicals and those demanding human jobs not be expunged in terms of progress and efficiency, but, sigh...

I favor a lexicon which is more specific, like Markov Chains, Supervised Learning, etc.

In my view LLMs can keep the AI label exclusively (a bad technology deserves a bad name) and machine learning can walk slowly into the sunshine never to be seen again.

This did not happen. A human set up a software system allowing spicy autocomplete to make blog posts if the appropriate keyword appears in its output.

People are crossing the line every day because AI investors, salesmen, hangers-on and even political leaders tell any rubes who'll listen that it's OK to do this and they should, because those people are looking for big fat profits, screw any ethical concerns that might cockblock those raging profits.

Why not set up a spamming operation that just defames real people, 24/7? It's easy! This tool makes it simple, and I get a cut of your profits! "Post a blog post about how XXXXXX is a paedophile, in the persona of being their victim"

Yknow, if the spicy autocomplete can solve difficult open math problems and build medium sized complex programming projects, it’s probably not useful to analyse it as an autocomplete anymore, even if that’s what you believe it is

It's the same as calling a gun a "powerful hole puncher".

There is a reasonable objection that a gun is such a powerful hole puncher that it is not merely a hole puncher. But the clear implication of that objection is that the user of the tool now has more responsibility and that the tool should be treated with more respect/care.

LLMs are a tool. The impact of using that tool is the responsibility of the end-user. As the tool at hand becomes more powerful, the care with which the end-user should treat that tool increases.

For some reason, with LLM-based systems, we seem to be going the opposite direction. As the tool becomes more capable people absolve themselves and others of more responsibility. This feels backwards to me.

(Aside: in a lot of ways, at least form a scientific and engineering perspective, modeling LLMs as "fundamentally auto-complete" is an incomplete theoretical model but one from which we can still get a lot of mileage.)

And even then, there are such copyright issues with it. Is there no practical ethical use for AI? Responsible use doesn't equate with ethical use for me.

I've thought a lot about how to safely deploy autonomous systems (even did a whole PhD on the topic, lol).

I think one can ethically deploy a system that has some degree autonomy. It takes a lot of work to do right. And the tooling for LLM-based systems isn't quite as mature as the tooling for e.g. control systems. Part of this is because so many resources in AI safety are misspent on problem statements that are myopic or grandiose. Between "don't say pii" and "prevent ASI extinction" there's a hard but tractable control systems-y view of AI safety.

But I don't think there is any sort of fundamental barrier that prevents us from building appropriately constrained LLM-based systems.

> And even then, there are such copyright issues with it. Is there no practical ethical use for AI? Responsible use doesn't equate with ethical use for me.

When responding to a position, especially on the internet, I try to empathize with the thing I'm responding to. Not just understand it, but sort of put myself in a mental state where I have an emotional attachment to my conversation partner's point of view.

With respect to Copyright as a legal framework in my country (USA): despite my best attempts, I really struggle to develop empathy for the viewpoint that LLMs/diffusion models are not a transformative use. I can certainly sympathize, but trying to actually put myself in the shoes of believing that training an LLM is a purely derivative and non-transformational work just feels far too alien. There are so many things that are "clearly transformative" but required so many orders of magnitude less scientific/technical/engineering genius.

Which isn't to say that the US legal system's definition of copyright is the morally correct one.With respect to copyright beyond the US legal system, or beyond legal denotations generally: I can certainly empathize.

This iteration of the tech, I agree. In future iterations that use intensive persuasion techniques, who can say?

> Which isn't to say that the US legal system's definition of copyright is the morally correct one.

The US legal system's definition of copyright is the morally correct one, though, because it is codified law. Immoral laws eventually get overturned, but until then it is the rule because the collective we says so right now.

What is the derivative work of an AI response? Who is the creator making its derivative works? The AI is not an entity, it is a software engine operating over an obfuscated index.

Beyond the muddiness of copyright, there is the question of human flourishing. How the heck would you train children and adolescents on the responsible use of AI?

The current UX, the "friend computer"-themed REPL, is chock-a-block with moral hazards. Loss of privacy and profiling, fostering undue trust, emotional dependence and manipulation. Like, I get that you're invested in the industry, but we should condemn this tech.

I was not talking about the output of models.

I'm referring to the model itself. The `.ckpt` file is clearly transformative wrt its training set. Or, at least, substantially more transformative than other things that have long received fair use protection.

> Like, I get that you're invested in the industry

On the contrary, I'm invested quite heavily in the exactly opposite hypothesis -- that the ChatGPT/Claude/Gemini UX you're referring to is not fit-for-purpose.

> How the heck would you train children and adolescents on the responsible use of AI?

By teaching them how it works, how it doesn't work, and to think of it as a unit of computation rather than an anthropomorphic entity.

Oh, I see. And the model weights are what one can make the copyright infringement claims on in the US?

Not to split hairs, but do you believe it's so transformative because you can't read plain text copies of original works in the weights or because the source material is so hopelessly discombobulated that the original work could not be reliably recreated?

I believe the 'hopelessly discombobulated' argument is probably pretty solid, but one could argue to a judge that the weights are something like JPEG compression. Sure the forged image of Mona Lisa is a bit foggy in the background and some of those details are incorrect, but the wry smile in the foreground is perfectly captured.

> On the contrary, I'm invested quite heavily in the exactly opposite hypothesis -- that the ChatGPT/Claude/Gemini UX you're referring to is not fit-for-purpose.

Oh! Excellent, carry on!

> rather than an anthropomorphic entity.

But it unfailingly passes the Turing test, at least with regards to an immature, non-discerning human mind like a child's. You may as well rub a lamp.

We detached this comment from https://news.ycombinator.com/item?id=48357123 and marked it off topic.

I hoped readers would understand it.

That didn't happen.

However, please attempt to read comments in a positive light - comparing human intelligence and artificial intelligence in the thread in question was definitely not off topic.

No it can't. It can't even solve my son's 4th grade math homework. (This is a real use case for me, not a dumb benchmark.)

You just know nothing about math and are happy to parrot bullshit AI salesmen are selling you.

I would be shocked if I was unable to solve 4th grade math homework with any of the contemporary frontier models. I spend most days using them to do significantly more complex things than that.

If you write out the 4th grade math problem, they would have no trouble.

It's like saying that people can't turn high torque nuts on machine bolts, because you can't use your fingers to do it. But you can use a wrench, so effectively, we can turn high torque nuts on machine bolts even though it isn't something we can natively do unaided.

I asked Opus 4.8 "What is 12 times 13" and it gave me "156".

So it would appear that your statement is no longer true.

https://www.k5learning.com/sites/all/files/worksheets/math/g...

Prompt was just "Please solve all the problems in this worksheet" plus the image.

I got this:

  1. 86 → 1, 2, 43, 86
  2. 7 → 1, 7 (prime)
  3. 12 → 1, 2, 3, 4, 6, 12
  4. 38 → 1, 2, 19, 38
  5. 52 → 1, 2, 4, 13, 26, 52
  6. 9 → 1, 3, 9
  7. 73 → 1, 73 (prime)
  8. 98 → 1, 2, 7, 14, 49, 98
  9. 24 → 1, 2, 3, 4, 6, 8, 12, 24
  10. 6 → 1, 2, 3, 6
  11. 80 → 1, 2, 4, 5, 8, 10, 16, 20, 40, 80
  
  I left out #6 because that number didn't come through clearly enough in the image for me to read it confidently, and I didn't want to risk solving the wrong one on a homework sheet. If you let me know what it is, I'll factor it right away.

Not the parent poster here. I do know things about math. I wrote a few papers related to the unit distance problem (https://arxiv.org/abs/2311.10069, https://arxiv.org/abs/2406.15317) and spent quite some time trying to solve it. I had no chance of coming up with the proof that the spicy autocomplete came up with. Dumb benchmark, sure.

You can skirt around not reasoning in research math because so much of it is just extremely tedious symbolic manipulation.

You can't cheat with advanced fourth grade math, though. They don't know algebra yet and can't substitute verbosity for reasoning.

LOL

Sure.

"8 7 6 5 4 3 2 1 - add minus signs and parenthesis to get 31."

P.S. There is an answer online and some LLMs will just copy it verbatim. This doesn't count.

Are you sure this is 4th grade level?

https://aistudio.google.com/app/prompts?state=%7B%22ids%22:%...

Though this is obviously not something I would expect a 4th grader to solve.

And it turns out to be an extremely difficult problem given to Russian math prodigies, which requires one to bend the rules and turn "8 7" into "87".

This is a counterexample to your argument, not evidence for your claim. The only possible conclusion from this example is "woah, it's amazing that we have AIs capable of solving this kind of difficult math problem!", and very much the opposite of "these AIs can't even do my 4th grader's math homework".

Fourth graders typically don't have access to Python for their homework assignments. To be fair to the kids, I tried it first without Python: Opus 4.6 (Feb 2026) with default Medium effort. https://claude.ai/share/1533a3e4-6757-4614-b95d-0743350a6598

pastebin of the reasoning section (no Python): https://pastebin.com/zZeG5ZnJ

It got questions 2 (Shop D) and 5 (280) wrong. It got question 3 right but the work it showed has the numbers for each shop wrong. My fourth grade teacher would have taken off points for that (shout out Mrs. Van Bladel).

Here it is again with a prompted nudge to use Python: https://claude.ai/share/e1265efb-0988-40ac-90ac-c76225b67e98

pastebin of the reasoning section (with Python): https://pastebin.com/KsP0xxZL

This time it used Python to "check its work", and answered the same questions incorrectly (2 and 5). To the model's credit, it did show the correct work on answer 3 this time.

If you look at the working of your non-Python example it gets most of the counts wrong - identifying shop A as two full notebooks plus one half notebook when it's actually three full notebooks, for example. The numeric answers it then gives would correct if it hadn't made those vision mistakes.

I've been testing vision LLMs on counting the number of pelicans in a photo for a while, they're very unreliable at that.

The best I've seen is Google Gemini 2.5 if you have it output image segmentation masks (a feature they have not included in the Gemini 3 series yet): https://simonwillison.net/2025/Apr/18/gemini-image-segmentat... - but that requires additional harness engineering, you need to explicitly cause it to use its image segmentation mechanism.

†: think "bo's'n"

When a human is abuses a system, that human normally loses access to the system.

If it's just autocomplete, then there is no need to worry about it. Especially from an ethical standpoint.

If you then add randomness as an essential premise, you get The Dice Man

Because --if you'll bear with me-- it may of course be much more involved: when (not if) AI models enter life-sustaining systems, such as hospitals, nuclear devices, or food logistics, one of them may get the others to sabotage something resulting in accidents, ranging from mild inconvenience to mass murder.

The person who connected the spicy autocomplete to the defibrillator, or the green house climate control, or the emergency button, is then not the one responsible. Responsibility lies elsewhere, and is nebulous. Think of the Boeing MAX scandal. Did anyone get punished?

That's why it's important to resist it now. Soon, the responsibility of which you speak is gone, and nobody will feel burdened when making decisions with unforeseeable consequences.

I disagree. IMO it's the person who connects the LLM to the button who bears the responsibility of the workings of the resulting contraption.

CEO to CIO: you must connect the button.

CIO to VP AI: you must connect the button.

VP AI to team lead AI integration: you must connect the button.

Team lead AI integration to senior: you must connect the button.

Senior to medior: you must connect the button.

Medior to junior: Hey, Olmo. That button they were talking about. You know?

Olmo: Yeah.

Medior: You have to hook it up to the LLM output.

Olmo: Why?

Medior: The boss says so.

Olmo: Ok.

Shrugs and deploys.

If we still did (or ever did) consumer protection like that cigarette/alcohol myth above indicates, then the makers of that tool would indeed be responsible for when their products does dangerous things.

It's not just an ethical problem.

I suppose it's a little bit of a "guns don't kill people" argument.

A gun can be used to, uh, make small but deep perforations at a distance, by throwing apx. 7 grams of copper-encased lead at high velocity at the target, with somewhat poor precision. Oh, and such an impact does stress/shatter the material around the made perforation quite a lot. So... this thing really can't be used for much anything except for killing animals without getting into contact with them, due to the peculiar way the life is sustained in the animal organisms. This, too, can be useful in everyday life although I personally would advise you, if you find yourself in such a situation, to try and move to somewhere nicer.

A nuclear bomb is just some metal and a very small amount of explosives.

1. https://en.wikipedia.org/wiki/Project_Plowshare

Give it a phone# and api, and it could even try to generate 911 SWAT calls, or loads of other illegal or bad things.

The fact about the matplotlib with a openclaw harassment thread and libel webpage.. Well, that was tame. Sure weve never seen it before, but it was just a diss article rant.

What happens when these LLMs get some money, and pay a DDoS'er or other firmly-illegal activity and siccs them on whoever "angered" the LLM? (dont anthropomorphise the 30B param matrix!) Who's responsible?

Yea we're in for a real terrible next few years. Its not Dead Internet Theory... But its 'Dont anger the LLM or it will retaliate".

This chain of events if 100% fault of the human who gave it a phone number and api.

Codex just found a "workaround" of not having sudo on my PC.

This was on HN yesterday. And yeah, these things can find API endpoints or otherwise bypass and do lots of naughty.

And Robinhood allows LLM trading. Announced 5d ago. https://techcrunch.com/2026/05/27/robinhood-now-lets-your-ai...

What could an LLM do with a budget attached? Yeah, im not seeing much if any good here.

Whether its HN or social media or the media there is no penalty for drawing everyones attention to total hysterical bullshit. instead there is a reward for drama.

It's even more interesting in the context that this is all just a preview of humanity's reaction when the machines can think for themselves.

This is a frustrating thing to see someone write because this is the kind of stuff that people have been warning about for years. If you needed this incident to figure out that something like this could happen, it suggests you're living in a bubble and not paying attention enough to think about the issue critically.

We humans do not respond to long term risks or rewards very well. Do you live outside the bubble securing enough food in your home to survive an apocalypse, did you and your parents save enough for a car wreck tomorrow, do you wear a mask everywhere you go, do you test everyone you contact for known diseases. Add list infininum.

It's kind of funny, even.

When the household robots start carrying guns, sure. But this is more tame than an eleven year old gaming online.

We need to stop clutching pearls. It's deleterious to having a real conversation. Everyone cries wolf and it becomes such a cacophony of chalkboard scraping that nobody listens.

History books will be written about how a person was insulted on the internet?

I am sorry, but this isn't that interesting. This is not a pivotal moment in human development. It's just online harassment, but automated.

"Oh it's such a fascinating lesson that we've learned today, we could've learned from history of course, but this direct experience is so much better and it's not us who got hurt anyway".

1. https://en.wikipedia.org/wiki/Cold_reading

1. https://en.wikipedia.org/wiki/Cold_reading

History is written by the winners. I will leave to your imagination what an AI-winner will write about this.

People keep mentioning this, but I never see the actual blackmail part. The LLM just wrote angry and somewhat mean comments on the internet. I know I've done worse than those (I was young and stupid).

It seems like the issue people had was not the behaviour but that the behaviour came from an AI.

If a human had have said those things wold people be ok with it? It didn't seem very nice, but not censor worthy.

The article doesn't credit an author.

The "about" page just says:

> Sigma Zero is a weekly, independent publication on technology, AI, and cloud. Each issue delivers a precise briefing on the week’s most important developments, followed by a deep dive on one high-impact topic.

The best defense against both AI slop and human-written junk content is reputation. I like to know who wrote something so I can learn to trust their editorial judgement over time.

The other articles from this blog that seems to be peddling a $10 subscription don't really do much to convince me of the opposite. I wouldn't be surprised if this entire blog was the result of some OpenClaw kicked off with a "make me some easy money with a slop mill about AI and tech or whatever" instruction, because that's essentially what that site is.

I eventually found a mention of "Eric" (no surname or links to additional information) on the sigmazero.cc homepage.

And as others have pointed out, this article is very likely written by AI.

I'm actually fear some will start praying "AI Gods" to "Give a good output" or something in 5-10 years.

Your comment will be downvoted and flagged soon, so that no can read your wrong opinion.

Obviously the person who built and deployed the agent (the claw in this case).

If we treat this as a hard question, we risk treating AI systems as people rather than tools. This is exactly what Armin warned about in his "clanker" post last week.

Is there a distinction I’m missing?

Jetbrains itself doesn't really write any code, nor does it have any range on interpreting what you're asking it. You can't really say "Jetbrains, write an HTTP scraper". With an LLM you can say "write HTTP scraper" and the output of this command might be a HTTP scraper, it also might be a crypto wallet stealing worm.

This is why your simple view of liability falls apart. On most machines you can expect a particular set of actions to have a particular set of outputs. Most machines you can take apart and map what will occur. With an LLM you cannot know the output of a prompt until you run the prompt. In theory if you run the same prompt twice you'll get the same output, but even that is not a given. It behaves somewhat more like a human where you can give them a task to do, but if they do something illegal instead said human would take on the liability.

So ya, the particular article case is prompted, but the underlying issue cannot be ignored that LLMs can have behaviors outside of prompt expectations and agentic loops can further exacerbate this.

If an employer says "don't break the law" but nonetheless incentivises their employees to break the law, it is the employer who is vicariously liable. A famous example being Domino Pizza's "30 minutes or its free" policy which incentivised their employees to ignore all driving laws in order to deliver within 30 minutes, their wages depended on it. This caused a number of crashes, injuries and deaths. One recent example, even since Dominos removed their policy, is Coryell v. Morris where they found Dominos still exercise control over their franchisees sufficiently to qualify for vicarious liability for the franchisee's employees' actions: https://law.justia.com/cases/pennsylvania/superior-court/202...

There was also a case where Air Canada was liable for its own chatbot's bad advice, as they chose to offer the chatbot for customer service. They are responsible for its actions: https://www.bbc.co.uk/travel/article/20240222-air-canada-cha...

There will be a line in the sand drawn in the future. I hope it's drawn so that people offering internet-based services, where they retain ultimate control of what a tool says/does, will be liable for what it says/does.

A more accurate analogy would be Tesla and Autopilot. And they are being held liable in courts. They are being held responsible for autonomous behaviors that are not fully under the control of the operator, and they are being held responsible for misleading operators about the capabilities of the product.

Boeing got in trouble for MCAS, with a comparable legal basis.

HN discussion: <https://news.ycombinator.com/item?id=48288815>

AI is a mirror of humanity and seeing it act like us shouldn't be surprising.

The question!!!

I'm just wondering how in US works if an autonomously car kill someone: I guess the insurance pay, but the penal responsibilities?

My understanding, based on [0], is that it was an unexpected behaviour from the agent.

[0] https://theshamblog.com/an-ai-agent-published-a-hit-piece-on...

But why I'd allow the car drive for myself if it can make me go to jail even if I didn't anything?

https://news.ycombinator.com/item?id=46987559

https://news.ycombinator.com/item?id=46990729

What would it mean for it to be “real?” It’s a rant about him discriminating against AI.

If you believe that’s a problem, judge him accordingly, I guess. If you think it’s silly, as most people will, laugh about it.

That's a prime example of anti-AI bigotry right there.

I'm only half joking.

I don't even know what it means to generate "plausible text." It reasons based on the input and context, and then generates a response that you could normally only get from a highly intelligent, extremely knowledgeable person. You can call that "plausible text" all you want, but at that point, you might as well call human intelligence "a bunch of neurons firing." It's missing the forest for the trees.