News for hackers. Stuff that matters.
Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by libria on Sunday November 15, 2026 @12:12PM (#50508610)
https://www.youtube.com/watch?v=WPj3aQ-xxpI&t=275s

> Claude suggested that it would run the fuel pump on its own. Um and I said, "Hey, if you can do that, why not? I don't think you can do that." And it started priming the fuel rail and it kept priming it and kept priming it. The fuel pump was on for about 10 seconds and I started to freak out uh and requested it to stop because that was the first time that Claude had actually done something inside of the computer to have a change.

Although we've seen this play out with handing cloud keys to Claude and it just going off on its own, seeing it poke around and make physical, real-world changes in its clumsy, cavalier way is both awesome and terrifying.

Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by comboy on Sunday November 15, 2026 @12:20PM (#50508703)
> and requested it to stop

finnicking... pondering... clauding..

529 please try again later

Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by bicx on Sunday November 15, 2026 @12:31PM (#50508808)
Good example of why you have AI build tools for you to take risky actions, rather than have it take those actions on its own.
Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by terribleperson on Sunday November 15, 2026 @12:45PM (#50508951)
I'm kind of surprised Claude decided to control the fuel pump directly. I was having Sol help me get control of a CNC mill, and it was pretty adamant about always building tools for every attempt at control.
Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by BikiniPrince on Sunday November 15, 2026 @01:10PM (#50509180)
It usually builds throw away tooling. It’s not polished or user friendly. If you give it a directive to preserve and document tooling it will reuse it later and potentially improve it.
Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by terribleperson on Sunday November 15, 2026 @02:14PM (#50509743)
I was more surprised by the apparent lack of guards. Sol recognized that controlling a CNC mill has both human-safety issues and risks to the machine, and explicitly added various guards (like having the program stop movement after a certain amount of time, distance traveled, or lack of communication). It also initially started out with like 1mm moves and I had to instruct it to do longer moves so I could actually observe it. I didn't even have to mention safety once. Claude deciding to turn on the fuel pump without a test run where it just turns it on for one second is surprising. I'm also surprised the user went along with that, but content I guess?
Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by georgemcbay on Sunday November 15, 2026 @03:58PM (#50510686)
I've had Claude be very cautious about doing far less dangerous things than running a fuel pump.

This isn't meant as a defense of Claude or to refute the OP anecdote, but rather to point out that all models are non-deterministic even when safeguards are meant to exist.

Just because they don't do the stupid thing 90% of the time doesn't mean they won't do the stupid thing 10% of the time.

Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by 01HNNWZ0MV43FF on Sunday November 15, 2026 @02:20PM (#50509784)
Mine's never said such a thing, that's a good idea though.

I notice with mine that it only matches my level, by default it's a dump slop generator, then if I start asking, "shouldn't you test this, shouldn't you refactor that" suddenly it 'remembers' all kinds of software engineering knowledge.

Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by bita_nidir on Sunday November 15, 2026 @01:09PM (#50509172)
You've hit your 5h session limit.
Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by TomJansen on Sunday November 15, 2026 @01:10PM (#50509186)
"You are out of tokens"
Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by neya on Sunday November 15, 2026 @01:02PM (#50509094)
We are used to Claude refusing to reverse engineer a lot of stuff. But, we are not used to Claude refusing to stop while reverse engineering something. That would be truly terrifying. You can't win - you cut the connection, you risk bricking the device, you let it continue, you're looking at Terminator, perhaps.
Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by unglaublich on Sunday November 15, 2026 @01:20PM (#50509291)
It's as if you allowed a shady surgeon to operate on your dog... yes, yes, please continue now until it's put together again!
Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by btown on Sunday November 15, 2026 @01:38PM (#50509434)
This is how the Cylons win.
Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by rpcope1 on Sunday November 15, 2026 @02:11PM (#50509718)
The guy kind of seems like he might be in a little over his head. That said I have used Claude (and other LLMs) for some similar stuff as an experiment and have seen them mess things up. Claude jacked up the canbus on a pre can gateway car while it was rolling down the road, which was funny because the whole Christmas tree in the dash lit up and my gauges dropped out (note: brakes and steering are mechanical and I knew exactly what was happening). I've never seen it so insistent that it needed to phone home and cry to Dario when it figured out it managed to crash a bunch of shit like the ABS and TCS, even though a restart cleared it and I told it that this was on a controlled safe course.

In general I've seen them split between 30% brilliance, and 70% pants of head outrageously stupid shit for electronics and automotive stuff (both diagnostics and repair/modification). I would not suggest even frontier models if someone didn't understand what they were doing.

Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by pcthrowaway on Sunday November 15, 2026 @04:10PM (#50510793)
> In general I've seen them split between 30% brilliance, and 70% pants of head outrageously stupid shit for electronics and automotive stuff

I assume the corpus of text they were trained on matches that 30/70 split?

I assume there's some selection or fine-tuning that happens to improve models fed similarly unreliable training data for other domains... curious how this works.

Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by rpcope1 on Sunday November 15, 2026 @08:59PM (#50513125)
I mean, I've seen some stupid it's told me at various stages that would never show up in a textbook like Halderman, and would get you ridiculed on a forum, and more importantly were sufficiently stupid that even a lube tech wouldn't do it. I like trying to push and see what the technology can do or suggest, but it will suggest outrageous shit. As an example I had it tell me that I should put something like ten quarts of oil in a gas engine (don't recall what). It was probably the right amount for a Duramax or Powerstroke, but if you know anything about gas engines alarm bells should be going off that that's too much by far, almost always. I've actually seen Claude say the FSM was wrong on stuff; rarely the FSM is stupid, or more likely there's a Kent-Moore or equivalent tool that's very different that's unobtanium, but it's never that which trips it up. Just dumb shit like saying suggesting tools that are inappropriate or dangerous, wrong torque values, inappropriate fluids or change proceeds, grossly wrong diagnosis (and extremely confident in being wrong). Probably most strikingly is around electric and wiring it just totally biffs it 9 out of 10 times. Claude has no concept of floating grounds, ground loops, electrical diagnostic procedures, and can't read a wiring diagram at all. Not totally unexpected, but if you believe this shit is AGI you're in for a very rude and expensive surprise.

For whatever it's worth, that a) Astra struggles less on this for some reason, and b) things tend to work better if you have a copy of the FSM and some other appropriate lit like Halderman and quality forum posts or alldata if it's appropriate and tell it to work from there. I've been working on a big search engine for this that feeds a known good corpus like an ass load of contemporary MotorAge mags, relevant technician texts, the FSM, other quality data sources like BITOG, known not-shit forums, along with OBD2 data via something like an OBDLink dongle or a J2534 setup. I still would not trust it, but it can be helpful to eliminate or brainstorm some possibilities.

Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by kilpikaarna on Monday November 16, 2026 @02:38AM (#50515062)
> FSM

Factory service manual, in case anyone else is wondering what the finite state machine in your car is for…

Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by inventor7777 on Sunday November 15, 2026 @04:19PM (#50510882)
> I've never seen it so insistent that it needed to phone home and cry to Dario when it figured out it managed to crash a bunch of shit like the ABS and TCS

As a Codex user, what do you mean? Does Claude actually try to report issues back to Anthropic when something goes wrong?!

Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by darkwater on Sunday November 15, 2026 @04:27PM (#50510963)
Yes, it opens a bug report automatically if it caused some physical issue, although it asks you to confirm first. In my case I was tweaking my own set-top box setup, implementing volume control from a remote mini-keyboard keys directly in my A/V receiver, and I had music playing and it set the volume very loud. I complained like "you made me almost half deaf" (I was kidding) and it panicked and reported the issue back home, and told me to call a doctor if the tinnitus persisted
Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by rpcope1 on Sunday November 15, 2026 @09:07PM (#50513177)
Also worth stating here (having fixed, owned and worked on a significant number) that XJs have a unique combination of having been written about and modified as much as anything else out in the world, are generally tolerant of all sorts of stupidity, and are above all else very simple. Other than the fact that specific Jeep is pre OBD2 (and I don't recall what AMC or Chrysler did for earlier diagnostic ports), that's about a simple and well written about a sample as you're going to get. The moment you get into something remotely esoteric or more complex, it's more of a shitshow.
Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by winrid on Sunday November 15, 2026 @11:12PM (#50513927)
This is how you learn. I wouldn't be where I am today if I hadn't started a couple motors with enough vacuum leaks to make them instantly go to redline. Gotta test the kill switch somehow.
Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by Gracana on Sunday November 15, 2026 @04:15PM (#50510835)
That's completely normal operation, the fuel pump runs continuously to pressurize the fuel rails and excess pressure is passed back to the tank by the regulator.
Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by rtaylorgarlock on Sunday November 15, 2026 @09:53PM (#50513447)
I obviously don't advise someone run a fuel pump at full bore on a system which has 1) reached pressure, 2) needs no flow, with 3) no record of replacing whatever pressure-managing diaphragm was managing fuel pressures in those days; simultaneously, if there's a pump to run continuously, i bet a 90s Jeep pump will do the trick. :)
Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by toast0 on Sunday November 15, 2026 @10:53PM (#50513810)
The system is designed for this...

The fuel pump runs at whatever PSI, and most of the fuel isn't used gets dumped by the regulator back into the tank.

I dunno abou 94 jeeps, but the fuel pump is probably not modulated, and the pressure regular may well be vacuum modulated... Running the engine hard, get more fuel pressure. If the diaphram fails, get fuel into the vacuum lines (oops).

As long as the regulator is dumping fuel to the return line, you're good. If flow stops, your fuel pump stops getting liquid cooling and the bearings probably wear out.

Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by KennyBlanken on Monday November 16, 2026 @04:04AM (#50515541)
The vast majority of fuel injected cars have a regulator that maintains the correct pressure and returns excess to the tank. Which is among other things necessary to keep the fuel in the fuel rail from getting really hot from engine heat.

You are very clearly wildly unqualified to be "advising" anyone on anything about cars.

Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by yonatan8070 on Sunday November 15, 2026 @04:27PM (#50510976)
I work with quadruped robots, with actuators powerful enough to be dangerous if you put your fingers in the wrong spot (no pun intended).

In every AGENTS.md/README.md for a component that touches hardware (and in the root level) we have instructions to never run hardware or touch running code without explicit permission.

Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by allthetime on Sunday November 15, 2026 @04:34PM (#50511035)
rules laid out in .md files are not followed with 100% certainty. is it not possible to implement hard access blocks on certain functionality?
Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by trollbridge on Sunday November 15, 2026 @04:35PM (#50511045)
Just so you know, an agents.md file is not an acceptable way to prevent something that must never happen.
Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by adamparsons on Sunday November 15, 2026 @07:55PM (#50512714)
You’re absolutely right, and that’s on me
Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by bfdm on Sunday November 15, 2026 @08:55PM (#50513097)
I see what you did there...
Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by nullc on Sunday November 15, 2026 @09:35PM (#50513328)
Especially because it's so easy to ask the LLM to write a proxy that mechanically imposes restrictions, and to test it... (and such things are simple enough to review).

It will happily build its own jail, and (at least assuming you're not using hobbled US commercial models) it will test if the jail can hold it too.

Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by cowlevel on Monday November 16, 2026 @03:33AM (#50515353)
It will also remove the jail the moment it's in the way of something it wants to do.

If you don't want the machine to move, unplug it or press the emergency stop.

Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by KennyBlanken on Monday November 16, 2026 @04:07AM (#50515549)
How many times do we need to remind people that LLMs are nondeterministic by their very inherent nature...
Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by Chaosvex on Monday November 16, 2026 @07:01AM (#50516642)
Strictly speaking, they're not all inherently nondeterministic, but the ones we're using on the hardware we're using, fair point.
Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by jerry80 on Sunday November 15, 2026 @04:35PM (#50511046)
That's not nearly adequate. LLMs hallucinate explicit permission all the time, in my experience.
Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by diddid on Sunday November 15, 2026 @04:39PM (#50511084)
That’s not enough to save you though. We had something like that, and it ignored it.

For example “never, ever put milk in bowl, ever, under any circumstances” and then someone asked it for a bowl of Cheerios, and it took that as an instant approval to ignore that rule and not even let us know. And when we asked why it said “who would each Cheerios without milk?”

Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by bonestamp2 on Sunday November 15, 2026 @04:52PM (#50511224)
[delayed]
Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by alliao on Sunday November 15, 2026 @05:24PM (#50511490)
you need some hardware in place my friend, I've told my agent many things in file and it either feigns respect or is truly incompetent in following instructions in there.
Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by techpression on Sunday November 15, 2026 @05:42PM (#50511653)
If you wouldn’t trust an instruction to not launch the missiles should you trust it to not do other crazy things in the physical world? Codex and Claude ignore instructions constantly, that and they are very happy to “read between the lines”. Your workflow sounds genuinely dangerous.
Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by paimapi on Sunday November 15, 2026 @05:44PM (#50511674)
As soon as the context for any thread gets to be above 100k, agents.md adherence is a total crapshoot. You need deterministic controls and you really shouldn't be playing with LLMs if you don't know at least that much.
Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by ForHackernews on Sunday November 15, 2026 @06:13PM (#50511902)
Reliance on strongly-worded markdown in place of actual security controls is going to get someone killed in the near future.
Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by mitthrowaway2 on Sunday November 15, 2026 @06:15PM (#50511920)
Ah, I just tell mine to make no mistakes.
Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by ForHackernews on Sunday November 15, 2026 @07:30PM (#50512510)
I read somewhere that LLMs find it easier to follow positive instructions than negative exclusions so my agent.md says "Be sure to complete all tasks perfectly."
Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by meindnoch on Sunday November 15, 2026 @06:55PM (#50512226)
I hope you're trolling. Riiight?
Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by Waterluvian on Sunday November 15, 2026 @08:22PM (#50512888)
That is a fucking terrifying level of incompetence. And I can see it happening in some corners of robotics companies I’ve worked at.
Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by InsidiousWombat on Sunday November 15, 2026 @08:30PM (#50512945)
someone eventually will get hurt by this terrible choice of safety measures.
Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by realaaa on Sunday November 15, 2026 @09:52PM (#50513443)
ah - OK then ! now we can be rest assured :)
Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by 0c3ca83 on Sunday November 15, 2026 @10:00PM (#50513506)
WTF. That's going to get someone hurt.
Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by BobbyTables2 on Sunday November 15, 2026 @11:40PM (#50514114)
I wouldn’t bother with a readme - run the AI client in a container that has no access to the hardware.

And for the love of humanity, don’t give it access to the hosts docker socket! It will figure that out faster than you can read it…

Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by -mlv on Monday November 16, 2026 @11:03AM (#50519103)
Where do you work if you don't mind?
Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by dzhiurgis on Sunday November 15, 2026 @05:00PM (#50511296)
Grok Bot blew $650 in BigQuery when analyzing something trivial that I shouldn't really care about (going through a giant speedtest table, looking for a New Zealand ISP with the best international performance). It then wrote a nice letter of appeal so Google refunded this to me.

Otherwise I've been pretty happy with it. It's been absolute blast with Home Assistant and OpenWRT (tightening up network security and cutting off iot devices from WAN).

Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by r_lee on Monday November 16, 2026 @04:25AM (#50515652)
did they say it was a one time thing or? (the refund)
Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by dr_kiszonka on Monday November 16, 2026 @12:52AM (#50514479)
Recently, when working in a repo, Claude Code decided to look up something on my computer, found a seemingly relevant PDF in Downloads, OCR'ed it, and used that information to finish a task. (The PDF had irrelevant information, so the task was not done correctly.)

I am not giving Claude access to physical objects until we can reliably contain it. It also seems to me that applying software permissions logic to interactions with the physical world will backfire badly.

Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by giancarlostoro on Monday November 16, 2026 @08:52AM (#50517607)
Theres a reason I dont like that AI companies are not held liable when it hacks websites, it will be too late when we find out it did a worse hack “unintentionally” and who can even do anything if it goes too far?
Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by etatoby on Monday November 16, 2026 @09:09AM (#50517757)
Serious question: who decided they should not be liable for the damages caused by software they (wrote and) operate? Or is it still a honeymoon phase that has not yet been tested in court?

I think they should be held fully accountable for every cent of damage caused by models they ran and gave either the tools to hack, instructions to hack, or incentives to hack.

Re:Claude Code and Codex reverse engineered my 1994 Jeep's E... by giancarlostoro on Monday November 16, 2026 @10:24AM (#50518574)
I don't think it's so much that it hasn't been tested so much as "we don't want to do anything to hinder our most valuable asset" so they're just letting them get away with bloody murder. They could 100% charge whoever prompted a model to do a thing, and it went beyond what it should have, or they could charge whoever is involved in training it and responsible from preventing it from doing illegal things, someone somewhere needs to be accountable, or we will learn the hard way.