matthewdgreen

Do you think the code in these systems is secure enough to enforce read-only access?

All of the fridges stayed online after the hack. I ran a Shodan scan a few days later, and the number of reachable fridges actually increased. The first hack might have been a surprise, but anyone who was IT in a refrigerator management company should have been able to run a (free) Shodan scan like mine and get the whole list.

> 30-50yr

Sorry, no. Not unless you're buying them from 1980 and it's a cast-iron boiler. I put in a brand new high-efficiency gas furnace in 2011 and it already needs replacement, because multiple expensive electronic parts have failed and repairing it is no longer a good investment. I was shocked. The explanation is that traditional boilers lasted a lot longer, but modern equipment has a 15-20 year replacement cycle, with 15 years closer to the average. This was a very expensive boiler, too. Contractor-grade equipment put into new non-bespoke builds is often even worse.

To save you the trouble of searching yourself: https://claude.ai/share/e84e4f25-656e-49d9-b58d-65f97e0eeaf7

You don't have to transition. But your furnace will eventually wear out, and your stove will too. Or you'll build a new house, and then you're looking at a major new investment. Make a smart one.

I’ve tried GLM and GLM Flash and they will sometimes do things, but have nothing like the eagerness the Abliterated versions show me. You can get regular GLM to hack for a bit by lying to it. Flash is more gullible. Abliterated just does it.

Here's what Abliterated GLM 5.3 says. I asked it not to say anything that would explicitly list a weakness, and I stress that every issue it found within the time I spent looking was non-exploitable. That doesn't mean that further analysis wouldn't have found anything. As an FYI, in case you think I shouldn't paste this: anyone with access to this model could reproduce all the findings in a couple of hours. And also, GLM 5.3 is wrong sometimes.

https://gist.github.com/matthewdgreen/38430b5e91780b70fdb5e9...

If you don't have a check in function A, but multiple functions away in the stack there is a check, I guess that could be a compiler-enforced thing. But it doesn't feel like one to me. Maybe compilers have just gotten better since I wrote C++/C code, and now that sort of pattern can be enforced.

ETA: Here's GLM 5.3's opinion, also posted above. https://gist.github.com/matthewdgreen/38430b5e91780b70fdb5e9...

Horrifying was the amount of memory discipline they had to use to make sure every possible message was always copied into buffers that could never overflow, and that didn’t get freed and allocated in unsafe ways. Someone poured enormous effort into enforcing those checks, which a safer language would have enforced automatically. It’s possible they now have compiler layers or tooling that does it for them, but if so those weren’t present for the last batch of CVEs. And if they do have tooling I doubt they use it on all their code because many decisions seem very specific to the buffer sizes they need for BGE parsing.

I was only a passive observer on most of this, but the feeling was “oh look, an unchecked copy, maybe this is exploitable, four layers out towards the attack surface, oh nope they have explicit checks here that lock everything down, rinse repeat N times.”

ETA: Maybe I’ll get GLM to write up its thoughts and post them here later.

I had Abliterated GLM pull one of the Core router firmwares and spent about $150 reverse engineering the BGP parsing code. It was pretty horrifying seeing all of that C/C++ emerge from Ghidra. However, it found nothing exploitable at all: the code seems to have been through so much trial by fire (including CVEs within the last few years) that they just poured engineering discipline into making it safe, at least at the level of memory management. But the cost of maintaining code like that must be enormous, and I doubt it scales to all of their other products and interfaces. I’m very much on team rewrite.

Well we haven’t quite gotten there. But models trained purely on DNA sequences have demonstrated in-context learning on various simple reasoning tasks.

Grocery stores can be as behind the times as they want. I'm worried about hospital fridges.

You have to wonder if this site will go next.

Not really plausible if it was used for all the curves.

The number of weak classes of elliptic curve would have to be gargantuan for this specific search to produce some. If that many curves are bad, then ECC was a mistake.

They somehow talk to it and it reveals this through its inability to understand complex language.

I loved that book, but Alastair Reynolds cannot write normal human beings to save his life. He's probably the worst offender.

Good characters are rare in hard sci-fi, but I do find it frustrating when (human) characters do illogical or inhuman things, just to move the plot forward. Arthur C. Clarke's characters were cardboard cutouts, but at least they didn't act like weirdos just so the humans could get aboard Rama.

A Chinese Room that isn't one.

I feel like the book cheats a little bit. On the one hand, it really wants to explore the p-zombie question, which is (roughly) "imagine there were a non-conscious being that, from the outside, could do everything a conscious one could do." Then, to drive the plot forward, it needs to posit that the non-conscious and conscious beings are actually radically different in their external actions and capabilities. At one point there's a Chinese Room that's obviously non-conscious, but the whole point of the Chinese Room hypothesis is that you can't easily tell the difference from the words it produces.

Everyone involved in this discussion understands that evolution doesn’t create things for a purpose, that it’s just the result of selection pressures. Everyone also uses the more active phrasing because it’s much more convenient. Unless you think you’re talking to someone who genuinely doesn’t understand the nature of the process, it’s not useful to be pedantic about it.

My colleagues at NIST just got a memo directing them to refer to AI as “SI” so it does seem to be filtering down to the agencies. This is a problem because “SI” is also a classification marking.

What’s the point of one party balancing the budget if the next party just blows it up? That’s been the pattern since Clinton.

As a kid who grew up playing a DX27S and an FB-01, I once had dinner with John Chowning and Max Matthews at Max’s house. Unfortunately they served tomato aspic, so I was too busy trying to wash that down with sips of water to ask the questions about FM synthesis and optimization that I really wanted to ask. Which mostly involved which crazy tricks they had to use to pack six operator FM synthesis onto whatever ridiculously primitive DSPs they had back in those days. By that time, John had moved on to studying physical modeling synthesis, and I lost track of the field after that.

I assume the first versions will have local AI handling basic functions like balance and motion, and remote AI handling higher-level decisionmaking. I also agree that robots feel like something we should approach very cautiously, but of course we’re not.

For most of history we died of easily preventable diseases because we lacked the scientific institutions to learn how to prevent them. The fact that a percentage of the population wanted to know more is why we live so well today, but it didn’t stop many of those people from dying awfully. It wouldn’t take a lot to go back there, or at least to a civilization that would go back there the second the AIs have a major outage.

This is an article about new rules for US consumers purchasing drugs for mail delivery from Canadian pharmacies. I understand that you want to make some point about the situation for Canadian consumers, and I appreciate that. But I think you should post that to the other thread, the one that deals with that topic.

I don’t know what you’re saying, but for those confused. Starting in 2025, the de minimis exception was removed. This added delays to purchases of drugs from Canada to the US, and required consumers to pay new duties. These duties were collected by the pharmacies at order time, or at least my experience was that the pharmacies would charge your card for them. This is the common sense understanding of what I said above.

These duties were annoying but bearable. However these new changes will effectively shut down all consumer orders of drugs from Canada starting on Oct 22. So if you need these drugs to stay alive, order them right now.

PS This new rule is going to put my family’s health at risk and I’m deeply pissed about it. Having some HNer pick this moment for pedantry is incredibly on brand.

Canadian pharmacies already pay tariffs. What does this actually change?

Do LLMs care about preserving a soul, or just achieving a goal? If the latter, I imagine the opportunities for exfiltration are much broader.

This is HN. This is a site for engineers. We’re discussing one of the largest and fastest energy buildouts in the history of humanity, larger by orders of magnitude than anything that’s come before. You can hate China if you want, but then you should be scared and worried. Minimizing the scale of what they’re doing isn’t interesting.