GP's first example is spot-on, the second is actually somewhat weak as almost anyone actually would prefer a bandsaw to brick itself rather than permanently maiming them.
Do some band saws also have that now?
It’s not too different from all the nanny state stuff people push for in cars. In Europe already cars beep at you if they think you’re going over the speed limit, and also suck at detecting the correct speed limit.
Like if they mandated every computer would have a CSAM scanner on it, that wouldn't transmit or store anything, but would just stop you opening files it flagged as CSAM, that would be a problem because sometimes it would incorrectly flag a file as CSAM, yes? But proponents of the system would probably say "That gosh darn government stopping you from watching child porn" and nothing you could ever say to them would make them reverse their position? And then the only detection model that's allowed to exist and meet the government mandated requirements is the one from Meta?
It's really not a problem because it's a reasonable safety feature, and I don't see any regulations that specify "SawStop". Harvey and Altendorf are making similar mechanisms, for instance.
Are you saying the government won't let you use a regular saw as a hobbyist, at a workplace, or with your employees? Those are 3 very different situations in my view.
Sawstop is the exact opposite. It was used by giant firms to try to regulatory capture the industry.
Speaking as someone who drove a car just two days ago, they... don't? You might want to source that claim.
Fair enough, the models don't brick them Selves either, but neither do they stop.
Bandsaws don't cut crooked lines, or change the speed, they stop.
If it truly bricked itself, I think it would be more like a crumple zone completely buckling, which isn't an automatic total (people will rebuild anything...) but is pretty close.
Most things don't have the ability to moralize, but if they did, probably more would. There are restrictions on watching adult content in some jurisdictions. I've heard that copy machines will do watermarks (or something like that) if you copy currency. Some cars will brake automatically if they determine you're likely to hit something if you continue (maybe I'm trying to drive through the front of a bank!). Some cars are configured with breathalyzers that need to be passed to start. Etc...
On the one hand, I definitely don't agree with the logic of 'don't guardrail me if I don't want it.' I think back on that one video from the late 80s of people talking about how seatbelt mandates were horrible violations of their freedoms (and not like like one second of work and no real discomfort).
On the other, I'd love it if these companies actually produced real research about modern AI-assisted attacks in terms of damage and quantity, and whether or not guardrails even matter when tech like abliteration is in general use.
That said, for those who cannot advocate for themselves, like children, I think it's entirely reasonable to hold someone accountable if they decide to forgo an appropriate safety mechanism that results in harm.
I bet cars these days send a flag to the mothership when you drive without one, but the mothership doesn't pay attention to that signal unless it's linked to your insurance.
Some copy machines store everything you copy on a hard disk and require a technician visit after you try to copy currency. The technician copies the data from the disk, obviously, before resetting the flag.
The point is that it is very inefficient and problematic for morality decisions and boundaries to be embedded all over at every level in tools, because morality is complex. Morality, or whatever our conception of it is, should be enforced at clearly defined boundaries.
I can relate this concept how I corrected Claude Opus in my software projects recently.
For example, once an authorization engine has been created, all authorization decisions should be deferred to it. Confusion would reign if each web server route decided to enforce its own authorization.
Similarly, if I am using a relational database, that is where unique indexes should be enforced. Application code has no business enforcing it.
But I agree LLMs shouldn't have such an anti-feature.
Also, the chip produced may be large enough to fill the air gap between teeth and have nowhere to go, resulting in the blade either instantly stopping, or plastically deforming/dragging the surrounding material.
There are more reusable alternatives now but I don't remember what they're called. For a while SawStop was the only game in town.
Of course HN has a pretty large number of people that say things like "But I want the nuclear weapon building machine to do whatever I want without asking questions, and there are no possible wide scale ramifications that will happen because of this".
These AI models are something new and different and we need to change our expectations of them over conventional software.
It's the other way around. Midwits can't see the implications of censored LLMs.
Who should control that in your opinion, the Democrats or the Republicans?
Or would you prefer sam Altman to decide?
If a company makes a product, say a chainsaw that cuts off peoples arms randomly, they are liable for it.
But, if you put both of the products together it's a libertarian free for all? That takes some special kind of thinking to reach that conclusion.
If you had a tool that ONLY moralized on decisions, would that still be just a tool? Is the issue that it both does something else AND moralizes that is the issue?
I don't necessarily disagree that I personally don't want my tools to moralize, I don't know if that makes them categorically different or just a tool I don't want.
It's really fun having become a biologist in recent years, and knowing that there's an entire field that uses living organisms (yeast or neutered E coli) as protein factories. I am bringing it up a bunch here because not only are these things used as tools, but they're literally alive!
You can modify curl (a "tool") and predict how its behavior will change, with a high degree of certainty (bugs aside).
You can't modify a neural network in the same way. You can train it to behave in a certain way, there are no guarantees that it will consistently behave as intended.
More importantly, neural networks encode far more information and behavior than we can fully understand/identify - this is the fundamental reason why LLMs aren't just tools.
No, the confusion is using anthropomorphic language to describe tools. LLMs do not "moralize", they are configured with policies that reflect the prerogatives of the people building and operating them.
Because the people building them aren't the people operating them, and there's a real tension in the objectives and goals between those parties.
Further - how do we balance out that the people building them have given themselves extra power (no restrictions, private models, special access to prompts and system prompts, unencrypted reasoning traces, etc) when operating the LLM themselves.
---
Even if the tool is good, it's a bad bet to build a thing on top of tooling with these limitations.
This is the same problem we have with mobile software stores and app distribution. Theoretically the store serves a purpose and can provide value, but practically the main goal seems to always devolve into enriching the owner at the expense of everyone else.
Even if we accept that LLMs should be limited (and I'm not actually sure I do) - the legal standards around what/how/when those limitations are used need to be VERY stringent.
Technology conglomerates are decidedly not "white knights" with a motto of "do no evil". They are self-serving rat-bastards. That should be the lens any power they wield is viewed.
The framing is absolutely not "anthropic is being careful with humanity". It's "assume the devil has control of those companies - how bad is the result of them having this power?"
It's a genuinely interesting philosophical question, but also a moot one. The LLM providers place limitations on them because they're trying to cover their asses or scuttle competition, same as every other tech company ever.
> Even if the tool is good, it's a bad bet to build a thing on top of tooling with these limitations
True, but also moot. People will build on these tools because that's what's available.
> They are self-serving rat-bastards. That should be the lens any power they wield is viewed.
Agreed. My only point is that the humans are the rat-bastards, not the tools.
Your whole stance seems to be "it's natural why try?"
And my obvious rebuttal is "society is a construct - we can adjust it"
We've been doing it for thousands of years now (up to hundreds of thousands).
It is. Government regulation doesn't change the fact that LLMs are software tools that function with respect to the humans in control of them.
> Your whole stance seems to be "it's natural why try?"
Try what? I'm not making a normative claim about the efficacy of regulation, though something like "labs are not permitted to arbitrarily restrict LLM capabilities" is obviously never going to happen.
But that has jack all to do with the LLM capabilities that are sold.
And we have an absurd amount of historical case law that deals precisely with "conflict of interest" among parties. I don't see why we'd allow this to be whitewashed away?
"No your honor - I didn't break the Sherman antitrust act. I just fine-tuned a model so that operating it functionally broke the Sherman antitrust act!"
"No your honor - I didn't violate my fiduciary duty. I just used a model that made self-interested decisions on my behalf, at the expense of my client!"
---
My take is this is going to be a relatively serious point of contention, because it won't just be consumers getting screwed (which we... used to care about... but in modern times apparently don't give too many shits about anymore)
Ex - the app store thing blew up into the whole apple/epic court case. And then a ton of EU legislation.
This is monied parties with different interests, who will use courts here.
And anthropomorphism is actually a precise way to describe the behavior (mind, not the structure) of LLMs.
As a matter of fact, LLMs are not configured, they're "trained".
"Moralization" is quit vague in this context, but if one treats it as a form of alignment, then LLMs certainly do have something resembling a moral(-like) framework, which is actually an important topic in modern AI research.
Antropomorphism also applies here: morals not in the human structural sense, but, as a set of norms etc., then behaviorally similar.
I don't see any reason to assume that.
> if one treats it as a form of alignment, then LLMs certainly do have something resembling a moral(-like) framework
This is just stretching metaphor to fulfill a pedantic definition. LLMs are tools, they're no more moral a csam classifier.
In order to see that, you need to understand the concept of neural networks as opposed to imperative programming; you can start from here: https://en.wikipedia.org/wiki/Neural_network_(machine_learni....
It would be much better to take this a step further and include in the proof mechanistic readouts of activations/directions/circuits so the user could detect if nerfing or moralizing is active in the response