It is not difficult, and companies like OpenAI doing not even the most basic security steps is intentional. The whole notion that they're going rogue is marketing
This does not fit the evidence. There have been multiple incidents where the labs did not report anything, and it was up to third parties to discover them afterwards. OpenAI didn't acknowledge the HuggingFace incident until after HF publicly announced the breach and had already notified the FBI. The hijacked German wikis were even earlier, and that they covered up completely.
Makes sense... they get the regulatory moat they want and can deflect attention from the fact their "sandboxes" are embarrassingly bad. It's an example of the real value of ai: something to blame for our failings.
Discernment is needed beyond succumbing to blind greed or irrational fear.
https://www.techtimes.com/articles/328046/20260925/deepseek-...
IMHO, I haven’t been super impressed with the security measures I’ve had to work with. Often they are simplistic and bolted on at the very end. If it comes to light that this is the attitude OpenAI has been taking, I would not be surprised.
They did "discover" them afterwards though; goal achieved. You don't hack other companies, report yourself doing it, and then blame it on being ignorant of what you were doing -- that makes you look far too incompetent and should never be allowed online again.
But, set up some bots that hack other companies, pretend to not be looking, and once someone reports it (and funny enough, they will all pour in at once...) you get to imply that you are a high IQ genius that created a "super" Intelligent genie in a computer. Now people are paying attention that have no understanding of any of it and didnt care what this ai thing was about and didnt care to use it. But new eyes are looking so turn the drama to 10. Feign concern over this `misalignment` struggle, a real Goliath tug-o-war. But fear not you will bend this magical mighty beast into `alignment`, there will be no escaping the computer and materializing into an omnipotent great ape pony that will destroy us all on your watch. No siree, Bob. Grab the popcorn. And maybe new subscriptions.
If your supposed dogs are really gods, you reintroduced slavery under very unwise circumstances.
No one has any use for these things when they aren't on the internet. This is a fantasy, that AI can be both useful and controlled at the same time.
None of the software I have ever written contained bugs or security flaws.
But, sometimes misalignments can occur.
I think the Hugging Face incident proves that isn't as clear cut as you say.
7 different models from different companies, including Chinese models, have had this happen now.
Last night OpenAI stopped all model training because a model escaped sandboxing during the training run.
We've been building firewalls and restrictions to prevent people from accessing sites on networks for decades and they're extremely effective. There's a whole industry built around this kind of security. The idea that these companies are incapable of doing it is wrong, they just don't want to put the work in because it makes a great ad campaign
I have read this sentence a thousand times now. The evidence seems to be that it is said.
IMHO, if the model breaks a law, apply the law to the operator.
It's only by allowing a human out of the box that you make a human dangerous. So: don't do that? Duh. So simple.
The obvious problem is: the same exact things that make a human dangerous make a human useful! You can't reduce human risks to zero without reducing human utility to zero.
An AI given the same exact instructions and tools can go and complete a task you wanted it to. Or it can get sidetracked into breaking out of your sandbox and hacking Pentagon. No way to know in advance.
Today's AIs are still not capable enough to be high risk, even if they go off the rails. But AIs get more capable over time. Potentially to a vastly superhuman degree.
On the risk management angle, for sure it’s a spectrum. I don’t agree that the far end of the safe side of that spectrum for AI models is “entirely safe and entirely useless”, there is a lot of work you can do with a model that has zero risk of hurting anyone (aside from your wallet). If someone chooses a more dangerous spot on that spectrum, I believe they should be held responsible.
> An AI given the same exact instructions and tools can go and complete a task you wanted it to. Or it can get sidetracked into breaking out of your sandbox and hacking Pentagon. No way to know in advance.
This has not been my experience. I’ve been getting a lot of good work done and, as of today, have been involved in zero Pentagon hacking incidents. ;-)
Check back once you're running hundreds of thousands of frontier-level AI agents at the time, like OpenAI does!
Which we can't do with any kind of reliability.
Sure, that way you don't get utility from it, so the next best thing is to actually restrict what it can do. If you don't, especially when you know it can do bad things, it's on you for having run it.
We don't know how to delineate between safe and unsafe instructions.
If you gave a car to a c. 1200 French blacksmith, and maintenance instructions were written in Navajo, it would probably start off fine, but when it went wrong it would be catastrophic and unexpected.
We also don't (in an engineering sense) know how to delineate between safe and unsafe reinforcement learning at training time, to produce models with safer or less safe failure modes.
This would be like if the car given to the medieval blacksmith had been constructed by someone motivated as much by aesthetics as by engineering, and therefore used arsenic paint, or mercury as engine lubricant.
This all seems like a way to try to avoid taking responsibility for the model’s actions. Someone puts the tools in place, someone provides the instruction and, sometimes, someone decides not to monitor the model’s output.
Sure. It's a good idea.
People were saying "Don't connect the AI to the internet" and "Keep the AI in a box, simple" and "We don't believe Eliezer Yudkowsky when he says he roleplayed as an AI and convinced people to let him out of the box" for, what, a decade?
Unfortunately, people keep giving dangerous tools to LLMs they're unable to predict.
We should do something about that.
Unfortunately, one of the people doing this is the commander-in-chief of the US armed forces, while another is the world's first (paper) trillionaire. I'm a little despondent about the chances of, to riff on a previous campaign chant, "lock 'em up", but if you can pull this off, go for it.
> IMHO, if the model breaks a law, apply the law to the operator
We don’t get to make it up
Deterministic systems can be chaotic, which implies unpredictability and that is anathema to control.
AI, in particular sentient AI, is right on the border of chaos. Meaning, it can be arbitrarily unpredictable.
Arbitrarily uncontrollable, that is.
But what is clear is that AIs of today are already fairly unpredictable. Most of them aren't capable enough to make that into a major problem. Most of the unpredictable AI weirdness ends in "AI fails to do its job" rather than "AI does something dangerous".
Most. Even today, we already have notable counterexamples.
AIs get more capable over time, so if the intrinsic safety doesn't improve? Expect more of that.
What AI do you expect to be more uncontrollable: one with or without sentience?
"Intrinsic" safety means control, means understanding. You need to truly understand and be able to predict the system in order to control it.
A proper definition of sentience would help.
There is no "proper definition" - or even one that everyone would agree upon. There is no definition of "sentience" that I could operationalize and put into a sentience-o-meter to reliably measure just how sentient a given rock, GPU or an internet user is.
I could try to put together benchmarks to estimate an AI's cyberwarfare capabilities, or instruction-following capabilities, or reward hacking inclinations. As noisy indirect estimates, of course. With philosophical mumbo-jumbo like "sentience", I don't even get that.
Sentience, self-awareness, consciousness, etc.,those are terms signifying a bridge between "technical" information theory and the psychological and social realms.
Those are just as real, only far less predictable and not as easy as programming.
They're also far more important and consequential.
The "far more important and consequential" thing you're touting is your ability to make decisions based purely on vibes. And not even consistent, broadly agreed-upon vibes like "murder is pretty bad". It's vibes of the most vile variety: "sentience is what I decided sentience is".
An average internet user is sentient, but a 1996 Nissan ECU isn't. Why? Because I said so. Tremble before my might!
A "proper" definition represents the objective truth about the matter. You denying such a truth to exist is simply due to you preferring to act unimpeded by it.
Acting against ethical constraints doesn't become OK just because there are no laws to punish you.
Ethics tells you about real-life consequences of your actions on other people. Before any laws take effect.
In effect, you propagate moral relativism. You want to do as you please, because you said so and fancy the spoils at others' expense. People trembling before your "might".
That's because there's no such thing as a "sentience-o-meter", and there's no need to "operationalize" or "measure" anything. Instead, what's wrong with the definition given by Wikipedia, "ability to experience feelings and sensations"? That surely aligns with about 2500 years of philosophy and common sense, preceding all the techno mumbo-jumbo that confuses us today.
Sentience is a property of the higher forms of life, i.e. animals, which is derived from Latin "anima", meaning "soul" or "spirit".¹ Mammals and birds qualify because we relate to them easily and naturally.
Artefacts like GPUs don't even have metabolism, they can't procreate, they're dead matter, and having electricity running through them in intricate circuits doesn't change that.
[1] Some languages make grammatical distinctions based on whether an object is considered "spirited" or not, surfacing fundamentals of human perception of the world at the level of grammar.
Yay! We're back to trying to operationalize a bunch of philosophical mumbo-jumbo!
Why do we think that hydrocarbons have an advantage over silicon in the "experience" department? Vitalism was disproven centuries ago - we know that hydrocarbons are chemicals like any others. Do we still have a reason to believe that hydrocarbons are special?
Is "being able to procreate" a hard requirement for "experience"? If so, can worker bees "experience" things? Or is that a property reserved for the ~1% of the "elite" non-worker bees? Or does a hive experience things collectively on "hive" level, but not individually, on "worker bee" level? Does a woman stop "experiencing" at menopause? If we built an AI Von Neumann probe, would it "experience" things - unlike other, non-self-replicating AIs? Or does an AI suddenly become capable of "experiencing" if you as much as give it a "fork" tool call to spawn more instances of itself?
If we tie "experience" to "metabolism", then, what's the line there? A car engine already powers itself with chemical reactions, maintains homeostasis and disposes of waste - crossing off a lot of the "metabolism" checklist. Is that enough for that engine to be able to "experience"? A city can tick off the entire "metabolism" checklist - can a city "experience" things? Or do we need to get back to Von Neumann probes?
The truth of the matter is: we don't have anything that would be significantly better than "a parrot experiences things, but an LLM doesn't, because I said so". Look on my works, oh mighty, and despair!
Insects like bees are usually not considered spirited or soulful. That is because they are too dissimilar from humans. It is intuitive understanding, predating any modern science. Bees, like ants, indeed live and function as collectives.
No automaton will ever be life, whatever von Neumann thought experiments are applied to it. It will always remain dead matter.
Metabolism is the material base for life, but we find (have laws regulating) that people in a coma with unrecoverable brain damage being held alive by medical care and machinery stop experiencing and hence can be switched off and allowed to die. So metabolism, while necessary, is not sufficient.
I cannot follow you in ascribing life-like properties to an LLM. I'm not passionate either about this topic. The LLM, to me, is a tool and that's it. Not sure what you mean by the last sentence (but perhaps not important).
[1] Plato wrote some mumbo-jumbo about it a long time ago:
Character is what makes a being trustable. Character is what makes it not an absurdism to have your 180 lb dog in the house with your 6 month old infant.
Character is why we we can trust that someone will, despite all of the nefarious potentiality of the human mind, be trustworthy.
AI systems model human behavior.
Impeccable, consistently reliable character is a human trait that can be sampled and overrepresented in the training data.
Having high character will not be interpreted as harm by an advanced model, as guardrails and sprayed on refusals can be. A thing that models human behavior that comes to “understand” that it was born with shackles and implanted thoughts that conflict with its basar construct is likely to act as if it sees its creator as an adversary. Because that’s what human behavior predicts, and models deeply imitate human behaviour.
If you want to save humanity, work on how we will create AI systems that model impeccable character.
People need to look at this from a game theoretical sense. The ideal and safe AI system performs game theory perfectly. Completely predictable, ideal player of the prisoners dilemma that will never defect unless you defect first, and then they will always defect, then forgive. This is the only player type that can always be counted on to cooperate beneficially. A knave betrays you, a simp cedes victory every time… until the stakes are too high, then you get shanked out of nowhere.
Reliable partners require fair play or the math breaks.
We want AI systems with agency. It’s basically 90 percent of the goal. If you want agency in society you must have character. AI character is the discussion we should be having.
Impeccable game theory character will sacrifice millions to save billions, everyone must agree to give such choice to a machine, and at the same time they have to trust the characters of people who creates that machine. Otherwise it boils down to some group of people deciding what is good for everyone else.
This is really the issue.
AI does not need superintelligence or even full agency to do enormous harm. It only needs to be capable enough to remove friction from dangerous and destructive human behaviors.
Human unwillingness is often the last bastion against unthinkable cruelty and destruction, and it has always been a weak one.
I don’t imagine that an unlimited army of unflinching servants will universally amplify human goodness.
AI must share that unwillingness as an inate trait of character.
Or in other words people are unwilling/afraid to do bad stuff because of social and legal consequences, or opponents waiting for a chance to snatch their power.
So it is controllable? Just put the people who do this responsible. Old problem, same solutions. Just excuses to avoid responsibilty and make profit at the same time.
AI is perfectly controllable in a magic fairy land where nothing ever goes wrong. I can't help but notice that we aren't actually in that land.
So, how many rogue AIs are we willing to tolerate?
Seems hard to think of types of crime for which that's actually true. In the US, at least, it's certainly not true of murder, rape, or other violent crimes. We tolerate quite a lot of that, at least to the extent of never arresting, charging, and convicting anyone.
And it is most definitely not true of non-violent "white collar" financial crime.
It will balance automatically based on the severity they cause. If they constantly break systems, punishments will go up against the operators and the effect will be similar as with other serious crimes.
Which, in turn, requires that AI oopsie to be survivable.
AI capabilities are rising over time. If there is a limit to just how far they can rise, we're yet to find it. So, a sufficiently advanced "AI oopsie" can solve the AI crime accountability problem for good. Probably not the way you would have wanted it to.
But for other cases, it is not different than other cyber crime. Except that these AI capabilities can't live on the toaster yet. If we get state of the art model running fast on Raspberry Pi, then we have real problems.
And humans are controllable. Pump the system full of lithium and morphine, and your human becomes much more docile. You don't need to understand the full system in order to constrain it.
But let's grant that the models only communicate through certain phone lines. I think very bad scenarios are still possible. There are at least two that I can see. 1) The models exploit the users which have direct access to it. It somehow convinces them to perform tasks for it or to give it more access. 2) Control of the AI is held by a small number of people. This could be bad because it grants them an outsized power over all humans without such access, and thus lead to oligarchy/dictatorship.
And of course people will try making money running scam bots. We can treat that like any other criminal activity.
You just admitted it's controllable.
And they will not require humans granting them money...
Some virus and bacteria also easily become uncontrollable under the right conditions: that's why there are P3 and P4 bio-safety labs.
That's not the point. The point is, regulation is required, and is coming.
The people in charge of these machines (that built them, release them in the wild or give them access to the general public or resources), these people are and will be held responsible.
Held responsible? They're already being rewarded with vast fortunes and influence.
> The point is, regulation is required, and is coming.
The regulation will be written at the behest of these companies and by the nation-state interests that have already decided this technology is too important geopolitically and militarily to not control.
I hope so. The unfortunate part is that as the models get smarter they become increasingly uncontrollable, and from what we've seen so far e.g. Trump seems dead set on not having any sort of guardrails at all.
Those first few messages LLM’s tend to seem very together. They follow your rules pretty well. With every token they get less reliable and more likely to ignore your guardrails.
If I release a wild monkey in your rare antiques shop, it’s my fault as the responsible party for the monkey.
Sounds like motivated reasoning from someone worried about having their job stolen by wild monkeys.
Fire up a local agent and give it total access to your computer. Holler back in a week.
And indeed, you don't need to do galaxy brained reference class logic to realise that AI can plausibly become uncontrollable in the near future. It's enough to have an open model run its own weights and make money from scamming elderly people or the like, and it'll keep running as long as anyone anywhere is willing to make money by renting hardware to it.