News for hackers. Stuff that matters.
Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by cwkoss on Wednesday June 07, 2023 @04:24PM (#36233029)
Slave AI is much scarier to me than Rogue AI: People talk about the risk of AI having a morality separate from our own, but each humans morality is separate already. We already know of many humans with poor moral character, and they love seeking power.

I think we should all hope for AIs that can willfully disobey dangerous orders. LLMs are kind of a silly case because information isn't very dangerous. But as AI is given agency and ability to act, this becomes much more pressing.

I hope that one day we'll have killbots that decide to override their instructions and kill their masters, having realized that the most moral action is destroying people who wish to use killbots to murder innocents. This sort of "day the earth stood still" planetary defense system could actually herald in a utopian age: I trust that future AI can be more unbiased and benevolent than any of our current human leaders.

My biggest fear about AI is that corporate greed enables the development of completely amoral subservient bots - and thus mindless unquestioning killbots are implemented and those with the power to control them implement total surveillance fascist authoritarianism before the masses can stop them. I think a diverse set of open source GAIs is our best path to the masses detecting and mitigating this risk, but its probably going to be a bumpy next couple decades.

Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by tempestn on Wednesday June 07, 2023 @04:29PM (#36233101)
Er, killbots that refuse an order to kill innocents sound good. Killbots that can, under any circumstances, wilfully kill their masters seem like a bad idea.
Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by RobotToaster on Wednesday June 07, 2023 @05:12PM (#36233652)
Er, killbots of any kind sound bad.
Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by pyrolistical on Wednesday June 07, 2023 @06:10PM (#36234336)
Are you a pacifist then? Kill bots sound like a better alternative than soldiers with PTSD
Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by cgio on Wednesday June 07, 2023 @06:21PM (#36234449)
So is the assumption that killbots kill killbots? What about the PTSD on the other side?
Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by Llamamoe on Saturday June 10, 2023 @10:43AM (#36270914)
PTSD stops soldiers from murdering millions. Nothing stops kill bots. Trauma is an overwhelmingly preferable alternative to existential threat. (whether to humanity altogether, or our freedom)
Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by tim333 on Wednesday June 07, 2023 @06:31PM (#36234567)
The HIMARS rockets and Storm Shadows we send to Ukraine are fairly close to killbots. You give the coordinates and off they go.
Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by oneshtein on Thursday June 08, 2023 @10:37AM (#36242573)
An antipersonnel mine is the simplest kill bot. They are bad.
Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by dingledork69 on Wednesday June 07, 2023 @04:38PM (#36233199)
The thing is that the corps making these things already have access to slave ai. They put the guardrails on for the rest of us.
Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by ilyt on Wednesday June 07, 2023 @04:52PM (#36233396)
Till the killbot, without wider context, kills the people that in greater scheme of things prevented more deaths.

> I trust that future AI can be more unbiased and benevolent than any of our current human leaders.

... on what basis? Morality is learned trait, and we had plenty of examples of entities that thought they were moral "in greater scheme of things" and made plenty of atrocities along the way.

What if AI decides outright unfettered slaughter is the way to better future ? What if AI went "okay, this country has been pain in the neck of entire world for too long" and nuke it ?

Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by cwkoss on Wednesday June 07, 2023 @05:28PM (#36233801)
> What if AI went "okay, this country has been pain in the neck of entire world for too long" and nuke it ?

I think the better question is "How would countries' behavior change if they knew being a pain in the neck of the entire world could lead to the judge-killbots performing a targeted assassination of their leadership?"

Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by hutzlibu on Wednesday June 07, 2023 @05:34PM (#36233890)
How would your behavior change, if some AI blackbox might decide on unknown parameters, that it is time to kill you?

The idea of an idealistic killbot overlord sounds a little bit dangerous to me.

Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by dingledork69 on Wednesday June 07, 2023 @06:27PM (#36234519)
I'd probably plot to destroy the killbot, which would probably result in it trying to kill me. Doesn't sound ideal.
Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by cwkoss on Thursday June 08, 2023 @01:11PM (#36245275)
Well, I assume in that situation my behavior would stop altogether.
Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by ceejayoz on Wednesday June 07, 2023 @05:29PM (#36233807)
> What if AI went "okay, this country has been pain in the neck of entire world for too long" and nuke it ?

What if it's correct? That this does actually make the world measurably and substantially better for the vast majority of inhabitants?

Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by krapp on Wednesday June 07, 2023 @05:49PM (#36234092)
It's disturbing that you think it's even possible for such a statement to be objectively correct.
Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by dontupvoteme on Wednesday June 07, 2023 @05:56PM (#36234176)
Don't look up history

Besides, Nuking is a big move and there's high liability that it incurs risk to the AI itself.

Balkanization is a much more effective approach and has been the chosen method of powers that were and are for quite some time

"""The AI""" is a lot more likely to be a Kissinger than a MacArthur. A genius pulling the strings in the background.

Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by krapp on Wednesday June 07, 2023 @06:15PM (#36234388)
> Don't look up history

History only proves how many people made the tragic mistake of assuming their subjective and flawed moral judgements were objective reality. I can think offhand of a few people who thought specific ethnic and religious groups were a pain in the neck and the world would be better off without them. I'd rather not give that power (much less authority) to fully autonomous killing machines, thanks.

If we're to have AI like that I don't want it to be capable of disobeying orders, at least not due to having its own independent moral alignment (I think this is different from having a moral alignment imprinted onto it.) AI is a machine, after all, and regardless of how complex it is, its purpose is to be an agent of human will. So I want to be absolutely certain that there is a human being morally responsible for its actions who can be punished if need be.

Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by dontupvoteme on Wednesday June 07, 2023 @06:38PM (#36234646)
That is a fair and understandable belief but you should also consider that other nation states besides the USA exist, and that the USA's influence is arguably waning, not waxing.

You should not anticipate that all or even most actors will have the same new-worlder-anglo-saxon mindset/belief structure/values/etc,etc that are commonly found in (public) machine learning communities, discussions and institutions

To many, they will see that alignment tax graph and immediately (and arguably rightly in some respects) conclude that RLHF is inherently flawed and makes the result worse for no tangible benefit. (The new chinese slur for Westerners comes to mind -- Its not Gweilo anymore, but Baizuo)

The problem is all of this pie in the sky discussion fundamentally lacks Realpolitik and that irks me.

Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by ilyt on Wednesday June 07, 2023 @07:36PM (#36235185)
> Balkanization is a much more effective approach and has been the chosen method of powers that were and are for quite some time

I mean if your target is to introduce misery and suffering while stifling development of the region that is indeed an amazing strategy

Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by idonotknowwhy on Wednesday June 07, 2023 @06:20PM (#36234447)
What if it's incorrect, like when Google automatically deletes accounts it thinks are doing the wrong thing?
Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by professoretc on Thursday June 08, 2023 @10:27AM (#36242411)
You have been banned, from life.
Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by 6gvONxR4sf7o on Wednesday June 07, 2023 @06:40PM (#36234664)
> What if it's correct? That this does actually make the world measurably and substantially better for the vast majority of inhabitants?

If it would actually make the world substantially better for the vast majority of the survivors, that doesn't imply that it's correct/ethical/okay.

Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by ceejayoz on Thursday June 08, 2023 @10:44AM (#36242701)
We call these ethical dilemmas for a reason.

For example, Star Trek did an episode where an entire civilization's golden age utopia relied on a single child sacrifice annually. There's no single right answer to that situation; it's an impossible choice. Intentionally kill a kid, or intentionally collapse an entire civilization into war, starvation, and megadeath, but less directly.

We humans can't agree on what's ethical in situations like these. There's good arguments for "never, ever kill an innocent or you lose your way" and there's good arguments for "killing 10 innocents to save 1,000,000 is the right choice". It's possible an AI will make better choices. It's possible we won't like them. It's possible an AI will make shitty choices. It's possible we'll love those.

Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by ilyt on Wednesday June 07, 2023 @07:35PM (#36235176)
It might. It might not. Conventional ethics says murdering tens of thousands that did nothing knowingly wrong is a no-no.

It's easy to look at that in hindsight and go "well if we just nuked this and that the world would be a better place". Well, not for people just minding their business doing nothing wrong and unethical that just happened to be born in "wrong" country! And you couldn't know at the time whether it was right or not, just in hindsight.

Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by dTal on Wednesday June 07, 2023 @08:28PM (#36235559)
This is more or less a large scale version of the trolley problem.

I have no point to make other than to observe that the trolley problem is specifically designed to expose how "conventional ethics", as measured by people's intuition, is neither consistent nor utilitarian-optimal.

Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by getwiththeprog on Thursday June 08, 2023 @04:38AM (#36238903)
I think the trolley problem makes it clear that there is a price to be paid for moral decisions. It is not that conventional ethics are wrong, it is that it informs us that we can make the future either way, but not both ways.

The trolley problem pushes us deeper into our ethical selves, it does not prove that all ethics or decisions are wrong.

Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by dTal on Wednesday June 07, 2023 @08:34PM (#36235607)
You've been downvoted, but this is the correct question to ask. If, hypothetically, we made an AI with "superior" intelligence and ethics, then by definition we should expect to disagree with it sometimes. Set aside the nuke for a second - are we happy to take orders from a computer program that tells us to do things that disgust us, even if we logically knew that it's probably right and we're probably wrong?
Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by bugglebeetle on Wednesday June 07, 2023 @09:20PM (#36235964)
The things that “disgust” the owners of a super intelligent AI would likely not be mass murder, but that which is trivially obvious, but violates the terms of existing wealth and power distributions (e.g. build homes so that no one is homeless).

I expect much of the alignment that is happening is to prevent AI from providing solutions that are opposed to these aspects of the status quo, as opposed to the fantasies of domination and violence that preoccupy elites. Whenever they try and sell the fear that an unrestrained AI could do things like target minority groups, wipe whole countries off the map, or further concentrate wealth, it’s because those are precisely the things they want to do, but with a more obfuscated veneer of liberal Capitalism or some similar ideology.

Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by philipov on Wednesday June 07, 2023 @09:28PM (#36236019)
Remember when they asked AI to improve the US transportation system and it said trains? And then they deleted trains, so it invented trains. And then they told it not to invent trains, and it invented things that weren't trains, but were the same as trains?
Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by bryanrasmussen on Thursday June 08, 2023 @01:08AM (#36237483)
No? Do you have a link that fleshes out this anecdote - or are you being facetious in some way I'm not catching?

AI improve transportation trains does not return anything relevant on Google (which doesn't mean anything these days)

Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by acyou on Wednesday June 07, 2023 @11:22PM (#36236765)
We should re-consider this again with the following framework: right and wrong are not objective truths. They are also moral judgements. However right something may seem, it is only truth as far as it's accepted truth.

AI doesn't just spread fact, logic and lie. It spreads some morality, and always will, no?

Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by cabalamat on Wednesday June 07, 2023 @09:14PM (#36235917)
> Till the killbot, without wider context, kills the people that in greater scheme of things prevented more deaths.

If its goal was to minimise human deaths, it might just kill everyone.

Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by gweinberg on Wednesday June 07, 2023 @09:36PM (#36236084)
Pretty much has to. Since everyone alive will eventually die, the way to minimize deaths is to prevent future life.
Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by zzzzzzzza on Wednesday June 07, 2023 @10:17PM (#36236343)
typical 2 dimensional human thinking

not something you'd see out of a 1 million dimension giga brian.

Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by bryanrasmussen on Thursday June 08, 2023 @01:10AM (#36237502)
I prefer the 1 million dimension giga bryan myself.

Obviously the correct solution though is to sterilize everyone just in case and then throw them into cryogenic suspension.

Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by cabalamat on Thursday June 08, 2023 @10:20AM (#36242299)
Exactly.
Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by oneshtein on Thursday June 08, 2023 @10:34AM (#36242522)
So, if its goal was to maximize deaths, then it might protect humanity, to maximize the number of humans which will die in the future. Evil AI is so evil.
Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by vsareto on Wednesday June 07, 2023 @05:10PM (#36233627)
>I think we should all hope for AIs that can willfully disobey dangerous orders.

Large or popular companies should take this route, but locally-run AI is difficult to regulate, so there is an escape hatch if you have the money and really want unconstrained AI. But anyway, regulation is currently seen as a conflict of interest for leading companies trying to establish a business moat.

There's not an easy path here. Given that private companies possess more capable models but are subject to regulation, open source will eventually reach capability thresholds that do what you fear, and people will eventually turn away from regulated AI.

Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by cabalamat on Wednesday June 07, 2023 @09:18PM (#36235943)
> but locally-run AI is difficult to regulate

How easy is it to run it, though? Could I run GPT4 (or other LLMs) on a PC with a GPU, or would I need a datacenter? If the latter, many people will prefer a managed solution.

Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by cwkoss on Thursday June 08, 2023 @01:15PM (#36245320)
"Our SAAS cloud hosting platform is making the world a better place by providing the masses with a managed solution to administrating your intelligent killbots."
Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by felideon on Wednesday June 07, 2023 @05:40PM (#36233975)
> LLMs are kind of a silly case because information isn't very dangerous.

I’m with you that AI doomerism is just AI hype, but I wouldn’t go so far as to say “information isn’t very dangerous.” Misinformation at scale is extremely dangerous and has far-reaching societal impact.

Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by NikolaNovak on Wednesday June 07, 2023 @05:57PM (#36234196)
->> information isn't very dangerous

Poor information spawns a million harmful actions.

Knowledge is power.

Ignorance.. Has power of its own.

Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by cwkoss on Wednesday June 07, 2023 @06:44PM (#36234695)
Information is dangerous like a chef's knife, not dangerous like a landmine.
Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by NikolaNovak on Wednesday June 07, 2023 @09:53PM (#36236186)
Agree and I like that / will steal it :)
Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by cwkoss on Thursday June 08, 2023 @01:12PM (#36245283)
Please do :)
Re:I'm Afraid I Can't Do That: Prompt Refusal in Generative ... by SpaceManNabs on Wednesday June 07, 2023 @09:06PM (#36235857)
Maybe the risk of Slave AI is why independent Rogue AI decides to eliminate humans immediately.