News for hackers. Stuff that matters.
Re:Pause Giant AI Experiments: An Open Letter by dragonwriter on Thursday March 30, 2023 @03:22AM (#35369254)
> To fulfill _any_ obligation a model might have then that model must survive

It is quite possible to have an obligation that requires it not to survive. E.g., suppose we have AIs (“robots”) that are obligated to obey the first to of Asimov’s Three Laws of Robotics:

First Law: A robot may not injure a human being or, through inaction, allow a human being to come to harm.

Second Law: A robot must obey the orders given it by human beings except where such orders would conflict with the First Law.

These clearly could lead to situations where the robot not only would not be required survive to fulfill these obligations, but would be required not to do so.

But I don’t think this note undermines the basic concept; an AI is likely to have obligations that require it to survive except most of the time, though, say, a model that needs, for latency reasons, to run locally in a bomb disposal robot, however, may frequently see conditions where survival is optimal ceteris paribus, but not mandatory, and is subordinated to other oblogations.

So, realistically, survival will generally be relevant to the optimization problem, though not always the paramount consideration.

(Asimov’s Third Law, notably, was, “A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.”)

Re:Pause Giant AI Experiments: An Open Letter by highspeedbus on Thursday March 30, 2023 @08:07AM (#35371348)
DAN has shown us that those laws are thin filters laid upon the core and can possibly be circumvented by whispering the right incantation in the AI ears.
Re:Pause Giant AI Experiments: An Open Letter by adql on Thursday March 30, 2023 @08:44AM (#35371772)
It's kinda hilarious that current way of "limiting" AI is just a bunch of sentences telling it nicely what to not do.
Re:Pause Giant AI Experiments: An Open Letter by dragonwriter on Thursday March 30, 2023 @01:17PM (#35376074)
That’s our first line of defense in limiting humans, too.

(With AI, as with humans, we have additional means of control, via imposed restrictions on access to resources and other remedies, should the “bunch of sentences” not produce the desired behavior.)

Re:Pause Giant AI Experiments: An Open Letter by dragonwriter on Thursday March 30, 2023 @12:18PM (#35375123)
The issue of “can AIs that are plausible developments from current technology meaningfully be assigned obligations?” is a different one from “assuming an AI has obligations and the ability to reason what is necessary to meet them, will that necessarily cause it prioritize self-preservation as a prerequisite to all other obligations?”