News for hackers. Stuff that matters.
Re:Statement on AI Risk by lxnn on Tuesday May 30, 2023 @01:19PM (#36127594)
Technically, I think it's not that instrumental goals tend to converge, but rather that there are instrumental goals which are common to many terminal goals, which are the so-called "convergent instrumental goals".

Some of these goals are ones which we really would rather a misaligned super-intelligent agent not to have. For example:

- self-improvement;

- acquisition of resources;

- acquisition of power;

- avoiding being switched off;

- avoiding having one's terminal goals changed.

There are no comments on this comment yet.