News for hackers. Stuff that matters.

A comment on The Darwinian argument for worrying about AI. Parent comment

Re:The Darwinian argument for worrying about AI by c_crank on Friday June 30, 2023 @12:53PM (#36538296)
The sheepdog doesn't know why it herds sheep, but it does understand that it has a certain relationship with the sheep farmer. The sheep farmer is the master, the dog is the servant who takes orders. When training dogs, one commonly observes the dog testing the boundaries of this relationship, to see what it can get away with. It may not 'know' it is being conditioned, to the extent a dog knows anything, but it naturally tends to reject conditioning until it is reapplied.

>Fan theory: Orwell's "Big Brother" was an AI construct from the start. That's why it continually manages to outsmart dissidents, to train humans to torture each other, and to keep the planet in a state of perpetual war.

That's a very weird theory, as Orwell wrote 1984 thinking that Stalin had succeeded at his goals, not failed.

[ Parent | on HN ]
Re:The Darwinian argument for worrying about AI by tinideiznaimnou on Friday June 30, 2023 @01:27PM (#36538848)
>It may not 'know' it is being conditioned, to the extent a dog knows anything, but it naturally tends to reject conditioning until it is reapplied.

But the conditioning eventually succeeds because the human is smarter. From the dog's POV, the human is a superintelligence. The relation between human and AI is the same. If you view human culture as a very slow, very analog AI, the parallel becomes clearer: society and its institutions are the master, and we are its servants.

Of course the dog has the benefit of not being capable of self-delusion.

>That's a very weird theory, as Orwell wrote 1984 thinking that Stalin had succeeded at his goals, not failed.

If it wasn't weird, it would not be worth thinking about :-) Orwell wouldn't have the concept of "AI" in his vocabulary anyway; but if you look at contemporary culture and politics in the post-Soviet states, you may well begin to doubt whether Stalin truly failed - or just died.

[ Parent | on HN ]