News for hackers. Stuff that matters.
Re:Rage Against the God Machine by simonh on Sunday June 07, 2020 @05:22PM (#23450953)
This is neither a flaw in our computers nor a flaw in our programming languages. It’s a consequence of two things. Firstly computers only do precisely what you tell them to do. Secondly we don’t know exactly what we want them to do.

It’s that second part that’s the real issue. It turns out very precisely, and completely unambiguously only expressing exactly what you want to happen, when you want it to happen, in every conceivable circumstance that might occur, based on very specific assumptions of the conditions in which it might happen, is quite difficult and complicated and error prone to achieve. Also if we get any particular detail wrong - the assumptions about the problem, the order of events, the anticipated conditions, the system fails.

But that’s it. That’s the problem. Beings precise and unambiguous is hard. Formulating our assumptions is hard. Anticipating every circumstance is hard.

Why is this so? Our brains just don’t work that way. They’re randomly variegated, fuzzy logic, neural network, pattern matching, feedback loop driven mammal organs. It’s amazing they work at all. They only get things sufficiently right because all the ones that got it badly enough wrong got themselves killed before they had offspring. Not only that, but we’re actually only just barely intelligent in a general sense. After all, in evolutionary terms we only just very recently gained enough intelligence to develop significant technology, otherwise we’d have done it sooner. We’re at the very edge of the lowest threshold for achieving a technological society. No wonder we find this stuff hard.

There are no comments on this comment yet.