Re:That Alien Message
by jemfinch on Sunday August 20, 2017 @09:57PM
(#15061398)
The reason you're not contributing to the conversation is that your comment seems to reflect a naivete borne only by those few who haven't read Thompson's "On Trusting Trust".
tl;dr: We can't even trust the compiler, let alone a transhuman superintelligence's source code.
Edit: Beyond that, your post demonstrates an additional naivete in that you hold that a superintelligence's would even have source code. Today's ML models consist of thousands or millions of node weights, and the source code is uninteresting. A superintelligence would almost certainly be shifted even further toward data and away from code.
tl;dr: We can't even trust the compiler, let alone a transhuman superintelligence's source code.
Edit: Beyond that, your post demonstrates an additional naivete in that you hold that a superintelligence's would even have source code. Today's ML models consist of thousands or millions of node weights, and the source code is uninteresting. A superintelligence would almost certainly be shifted even further toward data and away from code.