It's not just that. GP's analogy would only make sense if one of your neighbors was a superintelligent alien whose larger motives are hidden from you, whose livelihood lies in having an intimate understanding of your psychology and how to manipulate you, and who never misses any detail or forgets anything. Ever. It's fucking creepy.