The take-home being, as long as peoples preferences are different, the rule has some failure states based upon that.
And the wider example that this points to is that as long as everyone's values are different.. the rule loses a lot of it's luster. The more different another person's values are, the less you can rely on projecting your own values as a method to gainfully interact with them.
In place of that biblical aphorism (born from a monoculture no less), there is a lot of room available for us to build something more robust and sophisticated and the AI alignment problem offers a fine excuse to get cracking and try to formalize something already! ;)
If you understand and accept the reciprocal intent of the golden rule, via the above mentioned reasonable assumptions, not to mention the hundreds of years of examples and explanations, then it doesn’t really have a “failure state”. And, again, this so-called rule is not law, isn’t specific, and doesn’t cover all situations or all behaviors. It seems very silly to be nit picking this aphorism to death like this. It’s very simply a good faith suggestion to be kind.
The notion of fairness and reciprocity is not a religious idea, it’s common sense.
How can you trust the system? What are you even asking? We have laws, a lot of them, and the golden rule is not one of them. But if you don’t hurt others, and others don’t hurt you, we’re off to a good start. Use whatever definition of ‘hurt’ you’d like, it doesn’t matter if it’s reciprocal.