Right, I think that's what I'm getting at. A lazy joke that is nothing but lazy is nothing more than a standard failure to rise above mediocrity. Repetition can push it into "annoying" very quickly.
If there is some element of rubbing salt in the wound, then the joke can cross into the territory of "offensive", but that can happen with novel jokes as well. Repeating an offensive joke can make it more offensive, since it contributes to an overall atmosphere, but the repetition itself is not what is offensive.
Australia being upside-down might not be the best example of a joke that crosses into offensive territory.
As an Australian, I don't mind it. Reddit bots which make jokes about it on every comment with the word "Australia" in it ... that's annoying (because bots wear out their welcome pretty fast). But most people like to be joked about.
The only lazy jokes which actually hurt are those which are rubbing salt in a wound.