With a sufficiently large key that attack you're proposing is effectively impossible (people would notice the yottabytes or however many you need to pull it off).

Although you still do need to protect against people making false claims.