Its turtles all the way down. You need an anchor of trust. A trust root. This is the public PKI system trusted root store.

Even if you obtain the self-signed cert out of band (and explicitly trust it), how do you authenticate that channel?

Self-signed certs are not scalable or particularly useful for internet users. Please don't recommend this.