Servers would have relatively static keys, while client keys could be as ephemeral as wanted. So rather than specify an IP address (location) as destination, you specify the recipient, by encrypting a request with eg. Google's SSL public key, signing it with your public key, and delivering it to something that will route this packet to Google (at whatever IP address that may be), so they can decrypt it, and produce a packet encrypted with your public key.
Certainly, networks can work without any delegation in this form; tor's hidden services show that this can work in practice. Pretty slow though. Reasonable performance (and accuracy) depends on not having every node on every network know the correct route to every other node, or having nodes have to reach a consensus (slow) on where to forward your packet at each hop. What's your plan for that?