You are forgetting round-trip request latency, handshakes, and dependent resources (don't know what to fetch until receiving the first thing, then the second level of things, etc., this can stack to multiple seconds).
High latency is very bad for browsing. The main things it is fine for are streaming audio/video (non-interactive), large downloads, and backing up/uploading files.
And the handshake time doesn’t compound; there is no need to open a new connection to the server for every resource needed by a webpage. The web browser just keeps reusing the same connection until it runs out of resources to fetch from the server.
Third party analytics and other resources often means it compounds even for handshaking, but dependent resources can compound too even on the same server even if they don't each need a handshake.