The latency hit of the extra trip will definitely be felt by end users if the endpoint is far enough away (e.g. ~100-200ms.) You can mitigate the initial setup other ways though, like the CDN approach: terminate TLS much closer with a proxy and use a warm, pre-established backhaul connection to the origin.

Less round trips are always good, though, without any extra stuff to put in place.