It's true that total latency is what matters, and minimising latency by sitting near one of them and taking advantage of low-latency connections between exchanges sounds reasonable.