I don't know what the Vultr instance uses but i had cable connection on both of my devices (home server and local machine).

I suspected ssh over tinc might bring latency since there is an extra layer of encryption, i will try ssh tunneling. Thanks!