You can set up PhotoStructure to build smaller (4k, 1080p, whatever you want) previews on your home server, and rsync just the previews dir and DB periodically up to your VPS. A couple of my beta users do this. (I'm the author, btw).

https://photostructure.com/faq/why-photostructure/