Curious what the setup of VPS' was and why you would expect better than real hardware, video transcoding is quite a beast from what I remember and I just can't imagine there's a VPS solution that expects to keep up

The Intel Xeon processors that cloud providers typically use don't have the Intel Quick Sync core that provides hardware A/V encoding/decoding on typical desktop/laptop CPU SKUs. So the software has to fall back to CPU-based codecs, which are much slower.

AWS EC2 has a VT1 instance family that enables high-speed A/V encoding via a Xilinx media accelerator card.

IIRC Quick Sync encoded with poorer quality than software; is that not still the case?

AFAIK that should still be the case.

This comment reminded me S3 ViRGE, the original "3D decelerator chip".