I do the same thing for a "virtual gaming pc" on proxmox with qemu/kvm. it was a pain to get working right with my specific hardware and with proxmox as a base, but is pretty nice now that it works. Many people use Arch linux or redhat distros to do this also, but pretty much any major distro should be capable. AFAIK it works on VMware ESXi too.

Wow, I just assumed virtualisation would kill performance - didn't realise it would even be an option

works out of the box on debian stable with KVM these days, assuming your hardware supports it