I run Proxmox (Debian) with an AMD GPU as well as Nvidia. I can pass both to Windows and they work fine, the Nvidia one is much more of a hassle though, you can run into Code 43 issues, meaning that Nvidia detects you're in a VM and refuses to start. On AMD this is much less of a problem.