the vga arbiter stuff went away with UEFI based cards (about 2 years ago), consumer level cards now work without issue.
NVIDIA attempt to detect the VM and artificially disable the driver, but works perfectly if you tell KVM to hide the fact it's a VM (requires a flag to be set in the config).
the technical difficulty of pci passthrough consists of clicking "add pci device", then selecting your graphics card from the list.
To get a graphics card actually capable of this, you either need to buy the commercial grade versions (Quadro and FirePro) with are much more expensive, or buy a specific card for this purpose and deal with tons of issues[1] like not being able to restart the guest without crashing the host.
[1]: http://wiki.xenproject.org/wiki/Xen_VGA_Passthrough_Tested_A...