On Ubuntu ish systems (not sure about base debians), you can get full KVM support in qemu by installing the "qemu-kvm" package right out the repos and then starting your VMs with the "-enable-kvm" switch. You may also need to get EFI working in qemu to support OSX EFI bootloader, which you can get by installing the "ovmf" package right out the repos, and then adding the "-bios OVMF.fd" switch to your qemu command (OVMF.fd being the firmware file used by qemu to do EFI, which the "ovmf" package seems to install near /usr/share/qemu/OVMF.fd). I'm not sure about GPU passthrough, but with full KVM support enabled, you will immediately notice a pretty huge difference in the performance of the VM.

Thanks, I think I did enable KVM in QEMU and OS X is pretty snappy. There's a bug where the mouse can't tell where the screen bounds are, and stops in the middle of the screen, but that's unrelated. Thanks for the tip!

Qemu can do it, but you explicitly need to configure it to use kvm. And then you need to set it up to use GPU passthrough.

It's a somewhat involved process, especially for GPU passthrough.