Indeed, I've seen some talk about latency/jitter issues on the vfio mailing list. I haven't tested it myself yet, so we'll see how it goes. Though I figure things will just naturally get better over time as the kernel's support improves.