I believe the most promising route is to not try to do passthrough but to use a "virtual" gpu, aka a paravirtualized device. Thanks to the contributions of an open-source developer*, virtio-gpu has been ported to macOS (it already works on Linux but not on Windows and soon it will be on macOS). Apple is also reportedly working on bringing better support for macOS virtualized environments [2].

It means that users will be able to create any number of macOS instances with 3D acceleration, without having to care about trying to do complicated things (mediated devices or single-gpu passthrough or pci-passthrough of a second GPU).

As of now, the performance hit is high (more than 50%), but it will certainly improve over time, and many users might already find it acceptable (= people NOT playing recent video games).

Icing on the cake, thanks to the EGL-headless display (which is independent of virtio-gpu) and VNC, it is possible to connect to these instances remotely.

Shameless plug: I am working on Phyllome OS [3], a Fedora Remix with the goal to include the necessary plumbing to ease advanced virtualization techniques for end-users, and to offer curated virtual machine models.

I currently only use virtual machines as desktop machines, including locally, and even my poor x230 laptop can do it. I keep the host operating system intact (Phyllome OS), and use VMs as desktop machines. It has drawbacks (difficult USB access; displays models that were not designed for local use; sound support), but I firmly believe it will get better over time and, perhaps, one day, end-users will be able to easily deploy any modern OS on any modern hardware...

[1] Sorry, I could not find the link to the repository [2] https://passthroughpo.st/mac-os-adds-early-support-for-virti... [3] https://phyllo.me/

[edit] Delete mention of KVM/QEMU

> without having to care about trying to do complicated things ([...] pci-passthrough of a second GPU)

Actually, GPU passthrough is quite simple: configure host Linux to assign a virtual driver to a secondary GPU on boot and configure QEMU to use that driver on guest boot[1]. If you buy a GPU that is natively supported by macOS (e.g., Sapphire Radeon RX 580 Pulse 4GB), you get 99% performance and 100% stability of a native Mac (not a single macOS/QEMU crash in years for me).

Moreover, if you configure your QEMU mouse/keyboard to use evdev[2] (event devices), you can game under macOS/QEMU or Windows/QEMU without any input lag.

[1] https://github.com/kholia/OSX-KVM/blob/master/notes.md#gpu-p... [2] https://passthroughpo.st/using-evdev-passthrough-seamless-vm...

By complicated, I mean that you have to modify your the default behavior of your host OS (which by default will grab any hardware) and pick a GPU that is supported by the guest OS. People with Nvidia GPUs cannot do that for macOS. Ideally, out-of-the-box, with a few clicks, most people including non-tech savy ones should be able to enjoy 3D acceleration in their VM, regardless of their host configuration.

Besides, in the long term, Apple is likely to drop support for AMD or Intel GPU in their OS (but yes, they might also drop support for virtio devices).

Yes, evdev is currently the best way to share your inputs devices to a VM, thank you for pointing that out (alas, it is only available in distributions with very recent virtualization-related packages).

In my (perhaps narrow) view: virtualization should be about making you less reliant on the underlying hardware. I wish that GPU manufacturers would agree on a common standard to allow users to split their beefy GPU into smaller parts (Intel is apparently dropping support for vfio-mdev (Intel GVT-d), adopting SR/IOV instead for their latest offerings. But SR/IOV is only available on professional models for AMD and Nvidia. In summary, there won't be a standard any time soon, leaving virtio-gpu as the sole hardware-agnostic contender)

> Besides, in the long term, Apple is likely to drop support for AMD or Intel GPU in their OS

Off topic but I’m curious about this. On one hand I think it’s a very Apple like thing to do but on the other hand it would practically kill Mac for pro use cases, which is barely hanging on by a thread right now. Could you imagine buying a $10,000+ workstation and Apple telling you 1 8k display is enough?

I think it really hinges on what the next Mac Pro looks like. If it’s arm based and allows expansion then it’ll likely support AMD and intel gpus and support for those will stick around for years. If not , then the current intel macs and their egpu support will be the end of the line.

You correlate an Apple GPU with a hard limit on the amount of displays you can use, but honestly I don't see the point. Apple has bittersweet limits on the amount of displays one can connect right now, but it is no guarantee that this will continue in the future.

With what they unveiled in terms of GPU performance with the Macbook Pro, the writing's on the wall; Apple is not going to support third-party GPUs going forward. The Mac Pro will only offer Apple CPUs and GPUs, I'm sure of it.

IME you’ll also need to use isolcpus and affinity on the KVM threads to get stable GPU performance (with macOS or other OSes). Plus there can be some other fun issues depending on your setup, like having to flash a different BIOS on your card or having to deal with multiple devices on the same IOMMU domain (I had to pass through a USB controller with my card).

"you get 99% performance and 100% stability of a native Mac (not a single macOS/QEMU crash in years for me)"

I am curious how this is so stable for you. Are you running this on Intel? Just last week I installed this on my AMD Linux workstation to test some stuff with Xcode and qemu would crash randomly while compiling LLVM/Clang.

Yes, I run this on Intel i7, under Ubuntu 20.04, with QEMU 4.x, as "-machine q35".

Is there some info on virtio GPU for OSX development somewhere ?

I knew virtio support was happening but hadn't seen any info on the GPU bit.

The port I had in mind is this one [1], and the use case is to make it possible to run 3D accelerated Linux guests on macOS hosts. I may have wrongly believed that it would be useful for macOS guests.

Besides, it is correct that whatever happens on the driver side of things, Apple will have to approve it first (as the other reply pointed out)

[1] https://mail.gnu.org/archive/html/qemu-devel/2021-02/msg0423...

macOS guests use Apple’s own ParavirtualizedGraphics framework, which comes with none of the downsides of virtio-gpu. It pipes through serialised Metal at native performance.

I'm quite sure virtio-gpu for OS X will _never_ happen unless Apple approves of it. 3D drivers are like the most closed things in OS X.

Wow that's very surprising apple is trying to give better support for virtual environments instead of worse.. Do you know if they're allowing their iOS dev tools to run in there so you don't have to buy a mac anymore?

I have been developing a toy iOS/macOS app running macOS with this. I am able to login with my dev program credentials and do everything I need. I haven't tried submitting anything to the store.

So the virtualization hooks aren't just in MacOS? I could compile your app on my M1 iPad and run full MacOS virtually?

No, Apple doesn't expose virtualization on iPad.

Apple is adding better support for virtual environments running on the Mac. Not virtual macs running on other machines. That is all coming from the community.

Yes, starting with CPUs circa 2014. You need CONFIG_DRM_I915_GVT from Device Drivers -> Graphics support -> Intel 8xx/9xx/G3x/G4x/HD Graphics -> Enable Intel GVT-g graphics virtualization host support.

There are a fixed number of partitions and a specific setup you need to do with /sys nodes, refer to the manual. Considering this and that support only became possible some time after 2014 it's worth realizing it does not work like normal VFIO passthrough. Why? No idea, seems like an oversight. Need to stop providing lying interfaces to users.

https://serverfault.com/questions/1048811/vfio-igpu-passthro...

Not really pass through, no. If CONFIG_DRM_I915_GVT is enabled in your kernel, you can use Intel's graphics virtualization system... basically a virtio style virtual device that shares the GPU between VM and host. IMO this is way more convenient than real passthrough, where the device is only available either to the VM or the host. The downside is that you don't get full performance in the VM.

"Intel GVT-g is a full GPU virtualization solution with mediated pass-through (VFIO mediated device framework based), starting from 5th generation Intel Core(TM) processors with Intel Graphics processors. GVT-g supports both Xen and KVM (a.k.a XenGT & a.k.a KVMGT). A virtual GPU instance is maintained for each VM, with part of performance critical resources directly assigned. The capability of running native graphics driver inside a VM, without hypervisor intervention in performance critical paths, achieves a good balance among performance, feature, and sharing capability."

https://github.com/intel/gvt-linux/wiki/GVTg_Setup_Guide

Gvt-g is pretty much dead. New hardware e.g. Tiger Lake with Xe isn’t supported and it’s my understanding that Intel has no plans to support.

See the following link for supported devices.

https://github.com/torvalds/linux/blob/bbf5c979011a099af5dc7...

New Intel hardware supports SR-IOV, the successor to GVT-g, but unfortunately driver support is apparently missing for Linux: https://www.reddit.com/r/VFIO/comments/o7l2zr/sriov_on_intel...

We are more than 2 years past the launch of Ice Lake now… so…

oh wow, I had no idea. Seems a shame... Are they offering anything to replace it? Or is it GVT-d or bust?

It didn't work when I tried to pass through a virtualized GVT instance in Catalina a couple years ago.

Although this all will not matter once macOS becomes exclusive to Apple Silicon.