News for hackers. Stuff that matters.
Re:Proxmox VE: Import Wizard for Migrating VMware ESXi VMs by matthew-wegner on Wednesday March 27, 2024 @02:30PM (#39842870)
I'm in game development, and I run ESXi for two reasons:

* Unmodified macOS guest VMs can run under ESXi (if you're reading this on macOS, you have an Apple-made VMXNet3 network adapter driver on your system--see /System/Library/Extensions/IONetworkingFamily.kext/Contents/PlugIns/AppleVmxnet3Ethernet.kext )

* Accelerated 3D has reasonable guest support, even as pure software. You wouldn't want to work in one of those guest VMs, but for any sort of build agent it should be fine, including opening i.e. Unity editor itself in-VM to diagnose something

Does anyone know where either of these things stand with Proxmox today?

I imagine macOS VM under Proxmox is basically a hackintosh with i.e. OpenCore as bootloader?

Re:Proxmox VE: Import Wizard for Migrating VMware ESXi VMs by bonton89 on Wednesday March 27, 2024 @03:10PM (#39843335)
The lack of 3D paravirtual devices is a real sore spot in kvm. To my knowledge, virgl still isn't quite there but is all there is so far in this space. VMware has the best implementation IMO and everything else is a step down.
Re:Proxmox VE: Import Wizard for Migrating VMware ESXi VMs by moondev on Wednesday March 27, 2024 @03:52PM (#39843773)
Nested virtualization also works great on ESXi for macOS guests ( so you can run docker desktop if so inclined ). I believe this is possible with proxmox as well with CPU=host but have not tried it.

For graphics, another cool thing is intel iGPU pci passthrough - I have had success with this when running esxi on my 2018 mac mini https://williamlam.com/2020/06/passthrough-of-integrated-gpu...

Re:Proxmox VE: Import Wizard for Migrating VMware ESXi VMs by mysteria on Wednesday March 27, 2024 @02:47PM (#39843072)
The SPICE backend has decent OpenGL 3D support with software rendering, it's slow but it works for simple graphics. It's intended for 2D so the desktop's pretty fast IMO. That only works for Linux and Windows guests though, not Apple ones.

MacOS VMs do work in Proxmox with a Hackintosh setup but you pretty much have to passthrough a GPU to the VM if you're using the GUI. Otherwise you're stuck with the Apple VNC remote desktop and that's unbearably slow.

Re:Proxmox VE: Import Wizard for Migrating VMware ESXi VMs by zozbot234 on Wednesday March 27, 2024 @03:11PM (#39843342)
For paravirtualized hardware rendering you can use virtio-gpu. In addition to Linux, a Windows guest driver is available but it's highly experimental still and not very easy to get working.
Re:Proxmox VE: Import Wizard for Migrating VMware ESXi VMs by zbrozek on Wednesday March 27, 2024 @09:07PM (#39846647)
What's the best solution for remote viewing with virtio-gpu? I vaguely recall that it had some incompatibility with spice.

I have a bunch of qemu/kvm virtual machines for various tasks. For two of them I decided that graphics performance needed to be at least OK and ended up buying and passing through old Quadro cards. It'd be lovely to not have to do that.

Re:Proxmox VE: Import Wizard for Migrating VMware ESXi VMs by mysteria on Thursday March 28, 2024 @10:37AM (#39852121)
I don't think virtio-gpu works on SPICE. I only use it for VMs on my desktop as I can display the output on the local machine using virt-viewer.
Re:Proxmox VE: Import Wizard for Migrating VMware ESXi VMs by zbrozek on Thursday March 28, 2024 @11:33AM (#39852797)
Still the case, huh. That's unfortunate. All of my VMs run on a headless server, so my only means of use is via some form of remote desktop.
Re:Proxmox VE: Import Wizard for Migrating VMware ESXi VMs by rufugee on Wednesday March 27, 2024 @03:32PM (#39843554)
Does this work with VMWare Workstation as well? I'd love to run macOS in a VM on my Linux desktop for the few apps I have to use on macOS...
Re:Proxmox VE: Import Wizard for Migrating VMware ESXi VMs by zerkten on Wednesday March 27, 2024 @03:47PM (#39843720)
It has in the past for me, but I haven't run it since 2021.
Re:Proxmox VE: Import Wizard for Migrating VMware ESXi VMs by justinclift on Wednesday March 27, 2024 @08:38PM (#39846433)
This works pretty well for running macOS on Linux:

https://github.com/notAperson535/OneClick-macOS-Simple-KVM/

Mostly used it when trying to track down reported macOS bugs for an OSS project, so maybe once every few months. But it's worked quite well at those times. :)

Re:Proxmox VE: Import Wizard for Migrating VMware ESXi VMs by lathiat on Wednesday March 27, 2024 @09:00PM (#39846593)
quickemu also does the job, not just for macOS but many operating systems

"Quickly create and run highly optimised desktop virtual machines for Linux, macOS and Windows; with just two commands. You decide what operating system you want to run and Quickemu will figure out the best way to do it for you." https://github.com/quickemu-project/quickemu

Re:Proxmox VE: Import Wizard for Migrating VMware ESXi VMs by justinclift on Wednesday March 27, 2024 @09:22PM (#39846741)
Thanks, hadn't come across that before. :)
Re:Proxmox VE: Import Wizard for Migrating VMware ESXi VMs by t3rra on Wednesday March 27, 2024 @04:10PM (#39843980)
nobody asked.
Re:Proxmox VE: Import Wizard for Migrating VMware ESXi VMs by oneplane on Wednesday March 27, 2024 @04:29PM (#39844208)
Apple has been adding a lot of virt and SPICE things IIRC. Some of it isn't supported in VMware (it lacks a bunch of standard virt support), but the facilities are growing instead of shrinking which is a good sign.

On Proxmox you can do the same. You're going to need OpenCore if you're not on a Mac indeed. But if you're not on a Mac you're breaking the EULA anyway.

Re:Proxmox VE: Import Wizard for Migrating VMware ESXi VMs by mrpippy on Wednesday March 27, 2024 @05:52PM (#39845047)
Note that macOS guest support ended with ESXi 7.0: https://kb.vmware.com/s/article/88698

Running macOS is only supported/licensing-compliant on Apple-branded hardware anyway, and with the supported Intel Macs getting pretty old this was inevitable anyway.

Re:Proxmox VE: Import Wizard for Migrating VMware ESXi VMs by moondev on Wednesday March 27, 2024 @06:16PM (#39845261)
Mac mini 2018 is still the best Mac for vms

   6 cores 12 threads
   64GB DDR
   nvme
   4 thunderbolt3 ports for pci expansion
   10GbE onboard nic
   boots ESXi
   boots Proxmox
   boots or virtualizes windows
   boots or virtualizes linux
   boots or virtualizes macos
   iGPU passthrough
   Supports nested virt
Re:Proxmox VE: Import Wizard for Migrating VMware ESXi VMs by greggsy on Wednesday March 27, 2024 @11:02PM (#39847317)
If you’re just running a bare metal hypervisor then you may as well just go for a second hand tiny form factor platform. No point paying 15-20% extra for the same specs in a svelte case if you’re not running native macOS.
Re:Proxmox VE: Import Wizard for Migrating VMware ESXi VMs by moondev on Thursday March 28, 2024 @01:28AM (#39848051)
> No point paying 15-20% extra for the same specs in a svelte case if you’re not running native macOS.

The main point is proper non-hackintosh support for running (unlimited amount) of macOS vms on esxi - which requires Apple hardware. Running macOS, Windows and Linux on the metal is another benefit. IMO it's almost as versatile as a framework chromebook (chromeos, crostini, kvm, android with play store)

Re:Proxmox VE: Import Wizard for Migrating VMware ESXi VMs by unquietwiki on Wednesday March 27, 2024 @07:11PM (#39845748)
* Proxmox can emulate VMWare adapters. https://pve.proxmox.com/wiki/QEMU/KVM_Virtual_Machines

* If you don't mind dedicating the video card to a VM, you can do PCI-passthrough. https://pve.proxmox.com/wiki/PCI_Passthrough

Re:Proxmox VE: Import Wizard for Migrating VMware ESXi VMs by m463 on Wednesday March 27, 2024 @09:36PM (#39846829)
You can run macos vms under proxmox.

You can also accelerate video (though it's uncertain what "pure software" means?)

this guy: https://www.nicksherlock.com

has been writing up proxox vms running macos for a while, the latest being:

https://www.nicksherlock.com/2022/10/installing-macos-13-ven...

yes, it uses opencore.

I haven't really bothered with macos GPU acceleration, It is possible but a little fiddly with an nvidia card. I mostly rely on screen sharing to display the remote vm and that's really good (rich text copy/paste, drag and drop files, etc)

As to general guest 3d, you can do this with GPU passthrough, and although it's technical the first time, each vm is then easy.

basically I add this line to each VM for my graphics card:

  hostpci0: 01:00,pcie=1,x-vga=1
and this passthroughs a USB controller for my USB dac:

  hostpci1: 03:00,pcie=1
(this is specific to my system device tree, it will usually be different for each specific machine)

The passthough has worked wonderfully for Windows, ubuntu and arch linux guests, giving a full accelerated desktop, and smooth gaming with non-choppy sound.

two things I wish proxmox did better:

- passthrough USB into containers

usb is pretty fiddly to propagate into a container. It is marginally better with VMs but sometimes devices still don't show up.

- docker/podmain containers

proxmox has lxc containers, but using them is a little like being a sysadmin. Docker/podmain containers that you can build from a dockerfile would be a much better experience.

Re:Proxmox VE: Import Wizard for Migrating VMware ESXi VMs by wutwutwat on Thursday March 28, 2024 @12:27AM (#39847794)
I run macOS/OSX Sonoma in Proxmox. It does pcie/gpu passthrough (AMD rx580 8gb). The proxmox host is running a AMD Ryzen 7 5700G cpu and has 64gb memory. The mac vm disks are on a zfs fs on a wd black nvme ssd.

It's the fastest mac I've ever owned and it's virtual, executed on a machine running a chip that apple never supported, and you'd never be able to tell it was a vm unless you were told so. kvm and vfio are amazing pieces of software.

A good place to start: https://github.com/luchina-gabriel/OSX-PROXMOX

Re:Proxmox VE: Import Wizard for Migrating VMware ESXi VMs by rfoo on Thursday March 28, 2024 @03:35AM (#39848619)
> zfs fs on a wd black nvme ssd

Why? IIRC running ZFS on NVMe SSDs limit their performance seriously. With sufficient queue depth modern SSDs can easily get up to 1mln+ IOPS and on ZFS I can barely get 100k :(

Re:Proxmox VE: Import Wizard for Migrating VMware ESXi VMs by tpetry on Thursday March 28, 2024 @04:10AM (#39848764)
Most probably because ZFS still has the most extensive data correctness guarantess of all filesystems on Linux. Yes, bcachefs will have checksums too but it it is still in beta.
Re:Proxmox VE: Import Wizard for Migrating VMware ESXi VMs by bonton89 on Thursday March 28, 2024 @08:16AM (#39850281)
btrfs has checksums as well.
Re:Proxmox VE: Import Wizard for Migrating VMware ESXi VMs by wutwutwat on Thursday March 28, 2024 @07:07PM (#39858518)
copy on write, instant snapshots, data checksumming, easy rollbacks, auto trim, compression

proxmox takes advantage of zfs' features if available, like when you select block storage on zfs it makes each vm disk its own dataset which you can tune as you see fit for each one. cloning a vm is instant if you make a cow clone. suspending the vm to do snapshot based backups is also nearly instant, etc

Disable atimes and turn off the arc cache and metadata cache (don't really need either with fast storage compared to spinning rust) and use lz4 compression and you minimize write amplification or needless wear on the drive. zfs is perfectly fine on ssd for my use case