News for hackers. Stuff that matters.
Re:Why libvirt supports only 14 PCIe hotplugged devices on x... by EmilioPeJu on Sunday August 06, 2023 @03:01PM (#37025368)
If I'm not wrong, the pre-allocation of I/O ranges in PCIe bridges is needed only if you intend to hot-plug devices that were not present in the first enumeration.. but in VMs the hardware is known from the start and the PCIe enumeration can assign I/O ranges only if devices underneath actually needs them... is there a reason why hot-plugging is needed in VMs?
Re:Why libvirt supports only 14 PCIe hotplugged devices on x... by xyzzyz on Sunday August 06, 2023 @03:13PM (#37025484)
Cloud customers love it when they can just attach stuff to their VMs without having to recreate them or even reboot them.
Re:Why libvirt supports only 14 PCIe hotplugged devices on x... by josephcsible on Sunday August 06, 2023 @03:51PM (#37025887)
Isn't the cloud notoriously worse about hotplugging anything than on-prem systems are? For example, vSphere supports hot adding CPUs and RAM, but Azure doesn't.
Re:Why libvirt supports only 14 PCIe hotplugged devices on x... by tyingq on Sunday August 06, 2023 @04:00PM (#37025993)
Seems unsurprising. On Azure, if it goes wrong, the various tenants aren't all working at the same company.
Re:Why libvirt supports only 14 PCIe hotplugged devices on x... by mgerdts on Sunday August 06, 2023 @03:14PM (#37025500)
> is needed only if you intend to hot-plug devices that were not present in the first enumeration

Correct. I regularly use VMs with more that 14 statically configured PCI devices using QEMU with libvirt without having to resort to qemu:cmdline.

Re:Why libvirt supports only 14 PCIe hotplugged devices on x... by dottedmag on Monday August 07, 2023 @03:35AM (#37031430)
* * *
Re:Why libvirt supports only 14 PCIe hotplugged devices on x... by mgerdts on Monday August 07, 2023 @09:43AM (#37034346)
There are 14 hardware based PCIe devices (mixture of NVME drives and NIC VFs) along with various other emulated PCI devices (virtio block, serial, etc.).

I have not tried to hot [un]plug devices with this configuration. It looks as though I’m likely to be disappointed if I try. Thanks for the explanation.

Re:Why libvirt supports only 14 PCIe hotplugged devices on x... by Nextgrid on Sunday August 06, 2023 @03:16PM (#37025513)
It’s not a common use-case but I could see it being useful for sharing hardware that requires exclusive access like GPUs/ML accelerators.

Currently if you need GPUs they come with the instance itself meaning you need to boot your VM from scratch, do the work and then shut it down to relinquish the GPU.

With hot-plug you could have continuously running VMs that only attach/detach GPUs as needed, no longer taking the overhead of a full cold boot/shutdown every time.

Re:Why libvirt supports only 14 PCIe hotplugged devices on x... by jeffreygoesto on Sunday August 06, 2023 @03:14PM (#37025502)
Devices passed from the host to the guest?
Re:Why libvirt supports only 14 PCIe hotplugged devices on x... by mgerdts on Sunday August 06, 2023 @03:17PM (#37025526)
Hot plugging refers to adding devices to the VM while the VM is running. Passing host devices through is commonly accomplished without hotplug.
Re:Why libvirt supports only 14 PCIe hotplugged devices on x... by jeffreygoesto on Sunday August 06, 2023 @04:55PM (#37026635)
Right. I should have been more clear that it can be you hotplug a host device and pass it in. Admitted, this is typically USB and not PCIe. And the PCMCIA days are over...
Re:Why libvirt supports only 14 PCIe hotplugged devices on x... by adql on Sunday August 06, 2023 @05:26PM (#37026968)
adding NVMe emulated storage would be one
Re:Why libvirt supports only 14 PCIe hotplugged devices on x... by dottedmag on Monday August 07, 2023 @03:12AM (#37031253)
Author here. As correctly guessed in other comments: cloud infrastructure.

To make public IPs and volumes hotpluggable without a guest agent running inside every VM one has to manage them in a way guest OS will handle hotplug using regular mechanisms. For volumes it's PCIe storage hotplug, for public IPs it's PCIe network card hotplug.

If a VM is used as a Kubernetes worker, couple of dozen volumes and public IPs attached is not an unlikely situation.