I was playing around with this a while ago, using cloud-init and virt-install. But I found it to be quite cumbersome if you don't have some kind of metadata service that can provide cloud-init with the user-data, ie. not using the nocloud provider.

I started looking at this mock EC2 metadata service, but never did anything with it: https://github.com/sjjf/md_server

Ninja-edit: It seems like virt-install rencently abstracted the whole NoCloud-provider, this changes a lot of things: https://blog.wikichoon.com/2020/09/virt-install-cloud-init.h...

We implemented our own mock ec2 metadata service for KVM cloud-init, probably the most fun project I've worked on.

The tricky part is intercepting the requests and redirecting them to your service

I've had luck with Proxmox and this Terraform provider: https://registry.terraform.io/providers/Telmate/proxmox/late...

Other than a working Proxmox install, you'll need to create a cloud-init template, which is documented here: https://pve.proxmox.com/wiki/Cloud-Init_Support

virt-install with kickstart or preseed files and VMs attached to the serial console, like

virt-install -n centos -r 1024 --cpu=host -l http://mirror.cogentco.com/pub/linux/centos/6/os/x86_64/ --disk path=/var/lib/libvirt/images/centos.img,size=10,bus=virtio --boot kernel_args="console=/dev/ttyS0" -w network=internal,model=virtio --initrd-inject=centos.ks -x "ks=file:/centos.ks console=ttyS0" --graphics none

Setting up the serial console requires some configuration on the host part, too, unfortunately, depending on the OS.

Just wanted to chime in and say that, for my purposes, Multipass has been perfect for work and personal use.

https://multipass.run/

(For others, it's a simple-to-use CLI that provisions Ubuntu instances.)

You mention Terraform, I'm using https://github.com/dmacvicar/terraform-provider-libvirt quite heavily and it works a treat.

That's the one I've had suboptimal experience with. Maybe I need to learn it better. What kind of storage are you using it with? I ran into issues using file-backed trying to keep everything in tf.

I’m copying qcow2 volumes with it to file based storage and some Ceph RBD as well, I mean AFAIK it just talks to the standard libvirt volumes interface so anything you can do with that you can do with the Terraform provider.

vagrant with the vagrant-libvirt plugin. The roboxes project provides pre-installed and configured vagrant boxes for allmost all distributions. Its really just a matter of seconds to bring up whatever distribution box you like. Even pre-configured and sysprepped windows images exist.

virt-install or virt-builder.

I have some extremely clunky 50 line shell scripts around virt-install presently but was hoping to find something a little more streamlined (like the usual Vagrant or docker-machine workflow).

Do you just manually run a huge virt-install command each time?

Have you looked into Weaveworks Ignite?

https://github.com/weaveworks/ignite

I usually use virt-builder actually. But it's like 3-4 command line options for virt-install, not 50.