Unless you're dealing with a multi-tenant situation I'm not super convinced that a VM is worth the effort. It's not the perf, it's the need to make your kernel, root file system, and other infra needed to make it all work.

Compare that to a docker container where there's basically 0 additional work that has to be done to be up and running.

For most cases I'd be really tempted to work on hardening the docker container than on setting up a VM. Things like Apparmor and seccomp in particular would likely go a very long way.