You could also start whichever container it needs (the docker work) yourself and just have the two containers container with eachother (e.g. docker compose, without granting access to docker socket), provided that the docker work is static enough that this would work.

You could also look into user gVisor runtime for extra security.

If not, then I think you're right about full VM.

I've been eyeing Gondolin for a while for that, but haven't yet tried it.