You're correct and absolutely want it all wrapped up in a container (or multiple containers, depending on preferences).

And yes, you want the host as vanilla as possible. Generally just running the orchestration layer (Kubernetes, Swarm, Nomad) and maybe some logging / metrics stuff (and, depending on who you ask, sshd).