Another advantage is that you can run more up to date packages than your distro would allow, or different versions of the same one.

The downside is that you should rebuild the containers daily to be sure to have all the security patches. Not as convenient as apt-get. Maybe it's more cost effective to run another VPS or two.