You are looking it as a network level filtering.

I am looking at, make private network, and then you have to explicitly add an gateway (usually a load balancer) that can access it.

In short don't design system where you need to filter, design the system, where you need to take explicite action, to make something public.

This is easily, done on something like AWS.

The very concept of a "private network" leads you down the path of making a security boundary that has far too large a surface area, IME. Either it becomes a big bag with all your hosts in it, and you write systems that trust all the requests they get (even if you know you "shouldn't") because you know only your systems are on the network, and then an attacker figures out how to make one of your systems make an arbitrary request and you get owned. Or you put each component on its own private network but then you have to open up every port you're actually listening on so that your components can talk to each other (and you probably automate that in your kubernetes/puppet/whatever setup, like what happened here with docker) and the private network does nothing.

An independent firewall accomplishes the same even if every server has a public IP.