NodePort is a bad practice for a such case, you are right. You get additional unnecessary routing on CNI level between the cluster nodes just to get traffic into the Ingress controller pod, for example. The solution here is to use hostNetwork mode with the pool of workers solely dedicated to scheduling Ingress controllers. Traffic directly hits NGINX/Envoy/HAProxy/Traefik/whatever and gets into the cluster without additional intermediates. You need a load balancing solution for this pool of Ingress controllers, that's right, but as I said before, this setup gives you the flexibility to cook load balancing as you desire.

BTW, community NGINX Ingress controller is able to ingress L3 (TCP) traffic into the cluster.

> You need a load balancing solution for this pool of Ingress controllers, that's right, but as I said before, this setup gives you the flexibility to cook load balancing as you desire.

Sure, but this means that you cannot use LoadBalancers, which is painful. It means every payload has to be configured both at k8s level and then externally. That somewhat defeats the use of k8s as a self-service platform internally in an organization (other dev/ops teams need to go through a centralized ops channel to get traffic ingressing into the cluster if for some reason they can't use an Ingress).

> BTW, community NGINX Ingress controller is able to ingress L3 (TCP) traffic into the cluster.

Yes, but it's configurable via a single ConfigMap (which limits self-service if you're running a multi-tenant orga-wide cluster, unless you bring your own automation), and still you only have one namespace of ports, ie. a single external address for all ports - unless you complicate your 'external' LB system even further.

With all these caveats, I really don't understand why not just run metallb.