Could you elaborate on how you use Nginx as a service discovery load balancer?

I think most of the ideas are presented here: https://github.com/martensson/nixy

In essence there is a wrapper that:

- reads service information and current backend instances out of service catalog (either Marathon or Consul)

- writes nginx configuration

- runs nginx configtest

- tells nginx to reload its configuration

Nixy referenced above takes service information directly from Marathon event stream but I usually have Consul so I use homebaked consul-template + nginx solution.

>> where frontend/backend config can change as often as every few seconds

That's a red flag. Don't make architectures where you depend on crazy requirements like that.

Ok, I will go tell the 200 or so developers that they can only deploy to test/staging once an hour or so instead of, like .. whenever they need.

This is a PaaS supporting many micro-services. There is a hold-off to not reload more than once a second just as an insurance against runaway processes but beyond that nginx handles this without any sweat.

Frequent reloads are currently in the non-prod cluster only but I don't see any reason why it would be an issue in prod as well.

Depends what you handle. Some services really don't like being forcefully disconnected and not everything is a short lived HTTP(S) request.

P.S. I am in finance, assume my requirements are severely more extreme than yours. Just like my errors.

That's why we switched to nginx as it is perfectly fine reloading configuration without dropping existing connections. HAproxy can not do that. There is a black magic approach to having two HAproxies listening on the same port so there would always be something listening and connections could be drained, but it's dicey.