“Let’s use Kubernetes.” Now you have eight problems

Re:“Let’s use Kubernetes.” Now you have eight problems by notyourday on Thursday March 05, 2020 @11:46AM (#22495232)

> I think we tried to use that DNS solution a while back and found that the DNS lookups were adding a few milliseconds to each call. Which might not sound like much except we have some endpoints that average 10ms. And with fanout, those milliseconds start to pile up.

Don't resolve DNS inline rather on every DNS update, resolve it and insert new IP addresses.

[ Parent | on HN ]

Re:“Let’s use Kubernetes.” Now you have eight problems by hinkley on Thursday March 05, 2020 @02:59PM (#22497504)

Correct me if I'm wrong, but I believe Consul, lacking a mesh of its own, is leveraging the early 1990's era trick of using round robin DNS to split load over available servers.

Caching those values for very long subverts the point of the feature.

[ Parent | on HN ]

Re:“Let’s use Kubernetes.” Now you have eight problems by jen20 on Friday March 06, 2020 @01:44AM (#22501456)

By way of correction: Consul does not simply "round robin" DNS requests unless you configure it in a particularly naive manner.

Prepared queries [1] and network tomography (which comes from the Serf underpinnings of the non-server agents) [2] allow for a much wider range of topologies just using DNS without requiring proxies (assuming well behaved client software, which is not a given by any stretch).

Furthermore, Consul _does_ have a mesh as around 2 years ago - [3].

You are correct though that long caches subvert much of the benefit.

[1]: https://www.consul.io/api/query.html

[2]: https://www.consul.io/docs/internals/coordinates.html

[3]: https://www.consul.io/docs/connect/index.html

[ Parent | on HN ]

Re:“Let’s use Kubernetes.” Now you have eight problems by notyourday on Thursday March 05, 2020 @04:06PM (#22498258)

Not really - resolve all backend servers to IPs and list all of them as the nginx backends. When a backend server is removed, update nginx backends.

Round-robin balancing using DNS towards a small cluster is silly - you know when any new instance is added to the pool or removed from a pool, so why not push that load balancing onto the load balancer which in your case is nginx?

[ Parent | on HN ]

Re:“Let’s use Kubernetes.” Now you have eight problems by hinkley on Thursday March 05, 2020 @04:31PM (#22498542)

You're talking about layering the thing on top of Consul that I already identified in my top level comment.

Consul itself advertises DNS resolution for service discovery.

[ Parent | on HN ]

Re:“Let’s use Kubernetes.” Now you have eight problems by notyourday on Thursday March 05, 2020 @05:07PM (#22498869)

Maybe I was not clear.

Whatever is the technology that you use to register the active backends in the DNS, rather than doing name => ip address lookup per request, you can resolve all those names => ip address maps upon the service being brought up/taken down and push the resolved map as a set of backends into nginx config, thus removing the need to query DNS per request.

[ Parent | on HN ]