> Sounds like it requires a lot of cross-talk on my network, eating up bandwidth. That doesn't sound lightweight.
Why does that require eating bandwidth? It's a proxy. It can derive statistics from past and current connections and make decisions based on that data for new ones without talking to anyone else.
> Maybe I'm just an old fogie, or being dense, but I still don't see how this solves any problems I can't and aren't already solving.
So? If you're happy with your solution, keep it! But maybe others haven't solved all these problems, or don't like your approach to how you've solved them for other reasons or don't even want to spend time solving these problems in the first place. For many a component like this means it's 10 less things to deal with and more time to focus on building features that'll matter to their end users.
Besides, just because it can already be solved doesn't mean it's not worth trying out other approaches.
Wouldn't those stats get out of date really quickly if you're not constantly talking to that machine?
> For many a component like this means it's 10 less things to deal with and more time to focus on building features that'll matter to their end users.
But now you have a new component that you need to test and get to know. You didn't get rid of those 10 things, you just compartmentalized them in something else that needs to be maintained.
> Besides, just because it can already be solved doesn't mean it's not worth trying out other approaches.
Very true, I just don't see the particular advantages to using more basic and well-known tools like DNS and TLS.
Many companies, including mine, already run an internal load-balancer (such as HAProxy) that takes care of routing traffic between microservices. Linkerd replaces this with something that is explicitly designed to deal with inter-app communication. (I believe one way to run Linkerd is as a small sidecar app inside the same Docker container the man app, as opposed to a big central server, which is also a neat way of doing it.)
Re "why not a library" — sure, you can build all sorts of discovery, distribution and resilience into the client itself. But that consolidates all of this intelligence into just that client. What if you use multiple languages? What about third-party client libs (e.g. Memcached, Redis, PostgreSQL) that you don't control and that you don't want to reimplement? You'll find yourself reimplementing the same thing over and over.
> you can't "drop in" resiliency
I recommend reading the documentation, because drop-in resilience is really the whole point. The guys who built Linkerd previously developed Finagle [1], which is Twitter's RPC library, and when they talk about network resilience, they actually know what they're talking about [2].
[1] https://blog.twitter.com/2011/finagle-a-protocol-agnostic-rp...
That's just DNS.
> Many companies, including mine, already run an internal load-balancer (such as HAProxy) that takes care of routing traffic between microservices.
Why not DNS round-robin?
> I recommend reading the documentation, because drop-in resilience is really the whole point.
Fundamentally, the application itself needs to be resilient. No proxy will make it so. It could help, but it can't do it.
That's not just DNS. Well, it might be - it depends on your environment.
For example: In a world where you have some kind of containerization, or scheduler responsible for deploying your services you typically to have services (or different instances of your services), running on ports determined at runtime. You might solve this with an off-host load balancer, so your frontend applications don't need to worry about it, but now you need to worry about those other things: circuit breaking, retry mechanisms, etc between your application and its middle-tear load balancer.
A common deployment strategy for something like linkerd (or Envoy) would be as a sidecar process, on the same host where you don't need to worry about the reliability of the network to talk to a remote service, because something else is taking care of 90% of your concerns for you.
> Why not DNS round-robin?
There's functionality offered by load balancing at L7 that you can't do with round-robin DNS. You end up having to push that complexity into each service you develop - retry mechanisms, circuit breaking, and routing decisions if you're going to try do green/blue or canary deployments.
You might say "solve this with a library", but the reality is you have to reinvent the wheel, you have to keep that library in sync to ensure the behaviour is consistent, and if you're in an organization that develops services in "N" languages, that challenge can be even greater.
> Fundamentally, the application itself needs to be resilient. No proxy will make it so. It could help, but it can't do it.
Absolutely. Your services still need to be able to fail gracefully, but this just moves the complexity of a whole host of other issues to a single, and common spot.
In short, another problem introduced by containers and not relevant to non-containers. Why can't a container get its own IP and port like everything else? FreeBSD jails do. Why make it opaque and put the "ingress" component inside of k8s or Docker? It'd be reasonable for k8s to act as a DHCP/DNS server, but not to swallow the whole world.
No, it's a bit more complicated than that, as chrisboulton explains (so I won't go into detail).
You could use a dynamic DNS server such as Consul for discovery, and people do. Last I checked, you still had to fight eagerly caching clients, but that's probably doable. But there's still a host of other issues that Linkerd addresses.
Again, I strongly recommend reading about Finagle to understand what this is trying to solve.
Another thing Linkerd brings to the mix is instrumentation and tracing. Instead of building this into every single sender and receiver, Linkerd can collect tracing data for you and make it available for other subsystems to ingest, such as Zipkin or Prometheus.
Round-robin may be sufficient for your needs. However, the load balancing techniques that linkerd employs solved real production issues for us. These techniques allowed us to be paged less frequently for minor errors and we could more reliably make changes to the system without causing user-facing issues.
See https://blog.buoyant.io/2016/03/16/beyond-round-robin-load-b...
Unless you're constantly talking to a service, your stats will get old quickly. It could also cause everyone to target a specific host creating problems.
I'm still not sure what service discovery is required beyond knowing the name of a service.
Linkerd doesn't free me from anything. I still need to know how it's done, how it's configured, and how to fix it when it breaks.
What linkerd buys you is you don't have to write this type k8s-aware, zipkin logging library for every language you're running in production. But I think it's straddling a very narrow section; small users shouldn't care about this and just rely on the round robin, zipkin is a PITA to run anyway. Large users will probably want to write their own libraries (zipkin is a lot better if you put traces in your process)
OK, A/AAAA are more common.
> your networking layer should be k8s aware and issue a watch command on Endpoints;
Or, I could use standard networking concepts and not build my entire app to suit one new thing that I may not want to stay with forever, or even more than just test.
> What linkerd buys you is you don't have to write this type k8s-aware, zipkin logging library for every language you're running in production
There are already TCP/IP, DNS, DHCP, &c libraries for all languages, and that aren't tied to a specific stack. Why should I build into this stack and not into a more common subset?
Even if you pick all your impls carefully, you have to wait for your TTL instead of a push mechanism for changes.
If you wanted to implement the watch as a library, code wise, it's maybe ~500 lines per language, using the k8s client lib. I could hammer it out in two days.
One component we're interested in is its "automatic" tracing. It claims to be able to annotate requests with Zipkin stats without requiring any instrumentation at the software level.
You are correct that other solutions exist for service discovery, but that's an in-demand feature right now as well, and afaik the main reason that people use linkerd.
>Maybe I'm just an old fogie, or being dense, but I still don't see how this solves any problems I can't and aren't already solving.
Quick, turn back now from this whole containerization thing, because I assure you this is basically the answer for everything in this space. And you will be aggressively accused of being an old fogie for resisting it. Simplicity is just not appreciated anymore; at this point, "containers" may as well be an antonym for it (and by extension, an antonym for "sane engineering").
Why not use SRV records and TLS? Retries, expiration, timeouts, and back-offs can all be placed into the networking library, and are, commonly.
Also, regarding https://linkerd.io/, what's with landing pages having almost no useful information on them. "linker∙d is a transparent proxy that adds service discovery, routing, failure handling, and visibility to modern software applications" what's that even mean?
> Fast, lightweight, and performant > Handles tens of thousands of requests per second per instance with minimal latency overhead. Scales horizontally with ease.
Well, since you're calling yourself "transparent" I would hope it'd be fast!
> Any language, any environment > Runs as a transparent proxy alongside existing applications, integrates with existing infrastructure.
So, it's just proxying and retrying things and doing tls for me? Why isn't this better handled in the library I'm using to make the call in the first place?
> Latency-aware load-balancing > Balances request traffic using real-time performance, reducing tail latencies across your application.
Sounds like it requires a lot of cross-talk on my network, eating up bandwidth. That doesn't sound lightweight.
> Runtime traffic routing > Provides dynamic, scoped, logical routing rules, enabling blue-green deployments, staging, canarying, failover, and more.
OK, some of that does sound useful, but some of it still feels like something the caller should handle.
> Drop-in service discovery > Integrates with most service discovery systems, decoupling applications from specific implementations.
What's this mean? Why not just use SRV records?
> Production-tested and proven at scale > Powers the production infrastructure of banks, artificial intelligence companies, social networks, government labs, and more.
I know why this is here, but it feels like _everyone_ claims the same thing, so, to me, it just becomes noise.
> "linkerd allows you to drop in a transparent layer of application resilience and provides the operational affordances critical for modern, cloud native environments." > —Oliver Gould, CTO, Buoyant
This sounds doesn't make sense to me, you can't "drop in" resiliency. It needs to be built. The whole thing just feels like it's all marketing and doesn't tell me exactly what problems it's solving for me as a developer.
------------
Maybe I'm just an old fogie, or being dense, but I still don't see how this solves any problems I can't and aren't already solving.