The tradeoff here is to decouple services in order to allow them to be developed somewhat independently of each other. Monoliths remove the overhead of network requests but they present their own challenges. You have a lot of implicit dependencies and feature development becomes complicated with changes having unexpected effects very far from the source.
Ultimately engineering organizations need to decide the model that works best for them. Neither is inherently better, they’re solving different problems.
You don't need a service mesh for that, though? Heck, you don't even need an ingress for service to service traffic. ingress-nginx does the job well, without being overly complex, and most importantly to me, logs when something is wrong, which I cannot say the same for Istio which I was fighting earlier this week where it was just happily RST'ing a connection and saying nothing about why it was deciding to do that.
1. I can have a unified set of metrics for all services, regardless of language/platform or how diligent the team is at instrumenting their apps.
2. I can guarantee zero trust with mTLS, without having to rely on application teams dealing with HTTPs or certificates.
3. I can implement automation around canary releases without much lift from dev teams. Other projects leverage these capabilities and do it for you as well.
4. I can get the equivalent of tcpdump for a pod pretty easily which I can use to help app teams debug issues.
5. I can improve app reliability with automatic retries and timeouts.
Probably some other things as well... That said, it can be a big increase in complexity to your system the pains of which aren't always distributed to the folks getting the benefits.
But they're only going to be coarse metrics, like what requests/second. You're still going to be needing application-specific metrics.
> 2. I can guarantee zero trust with mTLS, without having to rely on application teams dealing with HTTPs or certificates.
I do like the idea of this feature of service meshes. It is a slog to get teams to do this responsibly. But, like I said: fighting Istio to understand why it was RST-ing a connection, for no apparent reason. Not logging errors is worse. Perhaps the idea is sound, but the implementation leaves one desiring more.
I should mention the same Istio service mesh above is a SPoF in the cluster it runs in, on account of being a single pod. I can't tell if the people who set it up were clueless, or if that's the default. I suspect probably the latter.
> 3., 4., and 5., as well as actually using mTLS in 2.
TBH, these are just benefits I've never been able to realize. I'm stuck slogging through the swamp of service mesh marketing and the people who want to bring the light of their savior the service mesh but without actually getting their hands dirty doing the work of deploying it.
The fact that you need other metrics does not substract from OPs original point. It's still good and much better overall to handle a set of comprehensive metrics at infra level than to orchestrate every app.
About the coarsness, i think it's not really true. Proxies are freaking powerful and they do a lot of stuff at l7, too much in fact (look at envoy, jesus christ). That's one of the reasons why despite the insane complexity of service meshes, they are paramount for observability.
ok, but what's your threat model for this? great you can tie service versions to each other, but they are just proxies.
2. I always wonder whether that's a timing thing vs. NetworkPolicies and encrypted inter-node traffic. Are the realistic attack scenarios where it's possible to read out intra-cluster traffic but not mess with the cluster, or even read the intra-pod traffic?
3. I've been quite disappointed with how little k8s provides here. I wish it was easier to move traffic off of an old version and only shut the pod down once the last connection was done :/ Maybe I need to look into a service mesh for that?
4. What's the difference to e.g. kubeshark, or just attaching a tcpdump debugcontainer to the pod? Another instance of first to market / potentially nicer ecosystem?
5. I get squeemish with infra-level activities like this. Yes, technically the http method and some headers should make it obvious whether that's save or might break at-least/at-most once or similar semantics. But that requires well behaved applications. While the premise here is infra imposing behaviour to allow applications to be looser around these kind of things.
But yeah I forgot this existed.
I have seen, developed, designed, managed, deployed, operated, fondled, and otherwise been around thousands of large systems that are anything between trivial importance to “must always be running, in the national interest”. I was around when SOAs were a hot new thing, and SOAP was being rumoured as the thing that was going to save us from everything. A fondly recall an overpaid Compaq consultant talking about “token passing systems” when they were describing message queues.
I have seen exactly two systems that really had to be designed and built along a microservice architecture. Both of these had requirements that introduces a scale, scope, and complexity you simply don’t see very often. All other microservice architectures didn’t solve for requirements, they solved for organisational inefficiencies, misalignments, mismanagement, and - in no small part - ego.
When discussing this topic, proponents of microservice proponents talk about many of the advantages these architectures bring, and they are often not completely wrong. What is lacking from these discussions is often a sense of perspective. “Is this solving real problems we have?”, “What is the compound lifecycle cost of this approach?”, and, of course, “How much work is involved in displaying the users’ birthday date in the settings page?”[1], and “when will Omega Star get their fucking shit together?!”[2].
Don’t start with microservices as the default. I will typically work out the monolithic approach as a point of departure. Want microservices? I’m open to that, just demonstrate how that will be better.
Be developed, maintained, and operated by the real organization that exists and not an ideal organization which doesn't is, in fact, usually a practical if not a theoretical requirement, and its usually easier to adapt architecture than to adapt organization.
I don't really understand the question you're asking, but I think maybe the answer is that network pipes are just bigger than the scale most people are operating at? I don't think anything I've ever done has really had that many qps, and if it has, it is more likely to raise an eyebrow that says "who's spamming requests" more that "I guess we've made it to the big time".
REST & protobufs are orthogonal. Empirically, literally nobody is doing REST, and most things are just ad hoc, poorly to not-at-all defined JSON/HTTP with a few HTTP verbs sprinkled in to make everyone feel good. It could be protobuf, too, if you like, but unless you have some truly gargantuan JSON, it really won't matter in the end. Compression will make up enough of the difference in size on the network. Some languages don't have to allocate the keys a billion times, too, though even in Python, it's a while before it starts to hurt.
What I see more of is processes just inexplicably using gigabytes upon gigabytes of RAM, burning through whole years of CPU time for no particular reason before just dropping back to nominal levels like nothing happened, and dev teams that can't coherently understand the disconnect between just how much power a modern machine has, and what their design doc says their process is supposed to do (hint: something that shouldn't take that many resources).
I like microservices, but there should be strong areas of responsibility to them. For most companies, I think that's ~2–3 services. At my current company, it's ~2 services + a database, with the rest being things like cron jobs or really small services that are just various glue or infra tooling.
Our DevOps team starts off the monthly all hands meeting by leading a ritual during which they ceremoniously sacrifice an animal from the Fish and Wildlife Service's Threatened & Endangered Species list while the rest of the company chants:
Exorcizamus te, omnis immundus spiritus
omnis satanica potestas, omnis incursio
infernalis adversarii, omnis legio,
omnis congregatio et secta diabolica.-- Anthony DeBoer
"SCSI is *not* magic. There are *fundamental* *technical* *reasons* why you have to sacrifice a young goat to your SCSI chain every now and then."
-- John F. Woods
We run 1800 services in our mesh. Most of them rest/http, a select few gRPC and graphql. We even have some soap/http services in it.
Or are the services just cogs in the machine, managed by individual devs / cogs in the machine? I imagine the latter and presume that's the main benefit of so many services, but genuinely curious as I've never experience that many services in an architecture
A well functioning service mesh (or even just a well maintained and discoverable ingress controller) is essentially invisible to the individual dev teams. Just think how modern stacks work from a frontend dev's perspective: team wants to use an additional feature, so they find the budgeted credit card, sign up to a random third-party provider, get their access token, and go. From the codebase standpoint, they merely added a new roundtrip to a random service and process the responses in their code.
From the dev team's point of view, having the same feature available internally, behind "just another URL", makes no big difference. Maybe less politics around vendor spend and, with luck, easier integration with the remote service auth. Almost certainly less wrangling with compliance and legal.
Whether that URL is provided by an ingress with a proper FQDN, or a service mesh entry with otherwise unresolvable name, is (and should be) irrelevant.
Modern distributed systems have long since become too large for any single person to fully comprehend them through and through. There is no Grand Design[tm], they are all results of organic changes and evolution. Service discovery and routing can be architected. Individual services within the system can be architected. The complete system where hundreds or even thousands of services interact can not.
Plus, applications built on an external database (e.g., postgres not sqlite) will already have 1 hop. Going from 1 to 2 hops is less dramatic than going from no hops to 1. And I guess implicitly any service sending lots of data to the client already has 1 hop.
Imo the only case where a network proxy would be egregious is an in memory database kind of workload where the response size is small relative to the size of the data accessed (maybe something like a custom analytics engine), but that's pretty niche.
Complexity is very, very expensive.
Most organizations will run their services spread across multiple data centers / AZs.
You check in configs into the monorepo and there is tooling to continuously sync the configs with the actual state of the infrastructure.
The advantage of microservices at scale is that team X breaking the build doesn’t affect team Y. This scale is probably not until you have 1000+ engineers however.
On larger projects:
a) It is sometimes impossible to run the entire application on a laptop. And so having a micro-service means you can quickly iterate and test before embedding it into the wider system.
b) You will commonly run into conflicting transitive dependencies which you simply can't work around. Classic example being Spark on the JVM which brings in years old Hadoop libraries.
c) The inter-relationships between component can become so complex that you really want to be able to use canary or green/blue deployment techniques to reduce risk.
Good architects will know how and when to do what E.g. start with a modulith instead of a monolith, to ease refactoring into micro-services once user count goes through the roof and vertical scaling won't do it any more.
I hesitate to tell you how many layers are between me an HN's servers right now.
How much traffic would be generated if, for every request to HN, six other requests fire? And every time you comment a cascade of requests fire in HNs imaginary K8s cluster?
It’s not that there is anything wrong with this, or that the tradeoff isn’t worth it… it’s just so much data flying back and forth over the wire.
Microservice architecture means there's lot of data flying around, but it keeps local resource utilization predictable.
A mesh doesn't add any extra data back and forth over a wire.
It adds some data being copied from one place in RAM to another place in RAM.
I find that microservices and this type of architecture have become a religion - you do it this way because you do it this way. You add another layer of complication because that's what you do now. You add this product because that's what you do now. Now you do it this way. Now you stop doing this thing and do this thing instead. It's all proclamations and a truly insane level of complexity and often a truly stunningly low level of performance achieved from some very powerful hardware because everything is behind at least twenty layers of abstraction and you're like, encrypting traffic which is just being passed between VMs which are on the same hardware, but because you can't guarantee that they're always on the same hardware you have to encrypt and use a proxy and... oh wow
Watching it from the outside is a bit exhausting, it just seems to be so much churn and overhead.
In my thinking, we might as well get used to the levels of indirection and complexity that AIs will be comfortable with. I suspect it will be more akin to what biological computation is "comfortable" with, and less like what our minds happen to prefer
But I digress :) yes, microservices strike me as wiiild
The companies that need service meshes couldn't possibly run everything in a single monolith. They already have several if not dozens of monoliths, each one typically coming into the architecture when a large enterprise acquires another company and its monolith, or simply different business units / product lines that don't talk to each other because of sheer organizational scale.
It's called a service mesh, not a microservice mesh. First you get the benefits wrapping, securing, and monitoring each of your monoliths, then you reap more benefit when you start to break up each of those monoliths so common concerns can be addressed by common services and provide a unified experience to the enterprise's customers/users.
How is it better have service A request to a proxy, which requests to another proxy, which requests to service B? I get the security benefits of that, but the network architecture is boggling. How many PBs of data are sent each day for what could be a monolithic service?
Actually, to the end — companies that embrace microservices, which see the value in them. How do they manage network traffic for these kinds of K8s setups at scale? Surely they’re not using REST and HTTP. Is it as simple as protobufs over HTTP? Quic? Something else?
Edit: lots of great discussion below but I really meant how do they manage network TRAFFIC, not microservices in general :)