I can even add that we only do web software, but we do them exactly that way.
>If anything, it adds more complexity because now we need to do all kinds of data marshaling, error checking, monitoring, have more infrastructure for something that should have been done in shared memory/in-process to begin with.
I couldn't agree more. Martin Fowler warned us a long time ago : "The first rule of distributed objects : don't distribute them".
>There are a few exceptions where its useful to scale out parts of a distributed application
Yes, and very very very few. As I always say : microservices are not an architecture, they are an optimization.
I can even add that not only do we do them exactly that way, but it's many times faster and more reliable than microservices, for reasons I have written about before[1].
The reality is that microservices add a whole host of failure modes, for just one benefit over monoliths: They are easier to scale horizontally. If you don't absolutely need horizontal scalability, and not many companies do, then you'd be foolish to pay the large cost.
If you are writing and deploying standalone applications, I agree. Microservices do have another benefit, though: forcing the application to be modularized. There are better ways to do that, though, like having proper systems engineering lay out the application architecture.
If you are writing applications that need to be integrated with third-party applications or need to be configured with different functionality for different customers, microservices can still make sense. This is what has pushed my company in that direction. We currently have a set of monolithic applications with varying but overlapping functionality, and customers who want feature X from application Y and feature W from application Z, plus feature V that we are developing just for them. Our code has become a mess of curricular dependencies and duplicated code in order to meet these desires.
The microservice (maybe not micro, but definitely service) architecture we are moving to will allow us to compose applications with reusable building blocks more easily than trying to maintain different git branches and library versions. It also allows us to better manage the non-code resources we need to deploy as we have hundreds of gigs of data resources used by various libraries. Furthermore, we will be able to deploy applications with either our in-house GUI and command line tools, as well as provide the APIs necessary for our customers to integrate our applications into their own third-party frameworks.
Can you elaborate why? Presumably, with microservices, you're always running the latest version, whereas you're mentioning a need for "different branches and versions" in your comment.
Microservices don't force you to modularize, you can still make a mess by not cleanly separating things or by cutting across the wrong boundary. You modularized well because you created a clean, separated, thought-out architecture, not because of any intrinsic property of microservices.
Not necessarily. Depends on the needs of the customer and what we deploy to them. In general, we try to deploy the most recent version of everything when we make a release for a particular customer.
The issue we have with our builds is dependency management. As a note below, that in itself is something that could be addressed by refactoring our projects[1]. Because of the way we need to deploy our software (shaded uberjars) even a well-structured dependency hierarchy would result in large data updates every time we had to update the monolithic application.
> Microservices don't force you to modularize, you can still make a mess by not cleanly separating things or by cutting across the wrong boundary. You modularized well because you created a clean, separated, thought-out architecture, not because of any intrinsic property of microservices.
They don't force you to modularize well. but they force you to think about how to break your application up. You still need the systems engineering to break it up well.
You're right that services aren't the only way or necessarily the best way to accomplish this. We could get the modularization just by refactoring the organization of out projects. In the absence of the requirements for third-party integration and an actual need for horizontal scalability, that is what I would have done. All-totaled, though, I think the service paradigm was the right way to go for us.
I need to note for context that the software we write and deliver needs to be delivered on physical disks (yes, we still literally ship software) and that deployment size is an issue. We also can't guarantee that we will have access to anything besides a Windows or Linux JVM, so our deployment needs to drag all of its supporting data along with it. being able to simply deploy a new version of a particular service makes updates much less painful.
[1] And actually using versions for our Maven artifacts, but that is a battle I lost a long time ago...
Funny that now the same person carries the flag with the symbol of microservices, which are seen as a way to "get rid of architects". That's exactly what Thoughtworks believes in - and you will understand more if you read the first chapters of the book "Building Microservices", or if you join their recent conferences. Just my 2 cents.
https://en.wikipedia.org/wiki/Fallacies_of_distributed_compu...
I once work for a giant fortune 500 corporation. Our department had a good 300 developers, and had been writing code for years. They had built the whose system using the classic, box-box-cylinder architecture. There were hundreds of little top tier 'services', but in practice, they all shared the same database, had to be deployed pretty much at the same time, and version upgrades had to go in lockstep. Every so often, database changes were necessary, and the world would grind to a halt for months to make the system still work after the migration: It was awful.
On top of this, having everyone using one stack really meant that a council of elders got to make tech decisions, and everyone else just did what they were told. This built a giant hierarchy. People near the top would never leave, because nowhere else would give them that much power. Talented developers would take years to gain influence, so they often just left. What remained was developers with no ambitions other than getting a paycheck... it was horrible.
The alternative was to let people maintain their own stacks, as long as they provided interfaces that other people could call. By limiting how much code could talk to a database, you didn't need negotiations to change something: Teams made any change they wanted as long as they remained backwards compatible, and then had to lobby users to upgrade to newest API versions if they wanted to remove the backwards compatibility mess. It was awesome in comparison.
A gigantic place won't have those problems, because they can invest money on making whatever tech decisions they made tenable: PHP is slow? Let's build a new runtime and compiler, says Facebook! If you are tiny, you don't need any of this BS, because if your team of 8 engineers can't agree, your company will fail regardless. But when you have 200 engineers, it's giving people more control over a piece of the pie or bleeding talent.
The one thing you still need to do is make sure teams have the right size, and people have enough accountability, that the product still works. You also need silly amounts of data marshaling and error checking compared to the monolith. But the terror of a company that can't hire engineers because the only way to have any control over your daily life is to have been there for 5 years is just hard to compare. When people say they don't want to work big corporate gigs, what they really mean is that monoliths become soul sucking.
So yes, I give thumbs up to fiefdoms, just like I'd rather have inefficiency in a republic vs a theoretically efficient dictatorship.
But there's now a whole bunch of new developers out there who are just starting out and read all about these microservices, and think that it's the way to write web backends, and that if you don't do it that way, you just don't know what you're doing. Even if you're a two person team - you must use microservices if you want to scale out in the future apparently. It's just unfortunate the amount of cargo culting going on.
Until it doesn't. And that change will come and it'll make your fancy architecture look like a tin shack instead of a stately manor.
These devs are loud. And management loves loud. Loud is confidence.
We need a Dilbert for developers. Truly.
I give the proponents/consultants credit for proposing a practical and incremental solution to these people instead of just insulting them or saying "you're doing it wrong".
That being said, the issue I have with microservices is the fact that the below is repeatedly used as a core technical argument :
> They had built the whose system using the classic, box-box-cylinder architecture. There were hundreds of little top tier 'services', but in practice, they all shared the same database, had to be deployed pretty much at the same time, and version upgrades had to go in lockstep. Every so often, database changes were necessary, and the world would grind to a halt for months to make the system still work after the migration: It was awful.
This is not a feature of "monoliths", this is a feature of shortsighted design (at the beginning) and cowboy design (at the end).
Let's be clear : apart for load balancing for performance critical areas (which only exist in 20% of the applications, and only cover 1% of a typical such application - or 60% if you are Google, Facebook, or Amazon or the rest of the top 10), there's nothing that microservices do better than in-memory libraries. Also, centralizing everything as opposed to having processes run on various servers creates the issue of the single point of a failure. Whole new classes of similar issues appear. It basically is buying troubles.
Adding microservices will never be a neutral operation, and incurs a real cost. This cost is the cost of distributed objects which has been known for over 20 years and has been well-documented by our fathers.
Once the above has been dismantled, it then comes to : "yes but teams can develop in different languages", "yes but dependency management becomes easier". But once again, when it comes these interoperability considerations, a solution also exists and has been created about 15 years ago. This solution is called .NET (30+ languages compiling to a single Intermediate Language, COM components, first-class library support, etc...). Even the "yes but I need to communicate with non .NET in-memory applications" is very maturely addressed by the framework. (it was one of the first-use cases addressed)
I also read in this thread : "the ability of teams to control their own destiny and not be blocked by other teams". With libraries teams can choose the version that works best for them, and a well-design in-house library would provide the extension points (Dependency Injection, Dynamic Binding...) that would allow the clients (the other teams) to specialize the behavior.
So in a gist, my main reproach for this, is that if people put the same effort they put in learning microservices to learning system and component design, they wouldn't even be a need to talk about the former. My regret is that people will always choose the hype and the new. But hey, that's how the game works.
Although unless Oracle changes its way it seems we will soon be a point were Apache will port Tomcat etc to .Net and we can use the best language with the best servers from the best communities.
— M. Conway
First, there's a difference between "I've seen it done like X" and "When done well, it's done like Y"
Too often we play this game where we talk, teach, and apply Y, but then in the real world it gets done like X. Turns out that X sucks a lot, so then we throw away Y.
Microservices may be done poorly in most real world applications. In fact it would surprise me if they weren't.
This article doesn't help much. Microservices are not just another version of SOA. It doesn't work that way. In SOA, you start with a general category of service, say "logging". You write it simple, yet broad, and it's supposed to handle all folks that need logging. In microservices, you're doing one tiny little thing, like "make a record of the errors of this process available on the web". The vast majority of times, if you define what you're doing narrowly enough? The O/S already does it. Whereas if you start broad? You're writing code.
Then you slowly and methodically expand on that mission a little bit at a time, refactoring your microservices as you go. It's both a different way of structuring apps from monolithic days and a different way of looking at constructing and maintaining apps. If you think of it as the same blob of binary bits broken into smaller pieces, you've missed it. Likewise if you think of it in terms of "services". The "micro" is the key word here, not the "services" part.
This actually requires a much heavier interaction between developers, not setting up fiefdoms. If done correctly, it pushes larger groups of developers across teams to work more tightly together. If it's doing something else? You're doing it wrong.
To use your example, if my task were "make a record of the errors of this process available on the web", my first thought would be to write to stderr, pipe stderr to a file, and then serve that from the webroot of some commodity webserver like Apache, 'python -m SimpleHTTPServer', or even S3. If the format is too hard to read, I'd probably write a quick JS viewer that can collapse stack traces, aggregate similar errors, etc, all done in the web browser. Or use cut/awk/sed to process it before dumping it in the webroot. It's only if the data gets big enough that it starts freezing the browser that I'd reach for ELK or something similar. I don't know anyone who would call this a "service" though; indeed, it seems the polar opposite of what most people talk about when they say "microservices".
I agree with your characterization. All except maybe the "I'd probably write a quick JS viewer that can collapse stack traces" part.
Here's the thing: I'm so minimalist that I'm only going to add frameworks and libraries as a last result, if you stuck a gun to my head. I'm going to resist as much as I possibly can.
I love writing code. And every line of code I write has to be compiled, deployed, is prone to bugs, has version dependencies, and so on. Many times it adds unnecessary complexity to the solution. I love using cool frameworks and libraries. Same goes for them.
The magic here is that the vast majority of whatever you're doing? It's already been done. Use O/S code that's ran successfully for ten, twenty years. Write your own code only as a last result. Micrososervices shouldn't have more than 100 lines of code.
Yes, it's a function. Maybe 3 or 4 closely-related functions. If it gets past that level of complexity, refactor.
(The only thing I'd add here is that you'll end up factoring out common code as you go along. That code needs to be put into a shared corporate library, which also should be small, along with shared types)
When done well, you end up with the realization that you've been doing it wrong all along. When done poorly, you just make an even bigger mess than you did doing it the old way.
Here's one small example. I was writing a production project for some folks a few years ago. We needed some way of showing system status.
The old way could have used all sorts of things: logging systems, server metric systems, and so on. I would have probably done some research, downloaded a library or framework, set it up along the happy path, and made some really awesome html dashboards.
Then the library would change, or the metrics would be different, or the configuration would need tweaking, etc.
I figured out what we really needed to see was time/date stamps on certain files. So I pipe ls to a web file and chron it. Five minutes later it's working. It'll be working ten years from now. If we need something more complicated? We'll add it. One tiny little bit at a time.
I'm okay with calling that a microservice, since it's infrastructure code that needs to be deployed and maintained as part of the app. But it's really only a line of BASH.
Some people would say that what I did was completely trivial. These are people who do not understand what the hell is going on. That was a win. If I could have 5 or 10 microservices like that which provide business value? I'm freaking way ahead of the value curve as opposed to a similar team who's still slogging through 7,000 lines of mostly boilerplate code that does basically the same thing.
However, you are correct. Most people are unable to get the cruft out of their head, so when they talk about microservices, they mean something like "exactly the way we used to do things, only with a few differences"
I was talking to a team last year that wanted to do microservices. They had a framework, a lot of tools, and were busy doing research, most of which consisted of reading fanboy posts and walking through happy-path tutorials.
When I started listening to what they were talking about in terms of an architecture? Wow! Too much to go into here, but one of the things was a "microservice" that at the beginning of the pipeline cleaned all the data.
Think about that. One "service" which was responsible for knowing every data type, performing all safety checks, and ensuring that all of the other services could run correctly. All in one place. You screw that one up? The whole thing is broken. You couldn't ask for tighter coupling.
And this, they thought, was better than a monolithic app.
I'm a big fan, but micoservices are going to make a huge mess in the industry that it will take decades to clean up. That's because most people doing them will think of them just as you suggested.
(Having said that, the ones who don't are going to run over everybody else in terms of productivity. Good for them and the orgs they work for)
I'm under the impression that most people wouldn't call that a microsevice. Moreover, isn't that solution completely dependent on the implementation of the code that is writing those certain files, without making this dependence explicit - and pretty difficult to test automatically?
I think the key thing here is your word "explicit". There are lots of ways to make things explicit. The concerns of coupling and cohesion don't go anywhere. There are other ways to address them.
Testing is actually significantly easier, not more difficult.
> Like in SOA, services in a microservice architecture[1] are processes that communicate with each other over the network in order to fulfill a goal. Also, like in SOA, these services use technology agnostic protocols.[2]
I'd really like to see a non-trivial example of this because it's sounds ludicrous on its face
There are legitimate reasons why having those fiefdoms is beneficial despite the overhead. Especially in a SaaS world where you are not distributing binaries. Being able to use different languages for different purposes, and to deploy them to different hardware can be quite a bit more efficient in terms of hardware costs at scale. Also there is something to be said for "you write it, you run it" as it makes developers more careful and decreases finger-pointing. Again, don't take this as a rebuttal, the overhead is potentially a lot, but if clean interfaces can be determined and makes sense from a high-level business perspective then there can be a net gain.
Even if you hire high-end talent, they will be forced to worship people that have the knowledge about how all the bloated jenga-tower mess works.
And that's only the people problem, let's not even discuss how you build, deploy, test things in a reliable way without taking days to produce a working build.
That, and the fact that your quality of life will suck, since you won't be in control of such mess.
With microservices you have one service that does a limited set of things, that is easy to monitor, maintain, test, deploy and even entirely replace if necessary.
Specifically, like a microservice allows encapsulation of functionality, it also allows encapsulation of blame[1]. In a monolith (from whence the org I'm in came), a build failure or test regression could be caused by any number of failures across any number of horizontal bands in the organization. Oh, the build automation crashed because the build team updated to the latest version of Java but the build didn't. Oh, the UI Filters stopped working because the API team changed something without deprecating. It meant that development, in spite of agile efforts, still had a tick-tock cadence, where breaks halted work and tracking down the responsible parties and getting things fixed might take time (a lot of areas with deep specialties required to understand why something might be wrong). This also meant, because of the way the organization was structured and the way the build was structured, that "pressure" was directed along very hierarchical routes. Managers saw bugs from customers and pressured testers and dev-ops people who maintained automation to investigate causes and transfer responsiblity to developers who might be able to actually fix the problems.
As we've been decomposing into microservices, and likewise aligning along feature teams the blame gets allocated at API/service interfaces[2] instead of top-down. Since the build, deployment, uptime, and algorithmic functionality of each service is theoretically the domain of a single team, the blame-flow is more distributed and simple. An algorithmic bug, a build bug, and an availability issue are all addressed the same way: report the issue to the team responsible for that service, and let them work it out.
I'm not advocating that either way is better. There were nice aspects about a single-location debug tree in the monolith. I've seen teams that have become experts at deflecting blame and thus slow down the entire broader effort. And I know I'm possibly conflating two paradigms inappropriately (Feature Teams and Microservices). Just a notable pattern, to my eyes.
---
[1] I don't necessarily mean 'blame' here in a pejorative sense. Perhaps 'responsibility' would be a more neutral term.
[2] Steve Yegge describes to this being a top priority during Amazon's service/platform decomposition.
> Specifically, like a microservice allows encapsulation of functionality, it also allows encapsulation of blame
I think, the example of Joe's and Sally in parent [1] and your comment, makes the point in micro-services. During various Web Services integrations developers not only may blame specific part of system as faulty but also helps follow rule 'do not test your own code'. I remember few situation in pre-microservices era when during developing new API wrapper developers discovered a flaw in the remote web service (e.g. parameter doesn't change state) non-documented error code or raw Java exception as response. In these cases, owners of the API are are usually notified about issue so we may expect that will be fixed.
I agree with you that is hard way to advocate either way is better. It is worth noting that creating a micro services leads to problem which may end with _duplicating_ (creating monolithic) code to provide possibility to communicate between the hosts (imagine mesh network).
Someone builds a monolith, it gets some traction, and suddenly the server bill gets into five, six figures a month. Management have never been management before, and have never paid anything more than a few hundred dollars a month for servers, so their new cloud bill is mind blowing to them. They're spending as much money as a house costs every month on renting servers. They imagine they're hemmoraging cash right and left, they freak out and order engineering to rearchitect everything around a Service Oriented Architecture, because clearly it's time for the big boy toys. Google does it that way, so clearly that's what one does when one is at scale, when one is successful and has lots of paying customers, right?
99.9% would be better off financially and technically just rewriting a V2 of the monolith using lessons learned from V1, but that's perceived as outmoded, old-world thinking, because it's not what Google does.
Devs not creating good interfaces in OO design? Stick them on a different server, that ought to show them. Of course now your spaghettified object hierarchies are spaghettified api call hierarchies.
* Microservices don't need to be written in the host language, nor do you need to deal with creating wrappers. Granted interacting with kafka/ampq isn't frictionless.
* Microservices can be upgraded or scaled out individually without touching the larger application as a whole.
* Microservice calls can be deferred and scheduled easily.
But, that's kind of the point. They're a networked system of interacting processes, but not a distributed system. A distributed system generally is a tightly coupled and coordinated set of services running on many machines for purposes of resilience or scale-out. Microservices are more for organizational scale-out. A microservices might BE a distributed system, but several interacting microservices probably isn't one (pedantically speaking).
Microservices are an evolution of how large-scale integrated systems have played out as a pendulum swing between tightly coupled systems and loosely coupled systems.
After the mainframe, into the client-server era, most non-web software in a large organization (eg. most IT back offices dating back to the 90s) were NOT built as a single application, it was built as a set of independent applications that were coordinated by something: a shared database, data extracts, message queues, transaction processors and RPCs.
Data extracts were the ultimate in loose coupling. You got an update every night from other databases, and could do whatever you wanted with the data. But of course, this led to data quality problems, when customer records or inventory couldn't reconcile.
So we moved to "a service owns its data" type situations, and wrapped them in message queues or RPCs. This is basically how something like SAP works - it's a set of modules with their own schemas that are tightly coupled through a set of RPCs and messages.
But this eventually had none of the benefits of networked modularity - everything was so coupled and intertwined, it was a mess to change and upgrade - you had to do the whole thing at once (which is why SAP upgrades are so notorious).
What makes microservices different is that we've evolved them to have truly independent interface from implementation by having the implementation lifecycle of a module be tied to the server down to the metal, so it can be completely autonomous from other services. Therefore I can upgrade/replace/scale/move these runtimes whereever I want, whenever I want. I can also place proxy or gateway intermediaries between them more easily. This is "SOA done well", in effect.
We've also evolved our practice of service interface design beyond rigid IDLs and WSDLs and schemas into more permissive, extensible, and better documented APIs or Events, along with easier versioning, but the state of the art still has a long way to go there.
Not a panacea, but on the flip side, I've rarely seen a maintainable, scalable, and easy to change networked monolith. They do exist, but required the controlling designers to be deeply talented with full control over architecture decisions, and they didn't get into analysis paralysis. But even they eventually couldn't keep up with changes, so they're breaking their monoliths up today, IME.
What is a real distributed system?
Sorry...
It has nothing to do with building reliable software. You could just as easily build and deploy a single networked application (so called "monolith"), that is composed of many different libraries that have well defined interfaces which can be tested in isolation. In fact, that's how most non-web software is still written and done.
The real reason is that by having these microservices, it allows single developers or teams to own or control parts of the codebase and enforce their control via separate repo's, and when speaking runtime, via authentication: Sally can't see the code to Joe's service and Joe can't make requests to Sally's production instance of her service that gives a guess at how long the car has to arrive to pick poor end user Bob up.
I've seen this same thing play out countless times at large tech companies and startups alike. It has nothing to do with building scalable, or more maintainable, or more cleverly designed applications. If anything, it adds more complexity because now we need to do all kinds of data marshaling, error checking, monitoring, have more infrastructure for something that should have been done in shared memory/in-process to begin with. Not to mention all the issues and headaches caused by fan out of tons of API requests, complicated caching scenarios, etc. I've seen the horror of microservices architecture where no one person is responsible for the actual app, only their "service".
There are a few exceptions where its useful to scale out parts of a distributed application, but in 99% of my experience the services aren't a real distributed system anyway and are vaguely organized by function, developer interest, and yes, control.