From an architectural perspective, there is absolutely no difference between a micro service and a library. The only real difference is in the dispatch mechanism.
The problems around configuration management are the same problems we've had as programmers for decades. It's just that the people who are keen on micro services are usually not old enough to have experienced the pain in a different context.
Should shared libraries be in different repos, or should you put everything in a single repo? How do you deal with versioning? What happens if one app wants version 1 of the library and another app wants version 2? How do you deal with backwards compatibility of the API? Do you make a whole new library when you decide the old API is incompatible with the new vision? Blah, blah, blah, blah. None of this is a new problem.
I can't remember which version of Windows it was (maybe 7?) where they were seriously delayed mainly because they had so many programs using different versions of libraries. Integrating it at the end was apparently complete hell. Since they wanted to have separation of responsibility in their groups, each group was just pounding away implementing the features that they needed, but not integrating as they went -- because that would mean lots of cross team communication. The exact same thing is likely in a large organisation with a ton of micro services.
There isn't just one way to solve the problem. Mono repos and monoliths help in certain ways and cause problems in other ways. There are other techniques as well (should we implement ld.so for micro services? :-) ) But as you mention, the real answer is that the solution requires humans, not technology.
Architecturally speaking, there are some similarities between micro services and libraries because they’re both forms of modularization and usually have an API, but there are some stark differences beyond the “dispatch mechanism”.
The main difference is that a service’s deployment lifecycle is completely up to the service admin. Microservices are like websites - they can continuously evolve (within their API’s contract) without asking permission from consumers. This is their main superpower and why they’re a way of scaling a development organization without slowing it down too much.
In the case of a shared library, it’s completely up to the host admin as to when to upgrade. In the case of a static library, it’s up to the consuming software to determine when to upgrade. A service can upgrade when it feels like it.
Issues of API backwards compatibility, forwards compatibility, extensibility, self descriptiveness, versioning, etc. are old issues but usually have different answers when upgrades are truly happening all the time and not just in theory. It tends towards much fewer hard versions and more evolutionary backwards compatibility.
IOW, microservices aren’t a cure all, but they do encourage a set of behaviors. Many articles detracting from them seem to have not wanted those behaviors in their org in the first place.
Shared libraries can be an implementation of micro services. On a platform (e.g. Android) it can be that a shared library is updated and then all consumers are forced to update.
Services, whether SOA, web services, messaging services, or network services etc, as in SOA, describe a client/server architecture with an API. Usually these APIs are designed by the principles of Domain Driven Design, where different teams map to bounded contexts that have their own published API.
Microservices are a form of SOA where each API also runs in its own process and thus has independent deployment lifecycle. Many in the SOA world advocated for this 10-15 years ago (and often ignored), and now the industry has gone back and coined a term for this practice.
In an organization that does microservices properly at scale, like Amazon, you have teams that build and run and upgrade their service autonomously from others. Read the Steve Yegge rant about Amazon and Platforms to understand this. It allows tremendous parallelisation of effort and allows for thousands of deploys to production daily without breakage. This is hard to pull off though, and a new initiative doesn’t generally won’t more than a small handful of services.
If it’s a shared library, then call it a shared library. It has a completely different lifecycle from a microservice. In your Android example, shared libraries by definition are controlled by whomever can dictate OS updates, not the app developer.
Working exactly along those lines this week on multiple services exposed over REST APIs, I was wondering if tools exist to check compatibility between them. Said differently,
- I have a `swagger.yaml` for my service A managing chipmunks and it says endpoint `/chipmunks` supports an 'color' query parameter.
- In service B, I have a `handleToServiceA` that encapsulates calling A. Then say I write `const chipmunks = handleToServiceA.getChipmunks({'colour': 'blue'})`.
Are there (whatever the ecosystem) tools that would read serviceA's swagger.yaml, detect my error in serviceB (color -> colour) and report the issue at compile time rather than run time?
One step better is autogenerated client libraries where a new version is created every time a new version of swagger.yaml is deployed. However, I don't know an open source project that does this.
Could you share methods that worked well for you? Asking because those question come up a lot and there never seems to be any conclusion.
I always tell people if you can't write and maintain a library then don't do microservices.
It's so silly that we are wasting so much time rebuilding existing things, poorly.
I’ll stick to linking glibc, friend architect. Performance is also architecture.
An architect is just a another man with an opinion.
We need science and theory, and skilled architects who can apply it to the problem at hand.
That being said, practical medicine is much less scientific than many think. There is a lot of master/apprentice learning going on, just as in software engineering.
In the world of math you don't need empirical data to verify a point. It's all logic derived from a small set of axioms.
Everyone else, including doctors, who call themselves scientists are just trying to float on the cachet physicists earned with their astonishingly good predictions. Properly speaking, they are phenomenologists . Please note that I'm not saying what they do isn't of great societal and intellectual value! The study and categorization of phenomena is certainly a noble enterprise. But none of them can make predictions good to 9 decimal places.
*might be billionths or quadrillionths by now in QED, but doesn't change my point.
Things like algorithmic complexity are well understood and formalized but design patterns are not a science nor has the concepts ever been formalized..
There is no theory or formalized system that says monolithic is better than micro or vice versa, it's all opinion. That's why its' called design.
For your latter point, it’s a matter of elegance, which is a pretty way of saying cognitively manageable. Think of epicycles vs Newtonian mechanics as an analogy. With enough epicycles you can compute the same result, but Newton’s approach is still a clear scientific advance.
Your point about design is well taken. Any given design is analogous to a theory. So we should aim for the simplest and most cognitively manageable design that satisfies our needs. That’s not literally formalized, but it’s a well established principle with an excellent record.
I'll take a TLA+ specification over a diagram any day.
Systems design in real life is an "artistic science": there are always known limitations (and some expected unknown ones) that rule out the theoretic optimal design for good reasons. The problem is that limitations and compromises are often not disclosed, and that many programmers and architects are too inexperienced to really grok the implicit meaning behind specific design decisions.
So we struggle along, with bloggers, researchers and FOSS contributors halfheartedly collaborating to make point improvements as solutions are discovered. Stodgy enterprises suffer, big tech companies make decisions with global impact, startups are left wondering WTF to do, and the rest of us largely don't care. Why? Because nit picking doesn't solve business problems [almost ever. I'd argue this point in cases of things like SCADA systems and other mission critical control systems.].
Nobody argues about which algorithm is better for sorted data sets: linear search or binary search. Theory already establishes one is faster than the other, but no theory establishes which architecture is better than the other.
TLA+ is one such system that includes a language for writing models and a model checker to verify them for you. In the context of microservices you would write a model of your services and the checker would help you to verify that certain properties of your model will hold for all possible executions of the model. Properties people seem to be interested in are consistency and transaction isolation. You can develop a model of your proposed microservices architecture and work out the errors in your design before you even write a lick of code.
Or if you already have a microservice system you could write a model of it and find if there are flaws in its design causing those annoying error reports.
Amazon wrote a paper about how they use it within the AWS team [0]. Highly worth the read. And if any of this sounds interesting I suggest checking out Hillel Wayne's course he's building [1].
[0] https://lamport.azurewebsites.net/tla/formal-methods-amazon.... [1] https://learntla.com/
The certificate means nothing, the training and instruction is priceless. Add to that things like TOGAF and a deep understanding of the current state of existing architectures and you'll understand what I originally meant, but failed to explain.
UPDATE: misspelled "meant"
You cannot possibly have every destination be a separate repo and then have the development lifecycle of your shared code be so active that it ultimately puts at risk the architecture of your entire organization.
What makes shared code so perfect is having stability such that you extract your variant code into your non-shared code. Shared code should evolve at a much slower pace than your non-shared code or you risk this very outcome.
Microservices are not dead, nor are they the solution to everything. We need better architects.