Conway's law:
Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure.
This is why, BTW, I'm so dismayed Elon Must fired 85% of Twitter. Now many of their critical services have teams of zero or one developer. Those are basically dead services walking.
The organization is often not reflective of what’s actually being achieved. Instead of Catalog, Offer, Order and Shipment you have Alice’s org, Bob’s org and Charlie’s org. If you encode the interfaces between these (impermanent) group boundaries rather than the conceptual boundaries you enter a world of pain.
Charlie might own Order and Shipment because Charlie is the warehouse exec and 20 years ago, when only physical items were sold, all of the order-processing computers were located in a cage in the warehouse. Fast forward to today and the cage is gone, and the order processing system handles digital items and services in addition to physical goods. But good old Charlie still owns the Order and Shipment system because they were baked together 20 years ago.
If you are explicitly talking about business domains and bounded contexts, then you can see areas where the Alice/Bob split is not optimal (typically, lots of team coupling across org boundaries). Without those concepts I think it is easier to justify arbitrary political/territorial org structures.
There is no panacea; one can always provide contorted justifications for things. But I think having a framework really helps, and particularly, having one that includes business stakeholders and not just pure technical functions.
Sir, your assembly code is so hard to read with all them GOTOs. You should do structured programming, use constructs such as loops to make it easier to follow.
Man, your code is so long written like that, you should break it into functions to make it easier to reason about.
Dude, you have so many functions it's impossible to make heads or tails of the logic, you should break it into classes to be able to abstract away all of this detail.
Bro, you have so many classes I don't even know where to start. You should introduce modules so that it becomes possible to isolate different parts of the design.
Dawg, you have so many modules, you ought to break them into small different runtimes communicating via a rest protocol to isolate different parts of the code.
Choom, you have so many microservices, you ought to break them into different microclusters to separate the different parts of the API.
X'man, you have so many microclusters, you really ought to break them into different miniclouds to spearate the functionality and make it easier to follow.
𐀘𐀐, you have so many miniclouds it's impossible to deal with this code, you really ought to separate it into functional minicloud clusters.
What's the marketing name for that?
[1]: I apparently deleted the post so I’ll have to undelete it and come update this comment.
>> Otherwise [without microcompanies] you just can't do software engineering right.
So true
Oh, that's an implementation detail. The market will sort it out.
But it does sound enticingly pure, a conglomerate of blockchain companies without further function.
But this approach, Microcompanies as it is called (units developed by independent companies), is also partly no-code, in that the specifications and different functional units are glued together using lawyers.
Exactly! And it all runs on the invisible hand of the free market.
Our main mistake is that we invent a level of division, and never make it recursive, i.e. a class is badly designed to be an entire module, it can't contain classes (inner Java classes are not what I'm talking about), not can classes cross machine boundaries easily in many languages and so on.
We need a unit of division that works across machines just as well as it works across files, or across statements in the same file, up from data centers to CPU instructions.
Our languages and platforms are full of the SAME THING but SLIGHTLY DIFFERENT, under DIFFERENT NAMES. You listed many of them. Machine opcode, expression, function, class, module, service etc.
Some of the problem is that people need different capabilities from different abstraction layers. The rest of the problem is that there are a lot of incompetent people creating software.
I'll cite Erlang as an example that comes somewhat close to what I mean. An Erlang process is a function, a class, a module, and a remote service, all at once. It can be done. We just never stopped to truly think about it and try.
Oh, you are right about this. But a lot of people stopped to think about it. It always seemed to fail due to market failures, like most other innovations in development.
Mircoservices, Projects in a monorepo, Infrastructure as code, ...
Sometimes they don't even have abstractions at all.
And there are whole companies being built to make these things even remotely managable.
Computers today don't do significantly more than they did 25 years ago. They just do the same things with 1000 times more resources across a global computer network.
Likewise, velocity is startlingly bad if you follow conventional advice.
In my opinion, Microservices is an implementation detail, and the extend to which DDD principles are applied is about people and ownership.
It is. But it's a very strong one, where it's harder to blur the boundaries because you're putting your code into a service or another.
And that's the advantage. Not because it's easier to apply correctly. But because it's harder to do it badly.
And if you do it badly, it will show and the product performance will be a mess.
From my experience, breaking down a monolith without first identifying and isolating its bounded contexts (as in DDD) is a recipe for disaster. With your domains completely entangled in a single codebase, you won't pull only what you want and you will have to bring a lot of undesired code, behaviours and side effects with it.
The safest recipe, in my opinion, is to break down your monolith in bounded contexts (or ideally, to have focused on it from the start), so you can better reason and understand the advantages and trade-offs of moving each specific domain and responsibilities out of your unique codebase.
I saw often teams being motivated in moving their code out of a legacy codebase because of the huge technical debt, but they were bound to suffer a death of a thousand cuts by just bring the same debt in a new service, and adding network constraints on top.
Rarely have I disagreed with something more strongly :)
If a team lacks the discipline/skill/wherewithal to create good boundaries within a single code base when stakes are at their lowest, they absolutely don't have the skill or discipline to do ahead of time, across N code bases, and with a network in between all of them.
Breaking your system apart too early, and when you know the least about it, and with the rationale that doing so will magically stop the problems that a lack of engineering skill creates, has, at least in the small slice of the world I've encountered, resulted in nightmarish levels of complexity (some of which I'm directly responsible for the reasons stated! (whoops!)).
Part of the problem is the platform itself.
Most mainstream languages today ALL support shared mutable state. Meaning you can hold a pointer or a handle to something, mutate it, and someone else having the same pointer or handle has it changed under their nose.
When you fetch, say, JSON over the network, the party who sent it to you can't change it under your nose either intentionally or accidentally. It's a snapshot of data you own, and you can make decisions on at your own leisure. Sure, the snapshot may get out of date by the time you send your next API request, but this contract is clear and obvious.
While the entangled meshes of mutable code in say C, C++, Java, Swift, Python, JS, etc. etc. often make the mess unavoidable.
Rust goes to some length to stop this problem, but it's a language that's low-level (by modern standards) and technical, and I don't think anyone uses it for enterprise automation exactly.
Another problem are nominal (vs. structural) type systems. Well I don't have to explain, but when you fetch JSON from an API a nominal type won't stop you from reading that data, you care about the structure.
"Shared" implies shared ownership, this is where the confusion comes from. When you ask an API about a user's profile, it's that API's user profile. But the API response itself is entirely yours. It won't change right under your fingertips. The original profile may change, but you know that the profile is not yours already.
Swift has value types which use pass-by-value semantics. Containers use copy-on-write. This is a language where you can quickly build value trees that you know are a guaranteed copy, isolated from whoever sent you this tree.
But most languages have no such infrastructure. Java is adding record types, but they're immutable instead of in-place mutable, which is incredibly inconvenient to work with in nested data structures.
Java's new type offerings are immutable (records, with value and primitive types in the works). Hopefully a push in the right direction.
You say side effects are a problem but you also say the developer cares about his structure in microservices.
When you can’t handle side effects how will you handle microservices?
You could do the same without microservices by just abstracting in libraries which have a defined API.
Microservices will normally make your application more complex. Developers will mess with them too. They can’t change the JSON but some dev will change the API of some service and read the wrong property…
It is also possible to just police it. Supervise it. This is partly anathema now with agile etc. But the iPhone is what it is because there were a few dictators at the top saying "no". This is actually more effective than a physical boundary because:
1) People will hack around the physical boundary anyway.
2) It forces a conversation between the supervisor and the developer every time the supervisor raises an issue. At this point the developer may get a chance to learn something about the reasons for higher level structure. But more importantly the supervisor might learn something about what is wrong with the higher level structure.
But then your project design is now dependent on org hierarchy, and getting the right kind of "people". This isn't guaranteed or easy to do, so I would not consider it a serious situation.
For example from my experience, splitting an Auth/User service for security/compliance, or splitting a Payment service for the same reasons.
Or splitting out a core low-level platform layer that per-product teams can build atop, allowing each to iterate faster, while the Platform layer focuses on clean abstractions.
Nothing in DDD or microservices says you need to sit down and design the service boundaries up-front, indeed most of the advice I’ve seen in the community suggests the opposite.
https://www.uber.com/blog/microservice-architecture/
(Though only after they felt the pain of thousands of disorganized services.)
Eric Evans (inventor of DDD) has said in recent years that unfortunately once a team is big enough, all the invisible conceptual boundaries between domains blur and disappear. People do NOT have the discipline to do DDD correctly.
But do you know what helps? Having physical boundaries you can't cross so easily. Microservices. Or as I call them... just "services". Eric Evans has said that people using microservices may represent the most successful application of DDD's principles, not because people using them were fans or even aware of DDD, but microservices enforce the patterns DDD Evans was describing.