>You have to understand the entire system and all the interaction between all the parts.

Maybe if you're CTO. Never seen that in case of individual contributors.

I've been that IC. You can easily find cases where a feature implementation involves a vertical slice of the whole stack, just like you would in a monorepo. Do that enough times with enough different features and you end up touching most things. If you don't have organizational blockers to velocity, and you treat every task as a chance to learn something new about the architecture (as you should!), you don't need all that long to develop a sense of what's where and how the pieces fit together.

You're not likely ever going to be at the same level of detail on everything all at once, and there can still be fuzzy parts - third party partners in particular can be difficult, especially if the org is not good at making known who's the SME for what - but you absolutely can build an understanding of the architecture as a whole, and that makes you able to dive in and work on just about anything without needing much spinup time.

(Granted this assumes good technical and architectural leadership early on that has specified standards for interservice communication, intraservice code design, what layers exist and how they interoperate, etc. That said, Wild West microservices might still be less painful to work on than a Wild West monolith - I don't actually know, but I could come up with arguments either way.)

Putting on my business hat, the shift to microservices needs to happen top down and involves organizational structure as well. Each service should have a clear team which owns it and a single threaded owner for the team.

Depends where you are. That’s not going to happen at Big N size environments.

In such environments, I wouldn't expect it to be particularly useful or necessary, either - it's not a good use of effort to deeply understand foo-service if I can just ask somebody on the Foo team, or read their schema and docs, and get what I need much faster that way.

I'm thinking more from the perspective of a small team supporting a large operation, because those are the kinds of situations where I've spent a lot of my career. FAANG scale would be a novel experience for me, as I suspect it would for most.

In that scenario I’d be a lot more hesitant to do true “microservices” because IME it’s a good way to complicate interactions and development without achieving much.

That's fair. In the specific case I have in mind, load was very bursty and scaling was a major concern; we were always looking to minimize both latency impact on revenue and infra cost impact on profit. Having the critical path live in its own heavily optimized services made demand scaling very easy and very fast; I don't think we could have done as well with less separation, and once you've already got the necessary infra and practices in place, it becomes very natural to apply them to the other problems you have to solve.

Not to say you can't get in trouble that way, for sure. It's its own kind of tradeoff, not least in that you need more ops support than a simpler infrastructure would require, and you need someone with the architectural vision and authority to ensure things don't get out of hand. It's definitely not where I'd want to start with anything, and you need a strong team to make it work, but I can say from experience that it can work amazingly well. Like, about twenty people supporting about $2B ARR well.

The truth is that even for very large companies there's always push and pull between service separation and ops burden. If you're thinking about it you're already in a better place than many.

Any higher than senior engineer better understand the whole ecosystem at a reasonably sized company.

(Sure if you're google sized, nobody will understand or even know the entire system. But nearly every company on earth is smaller than that.)