Why not break a microservice into a series of microservices, its microservices all the way down.
Multiple services connecting to the same database has been considered a bad idea for a long time. I don't necessarily agree, but I have no experience in that department. It does mean more of your business logic lives in the database (rules, triggers, etc).
Not true at all.
You're conflating the need for distributed transactions with the definition of microservices. That's not it.
> Multiple services connecting to the same database has been considered a bad idea for a long time.
Not the same thing at all. Microservices do have the database per service pattern, and even the database instance per service instance pattern, but shared database pattern is also something that exists in the real world. That's not what makes a microservice a microservice.
You should read up on microservices because that's definitely not what they are not anything resembling one of their traits.
No, this does indeed match reality. At least for those who work with microservices. This is microservices 101. It's baffling how this is even being argued.
We have industry behemoths building their whole development experience around this fact. Look at Microsoft. They even went to the extents of supporting Connected Services in Visual Studio 2022. Why on earth do you believe one of the most basic traits of backend development is unreal?
> I work at ~50 ish employee company and we have layers of dependencies between at least 6 or 7 various microservices.
Irrelevant. Each service has dependencies and consumers. When you need to run an instance of one of those services locally, you point it to it's dependencies and you unplug it from it's consumers. Done. This is not rocket science.
I'm talking about how Microsoft added support for connected services to Visual Studio. It's literally a tool that anyone in the world can use. They added the feature to address existing customer needs.
How does that solve the problem of a mess of interconnected services where you may have to change 3 or more of them simultaneously in order to implement a change?
Yes. That's the point.
> How does that solve the problem of a mess of interconnected services (...)
I don't think you got the point.
The whole point is that you only need to connect your local deployment to services that are up and running. There is absolutely no need to launch a set of ad-hoc self-contained services to run a service locally and work on it. That is the whole point.
No. My whole argument is open your eyes, and look at what you're doing. Make it make sense.
Does it make sense to launch 50 instances locally to be able do work on a service? No. That's a stupid way of going about a problem.
What would make sense? Launch the services you need to change, of course. Whatever you need to debug, that's what you need to run locally. Everything else you consume it from a cloud environment that's up and running.
That's it. Simple.
If there's something preventing you from doing just that then that's an artiicifal constraint that you created for yourself, and thus that you need to fix. We're talking about things like auth. Once you fix that, go back to square one.
True. Your run-of-the-mill shop should have a simpler and more straight-forward system.
But you seem to want the reverse.
You have to test the changes you want to push. That's the whole basis of CI/CD. The question is at which stage are you ok with seeing your pipeline build.
If you accept that you can block your whole pipeline by merging a bad PR then that's ok.
In the meantime, it is customary to configure pipelines to run unit, integration tests, and sometimes even contract tests when creating a feature branch. Some platforms even provide high-level support for spinning up sandbox environment as part of their pipeline infrastructure.
No, not really. You only find yourself in that spot if you completely failed to do any semblance of integration test, or any acceptance test whatsoever.
That's not a microservices problem. That's a you problem.
You talk about feedback look. Other than automated tests, what do you believe that is?
I just also like to have an option to run service locally and connect to either cloud instances (test) or local instances depending on what I am troubleshooting/testing. Much better than debugging on prod which may still be required at some point but hopefully not often.
Ouch. Where they using macOS at the time with laptops having not-enough-ram?
I've seen that go poorly on macOS with java based microservices. Largely due to java VMs wanting ram pre-assigned for each, which really chews though ram that mostly sits around unused.
This was a few years ago though, at the tail end of Intel based mac's where 32GB ram in a mac laptop wasn't really an option.
"Oh it's not running locally, you need to also run service_18_v2.js, and include the right env variables"
I think nobody really talks about this, but unless you have a docker-compose.yml that includes everything you need for local development, it's increasingly more likely that you'll end up coupling things to Kubernetes to such a degree that running without it (and its abstractions) will become more effort than a person can muster.
So while people try to create services that are decoupled from one another, they end up instead coupling them to Kubernetes concepts and a service mesh, service discovery, configuration and secret management mechanisms, persistent storage abstractions and so on.
Which you can obviously do if you want to, but which might make running things locally in a minimalistic fashion that much more complex.
It's the same as with for example using a web server as a reverse proxy for my applications and ending up putting some logic in there (e.g. route rewrites, headers etc.) and then realizing that I must also run a similar web server locally for 1:1 compatibility because something like Vue dev server proxy to the locally running API won't be able to give me all that.
Is this a problem?
I mean, in this scenario docker compose serves two main purposes: launch a few mock services, and configure those services according to your needs. This means configuring them to consume services already deployed to a cloud environment of your choice. This is something you control.
Then all that's left is the service (or set if services) you need to modify. You run those locally and configure them to consume a mix of the mocked services and the services deployed to a cloud environment.
> (...) it's increasingly more likely that you'll end up coupling things to Kubernetes to such a degree that running without it (and its abstractions) will become more effort than a person can muster.
That coupling can only happen if you intentionally add it yourself.
If you need to run your services in isolation, you will be more mindful of the need to not introduce that sort of coupling.
Running the dev environments remotely (or rewriting in Go) were the options being considered before the whole project was canned and people redistributed to other things.
> Largely due to java VMs wanting ram pre-assigned for each
Not really. It all depends on what are your needs.
> It defeats a major feature of the jvm.
You're confusing things. Just because Java addressed the deployability problem for Java applications before containerization was even a word, this does not mean that deploying a JVM per service is a bad idea. Just think about it for a second. Why do you need to deploy and scale services independently? Do you mention the JVM in your response? No.
I did mention the jvm in my response. You quoted me doing so, in fact.
Your dev environment is expected to mimick your production environment, not the other way around.
This is certainly one of the critical mistakes you did.
No developer needs to launch half of the company's services to work on a local deployment. That's crazy, and awfully short-sighted.
The only services a developer ever needs to launch locally are the ones that are being changed. Anything else they can consume straight out of a non-prod development environment. That's what non-prod environments are for. You launch your local service locally, you consume whatever you need to consume straight from a cloud environment, you test the contract with a local test set, and you deploy the service. That's it.
> I guess one can grumble about bad architecture all day but this had to be solved.
Yes, it needs to be solved. You need to launch your service locally while consuming dependencies deployed to any cloud environment. That's not a company problem. That's a problem plaguing that particular service, and one which is trivial to solve.
> Both FAANG companies I’ve worked at had remote dev environments that were built in house.
All FANG companies I personally know had indeed remote dev environments. They also had their own custom tool sets to deploy services locally, either in isolation or consuming dependencies deployed to the cloud.
This is not a FANG cargo cult problem. This is a problem you created for yourself out of short-sightedness and for thinking you're too smart for your own good. Newbies know very well they need to launch one service instance alone because that's what they are changing. Veterans know that too well. Why on earth would anyone believe it's reasonable to launch 50 services to do anything at all? Just launch the one service you're working on. That's it. If you believe something prevents you from doing that, that's the problem you need to fix. Simple. Crazy.
Yes what you're saying is correct, but why many words when few do trick:
This hypothetical IT department isn't able to host its own development environment, yet suddenly they do have the skills if they switched to gitpod.
If your services are mostly stateless and/or your development team is very small that can work. If not, you will quickly run into problems sharing the data. Making schema changes to the shared cloud services. Cleaning up dev/test/etc data that has accumulated, etc. Then you are back to thinking of provisioning isolated cloud environment per dev.
At a previous place of work I worked with a monolith structure, and it was actually perfectly fine. Development got done separately on several large substructures in the monolith, and devs could install the whole project locally and run it just fine.
I'm really wondering why we're all using microservice architecture if we're all convinced that to actually develop on them, devs need to reproduce 50odd of those services locally for debugging. Then what was the point?
Resume chasing and trying to paper over the fact that you don't understand architecture, plus bad tooling that e.g. doesn't properly support incremental compilation and so makes monoliths painful.
No one does microservices for resume chasing anymore, because everyone is already doing it for practical reasons. I never came across a monolith that wasn't in the process of peeling responsibilities to either microservices or function-as-a-service. For project managers to open their eyes, all that's needed is something like a deployment going wrong due to a single bad commit, or things scaling weird because a background process caused a brownout, or even external teams screwing up a deployment after pushing bad code.
Why do you think that other people using a cloud environment prevents you from using the environment?
No, not really. The size of your teams have zero to do with whether your services can corrupt data. That's on you. Don't pin the blame on a service architecture for the design errors you introduced. Everyone else does not have that problem. Why are you having it and blaming the same system architecture used by everyone else?
> Making schema changes to the shared cloud services.
What are you talking about? There is absolute zero system architectures where you can change schemas willy nilly without consequences. Why are you trying to pin the blame on microservices for not knowing the basics of how to work with databases?
In all the times I had to work on schema changes, the development was done with a db deployed in a sandbox environment, and when work was done we had to go with a full database migration with blue-green deployments along with gradual rollout. Why on earth are you expecting you can just drop by and change a schema?
> Cleaning up dev/test/etc data that has accumulated, etc.
Isn't this a non-issue to anyone who works with databases? I mean, in extreme scenarios you can spin up a fake database with your local service, but don't even try to argue this is the justification you need to launch dozens of services.
The truth of the matter is that this is extremely simple: if you want to work on a service, launch that service locally configuring it to consume all dependencies running in a non-prod environment. That's what they are for. If you have extremely specialized needs, stub specific dependencies locally. That's it.
Make is just a framework for you to do your builds. Sure you can cram anything into it, but that is exactly the kind of area that other tools like Ansible or even Terraform shine.
Make isn't a silver bullet.
EDIT: Just to make sure, I'm using fetish as something you spend an unreasonable amount of time with it.
You would list the services you need (or service groups) in a config file, start a command and all services would start in containers. Sure, you need a lot of RAM with that but on 32Gb it was working fine.
So 64 G to include the ram for the ide and web browsing / javascript apps like slack?
Imagine the goal is to fix the problems (e.g. make it possible to run less of the services or something like that): How do you do that without first running all the services, making the proper changes, and then testing those changes? You need to be able to run all the services in that interim period.
So, wouldn't it be nice if there were a solution for this in-general? And, maybe, it would lead to better conditions later on. But in the meantime there is really no way around the existing design/decisions/etc. You simply have to deal with that reality and engineer around it.
Yeah, I get that, I was deliberate about the phrasing of "your company" rather than just "your".
Obviously we don't know anything about the parent commenters company and situation, perhaps 12 people per microservice genuinely is the right solution for them, but it seems like it would be better not to get into this situation in the first place, though once there you obviously have to tackle the problem as it presents itself.
Why do you need to run all services in isolation to be able to troubleshoot and isolate a problem?
This is certainly not universal among FAANGs though.
Requiring 50 services to be up is absolutely nuts, but it’s actually pretty trivial using something like Nomad locally.
Trying to boot the full service on a single machine required every single developer in the company installing ~50ish microservices on their machine, for things to work correctly. Became totally intractable.
I guess one can grumble about bad architecture all day but this had to be solved. we had to move to remote development environments which restored everyone’s sanity.
Both FAANG companies I’ve worked at had remote dev environments that were built in house.