Long story short, one year and a half passes and the mvp is still not finished, the architect leaves the company and some poor guys (from an outsourcing company) are still going at it with the same architecture.
Distributed logging:hard
Distributed debugging: hard
Distributed versioning: hard
Distributed transactions: very hard
And on 95% of projects these problems aren't worth solving for the benefits of microservices.
And especially with a team of 4 or 5 developers.
A lot of stuff in software engineering is just driven by what's popular, not what's needed.
In the web world, if you have less than 10s of millions of daily users, as long as you design an application that holds no state itself, the architecture is usually more important for scaling your team and the size of your codebase rather than the number of users you can handle.
And all I can think about is "Hey, you could build this with a small team as a simple monolith and with proper caching you could probably run this on one or two raspberry pi, that is the amount of power you actually need here".
Don't get me wrong I do think they absolutely have their place and in other parts of the company we have much larger software development projects and they are absolutely making great use of Microservice architectures and Kubernetes and is getting a lot out of it. But that is 100+ teams building a product portfolio together.
If your product has a global audience who needs to CRUD stuff, caching and a single raspberry pi won't get you very far in the game.
If it's just an intranet stuff with a few hundred users and your front-end isn't very chatty then you're right, you don't need much.
One could argue that you need the reproducible dev environments and CI to be solved BEFORE even start using Kubernetes.
So out-of-the-box support for blue/green deployments and fully versioned deployment history with trivial undo/rollbacks are of no advantage to you?
> Reproducible dev environments and CI you can get easily without Kubernetes, without having to add complex solutions for logging, profiling and other introspection tools.
I'd like to hear what you personally believe is a better alternative to kubernetes.
And by the way, Kubernetes does not support not requires distributed tracing tools not "logging, profiling, and other introspection tools". That's somethings entirely different and separate, and something that you only use if for some reason you really want to and make it your point to go out of your way to adopt and use.
In fact, distributed tracing is only a thing not due to kubernetes but due to you operating a distributed system. If you designed a distributed system and get it up and running somewhere else, you still end up with the same challenges and the same requirements.
In 2016, Stack Overflow ran on 11 IIS web servers, but they really only needed 1.
[0] https://news.ycombinator.com/item?id=18496344
[1] https://nickcraver.com/blog/2016/02/17/stack-overflow-the-ar...
At least that is what I often think, when I hear people describing micro-services. If there is no data sharing between computations, then the problem is embarrassingly parallel [1] and thus easy to scale. The problem is not the monolith, it is the data sharing, which micro-services only solve if each service owns it’s own data. To be fair people advocating micro-services also often argue that they should own their data, but in quite a few of the instances I’ve heard described, this is not the case.
It's priceless to see their expressions when senior mgmt. and business owners finally find out.
I don't follow your reasoning. I mean, the ACID vs BASE problem you described is extensively covered in pretty much any microservices 101 course or even MOOC, along other basic microservices tradeoffs such as the distributed tax and ways to mitigate or eliminate these issues like going with bounded contexts. Why do you believe this is a mystery that no one is aware of?
Given that, once you have more than one service in your architecture, you cannot coordinate transactions across the distinct storage mechanisms - they are distinct databases and are therefore subject to the CAP theorem and other complexities of distributed computing.
Of course, nothing's stopping you from putting all your features into one service to avoid this thorny problem. It's a wise approach for most of us.
When you do that you have a monolithic architecture, not a microservice one.
Strong module boundaries
Independent deployment
Technology diversity
And with a single database you severely limit the benefits from strong module boundaries and independent deployments. Suddenly you have to synchronize deployments, and anyone can pull or change any data in the database which now must be enforced with discipline which gets you into the same boat as a monolith.
Only worse because you don't have tools (IDEs, linters, etc) telling you what parts of the code have to be changed.
A calls B, C, and D. B puts an item on a queue that eventually causes a call to C. Both B and C call D. Which D call is slow/erroring? Is it the AD, the ABD, the ABqCD, or the ACD call?
Having heard mainly "nothing but guff" justifications for microservices in non-enormous orgs, I think (as usual, and some law I forget dictates) the latter is more likely.
I have a project where only 3 developers work; we re designed it to be cqrs. I was sceptical at first - I especially don't like some of the boilerplate it creates - but I'm now sold on it by combining it with the mediator pattern, now I can have validation, logging and performance checks on every command and query without repeating code, works like a middleware
And ofc being only 3 devs it would be nuts to have microservices so we are happy with a cqrs monolith
This way, if an implementation path doesn't worth the effort, just drop it and don't spend dozens of months with a full rewrite and realizing that you are spending a week to implement a CRUD for a simple entity. It's even worse when devs are new to the project and have no knowledge of the domain; I've been burned by this, never again.
At the end of the day, we are not in the research field, we are payed to fix business problems, to deliver something that brings a business value. Yeah, it's nice to write some really complex software that would make the system a lot more optimized, but we have to deliver it before our bosses are getting tired of excuses and pull the plug. Learned that the hard way.
Allaire were there with a workable solution, before jsp, php, (?) asp. Iirc only perl was serious competition.
I still sometimes see lotus notes '.nsf' links
Edit: Also want to add software installation files are very important to keep offline as well.
If they have “the answer” but don’t understand the “why we are here today” their decisions should be, at the very least, suspect.
that's how you earn your thousand yard stare
It is very rare to see a codebase several years old that can be fairly describe as "in good shape".
A core component of making great decisions is understanding the rationale behind previous decisions. If we don’t understand how we got “here,” we run the risk of making things much worse.
So you helped the new dev understand the current lay of the land. They listened, then suggested an improvement based on their new understanding. You agreed and together improved the code.
It's a pure cost/benefit analysis question. Switching has some cost, and some benefits, and you have to decide which are likely greater.
Possible answers:
• Legit technical or cost/benefit reasons
• Probably a good idea but no budget/time for the effort
• Somebody already tried and it was a disaster
• No manager has been stupid enough to humor the devs
Mostly devs want to go away from the old and increasingly creaky thing. And when the cost / benefit ration is finally right (that is, it's creaking loud enough and slows things down much enough), the move hopefully happens.
Most devs are not reasonable.
This has not been my experience, even within the past few years on occasion.
Sometimes I just can't find out (the guy who wrote it is gone, nobody knows) then I have to just do it the way I think it should be, test ... then I find out why, sometimes spectacularly ;)
Parts of the problem get worse as time goes on. (more code to convert, more complexity, more test cases, EOL for the python2 version of some lib, more user data, blah)
Parts of the solution get easier as time goes on. (easier syntax sugars, better testing frameworks, infrastructure abstracted easier, remove dead features, etc)
Parts of the desired solution become less relevant as time goes on. (Why not use golang or node or elixir, php and python are so dated!)...
Just because last year was a bad time for the upgrade doesn't mean today is. Knowing how to get these things done at the right time by the right people for the business is what separates a great engineering manager from one that is just "shipping features and bug fixes".
I am a senior dev and I always try to push new ideas to my team lead (senior dev and older than me), but all I get is blatant criticism because he says "I've tested it and didn't like it). A clear example: refusing to move to Spring Boot and still staying on the dead horse JavaEE, which got more complicated and fragmented than ever since Java 9
Your experience sounds great, but it also sounds like we don't see that level of middle-ground pragmatism enough - it's either "no, we have to stay using PHP4.3.3 because it's what I know" or "we have to rebuild to cloud micro services to be able to scale infinitely without being restricted by schedules or budgets".
Definitely have been one or two refactors that have been shutdown. We've been lucky to have clients that give us the freedom to retire some technical debt/risk instead of just churning out features. And we're small enough (200 kLoc approx, 6 devs) that full codebase refactors are still doable.
"I'll rewrite it in a week!"
A junior tried to advocate for browser test. Senior from the developer productivity team said browser test couldn't possibly be made non-flaky.
I didn't know how to advice the junior because I agreed with them.
Saying "something is infeasible/costly" is like a blanket statement. There was no way to quantify that.
On a flip side, we couldn't justify the impact of browser test either. But I felt, at the time, we should've been biased to implement every type of tests (than not) since there are only 3 types: backend, JS, and browser.
Akin to your example, it was the same argument "X is costly/infeasible". Your example doesn't have any issue because everyone probably agrees with it. But being infeasible to setup browser test sounds strange.
Also, if we don't plant the tree now, then when?
1) Keep a set of a few thousand URLs to diff, add new ones as needed.
2) Use a tool that can request these URLs from two versions of your app, make a side-by-side visual diff of the resulting pages, and present a report of any differences.
3) Before committing any change to your codebase, run that tool automatically on the whole URL set, comparing the new version of the app with the current version. Then the committer should review the diff report manually and it gets attached to the commit history.
This way, adding coverage for new functionality is easy (you just add some URLs to a text file) and the whole thing runs fast. And it catches all kinds of problems. Not just UI bugs where some bit of CSS messes up something unrelated, but also you can run the frontend diff after making any backend change and it will catch problems as well. It won't solve all your testing needs, but it covers a lot of ground cheaply, so you can concentrate the custom testing code where it's actually needed.
If the test is slow and hard to deflake, then let's add it slowly. Maybe only test the critical path. There are ways to manage it, instead of discarding it entirely.
I learn a lot from them, because I'm asking them to do a lot, and they also learn from me when I review the code or advise them on an alternative. They push back a lot and that's what I want. I want my reports to prove me wrong and show me better.
This approach means that if I really have to put my foot down or enforce something, then there is enough mutual trust to allow that to happen.
How does this benefit the user again? Could the junior have been working on that spa for marketing? That's one months salary..
As a manager I always hated these situations, firstly, because the problem the juinor dev is trying to fix is almost never as productive as "Hey let's upgrade to python3", it's more like "Hey let's migrate to this specific version of this specific tool that I happened to use on one of my pet projects" or "I've got this incredibly ambitious plan to change everything about this peice of code you gave me to work on" (You're only looking at that code at all because it's simple, relatively unimportant and we're trying to ease you into the team).
The problem is if it is the junior dev whose wrong you're now going to start seeing all these new issues because you've taken someone whose intuition isn't quite there yet and given them something incredibly complex to do. That's when you get into work 2 weeks later and find their mega-commit changing 2,395 files changing tabs to spaces and auto-modifying everything to camel case. Taking a chance on that intern's pet project means a lot of support from others in the team.
Anecdotes are by definition anecdotal; I am not promoting an anti-science position by relating a personal anecdote and I resent the statement that I am doing so.
If you'd like to write a blog article that promotes scientific thinking, I strongly encourage you to do so.
Some people will use any excuse to not have to change. Once in a while it's because they are genuinely too busy, but that just signals other issues.
Only if you just fix whatever breaks in the upgrade and never use the features in Python 3. The static typing benefits alone should either make your software more reliable (benefit to users) or speed up feature delivery (benefit to users).
That gave me a chance to explain the testing and refactoring cost that would come with changing python versions, and how the benefits to users would be almost zero. And then at some point one of the new juniors said, "hey, there's a lot of filesystem performance improvements and readability improvements (f-strings) in 3. I can get the test/refactor done in a month, and I think it's a net positive." They were right, and now we're on python3.
So, sometimes we all learn something.