In a start-up, having a team of young people will allow you to move fast, pivot and deliver. What you usually end up with though, is a tower built from spaghetti that's serving actual users. I see this again and again and then people get surprised that the software doesn't scale.
But... yeah. A colleague (25+ years) has to deal with this more than I do, but have hit it some over the years, and it happens more the older I get.
Often he or I may be 'right' but only because of existing constraints in a system. Someone with a different view may even be 'correct' in a nuts-and-bolts technical sense. Yes, A is 8% faster than B. But... we have an existing system in B that meets our current needs, and team who know B, have written it, and maintained it for a couple years. Switching tech stacks to something no one knows for an 8% network speed boost is not remotely helpful.
My experience was also that the young manager was very suspicious of the old fart. I've never had any problem working for managers younger than me; but some young managers don't know what to make of a team member that knows more than they do.
[0] No insult intended; almost everything I learned after the first 10 years, I learned from people younger than me.
Not trying to be harsh here, but not expecting an increase of network call in a system where each component is tied together with... network calls sounds a bit naive.
> We are now attempting to solve this with caches and doing batch requests
So you have built a complex and bottle-necked application for the sake of scalability, then having to add caching and batching just to make it perform? That sounds like working backwards.
Obviously, I have no clue on the scale of the project you are working on, but it sure sounds like you could have built it as a monolith in half of the time with orders of magnitude more performance.
Scalability is a feature, you can always add it in the future.
Yes, if you can't scale fast enough as you need to, it can hurt your business. Not being able to keep up with demand is a (luxury) problem that every business faces, not just in tech. They would often be called 'growing pains' in a business, and though they are bad, they rarely contribute to the failure of a company.
Starting a startup/service/platform with microservices before you even understand the bottlenecks/market fit/customers is usually not a good idea. You can come a very, very long way with a monolith before you hit performance and scalability limits. And once you do, you can always start breaking things up into smaller services for scalabity. Obviously you need to make sure you are scaling on time to keep up with demand.
'Nail it, then scale it', and 'premature optimization is the mother of all f-ups' are popular sayings for a reason.
> I was told ~1200 RPCs independently by several engineers at Twitter, which matches # of microservices. The ex-employee is wrong.
> Same app in US takes ~2 secs to refresh (too long), but ~20 secs in India, due to bad batching/verbose comms.
RPCs are on the server side. Why would they app take longer to refresh in India than in the US?
Some more explanation:https://twitter.com/mjg59/status/1592380440346001408
How the hell... like... who decided to do Microservices in the first place if they didn't know this? This is such a rookie mistake. It's like somebody right out of high school just read on a blog the new way is "microservices" and then went ahead with it.
And both are way more common than they should be.
Microservices can have both design utility and simultaneously been a major fad.
Lurking reddit and HN you can watch development fashions come and go. It's really really profoundly hard to hold a sober conversation on splitting merits from limitations in the middle of the boom.
tl;dr: gartner_hype_cycle.png
Let each microservice owner figure out how to achieve their latency+reliability SLA's in every location - whether replicating datastores, caching, being stateless, or proxying requests to a master location.
At my last job we created a whole fleet of microservices instead of a single modular project/repo. Some of them required non-trivial dependencies. Some executed pretty long-running tasks or jobs for which network latency is insignificant and will remain so by design. Some were few pages long, some consisted of similar-purpose modules with shared parts factored out. But there was no or little processes like “ah, I’ll just ask M8 and it will ask M11 and it will check auth and refer to a database. I.e. no calls as trivial as foo(bar(baz())) but done all over the infra.
(Because this is at the same time one of the defining elements of this architecture... and the first one to be opted out when you actually start using it "for real").
Lots of ways to solve a problem, though.
The primary reason for this is that PDFs can contain executable code and the common tools used to process them are full of unpatched CVEs.
Imagine if you are responsible, at runtime, for linking object files (.o) where each object is a just the compilation of a function.
Now why would anyone think this is a good idea (as a general solution)? Because in software organizations, the “linker’s” job is supposed to be done by the (“unnecessary weight”) software architect.
Microservices primarily serve as a patch for teams incapable of designing modular schemas. Because designing schemas is not entry level work and as we “know” s/e are “not as effective” after they pass 30 years of age. :)
> monolith
Unless monolith now means not-microservice, then be aware that there are a range of possible architectures between a monolith and microservices.
For this reason, I've been trying to push for building a monolithic app first, then splitting into components, and introducing libs for common functionality. Only when this is all done, you think about the communication patterns and discuss how to scale the app.
Most microservice shops I've been in have instead done the naïve thing; just come up with random functionally separate things and put them in different micro services; "voting service", "login service", "user service" etc. This can come with a very very high price. Not only in terms of network traffic, but also in debuggability, having a high amount of code duplication, and getting locked into the existing architecture, cementing the design and functionality.
And it is reassuring to hear that you seem to have success in avoiding these issues with a monolithic architecture, as I thought I was oldschool for starting to prefer monoliths again.
The main thing is that regardless of scaling, the app should always be able to run/debug/test locally in a monolithic thing.
Once people scale they seem to abandon the need to debug locally at their peril.
Scaling should just be a process of identifying hot function calls and when a flag is set, to execute a call as a network rpc instead.
[1] https://scholar.harvard.edu/waldo/publications/note-distribu...
I agree with sentiment of the paper in that it can never be fully transparent. When I say "it should just be a process of identifying hot function calls..." this is admittedly a little contrived - there would need to be a more thorough analysis.
The main point seems to be that partial failure is too difficult to handle without explicitly declaring some objects as "remote".
> Merging the models by making local computing follow the model of distributed computing would require major changes in implementation languages (or in how those languages are used) and make local computing far more complex than is otherwise necessary
Given that the paper is from 1994, I'm wondering if there have been language changes that make this more achievable.
What comes to mind is the event loop. The fact that in Node.js, almost every function call is now an asynchronous promise due to the non-blocking paradigm coupled with the convenience of async/await syntax. These event loop runtimes provide a universal way to hook into asynchronous calls.
I would imagine that in 1994, envisioning the code added to the languages of the day needed to handle timeout and asynchronous invocations would have felt overwhelming.
Also thinking about ORMs and databases. It seems TOPLink for SmallTalk was the first, released in 1995 after this paper was. ORMs seem kind of like "transparent remoting", as they usually can run against an in-memory database or db on a local system, or over an TCP connection, which then needs to deal with latency and failure. And for any app talking to a DB with an ORM, means that the failure cases of the ORM must prevade throughout the stack.
> Given that the paper is from 1994, I'm wondering if there have been language changes that make this more achievable.
As the paper argues, it is not possible to make remote calls look like local ones. But it is trivially possible to make local calls look like remote ones! Use call-by-value parameter passing, and make all call asynchronous. The most successful solution I have heard of is Erlang's immutable-message-passing model.
I first wrote a program which ran on a web server about 25 years ago. In that time, computers have experienced about ten doublings of Moore's law, i.e. are now over a thousand times faster. Computers are very fast if you let them be.
Either way, one of my biggest pet peeves is the near-ubiquitous use of HTTP & JSON in microservice architectures. There's always going to be overhead in networked servies, but this is a place where binary protocols (especially ones like Cap'n Proto) really shine.
I wouldn't call that entirely unexpected. :-) It's a rather well-known issue:
Microservices
grug wonder why big brain take hardest problem, factoring system correctly, and introduce network call too
seem very confusing to grug
> especially since some of these services are not even in the same data center.
I think you need to answer why? If you can't put all of the services in one data center, then by definition you can't write a monolith either. If the monolith would happily run in one datacenter, then you should have all instances of your microservices in that one datacenter.
It surprised me that you would conclude that this is a problem with microservices. It's like if a particularly architect always punches you in the groin in every meeting, and you've concluded that architects are bad people, rather than this one architect is a bad person.
Why is the other service in another data center? Does it need to be in another data center? If it does, how will a monolith help?
This experience has strongly impacted my view of microservices and for all personal projects I will develop in the future I will stick with a monolith until much later instead of starting with microservices.