You cannot stay up to date permanently, because that introduces risks all the time. Once you earn money, there are layers to protect the money stream.
You can loose tons of money by not staying modern, that is true. But you can loose everything when you ignore a risk. The trick is to find a balance.
And all of that is not considering investors and their idea how the money should be used.
I suspect corporates don't really want to get stuck with so much legacy, they just don't know how to avoid it.
The problem in enterprise tech like you found is that one is forced to use certain, non-productive old frameworks filled with legacy, over-engineered bloat.
Modern Java micro-services in contrast are really fast to develop. Green field Java projects where one can make personal choices of lean technology and libraries are simply amazing. They also have terrific characteristics under load. The JVM performs well even with bloatware. When you trim out and make your JAR's lean and mean then its utterly amazing.
This attitude is a bit surprising to me. The main point of micro-services is that they address a complexity problem when dealing with large organizations. That is, they allow a large organization to break into small teams that can work (relatively) independently so each team can iterate faster. However, microservices definitely make a host of issues harder:
* cross-service transactions are much harder
* cross-cutting concerns can be more difficult to change.
* can add organizational complexity of a feature you want to add needs corresponding changes it up or downstream services.
Even Martin Fowler has this quote:
Don’t even consider microservices unless you have a system that’s too complex to manage as a monolith. The majority of software systems should be built as a single monolithic application. Do pay attention to good modularity within that monolith, but don’t try to separate it into separate services.
Micro-services allowed us to break this. Micro-services also allowed us a faster turnaround time in feature delivery and reliability. It is also easier to isolate problems.
Generally stuff in the monolith that are already abstracted by large service facades are a good candidate for a separate service with their independent data model. Avoid cross-service transactions completely. If you have a cross-cutting concern, it generally means you need a separate service managing that cross-cutting concern.
Organisational complexity is definitely increased. This can be mitigated by tooling. Our build pipeline shows the full graph dependency chain, what is built, what is getting built, what has been deployed, etc.
We have the concept of a "system" that is basically a versioned set of micro-services running off a build trigger. We developed the capability to namespace systems - ie each system of services uses separate resources (kafka/db/etc) and separate URL's (via custom domains) when deployed on our cloud platform. You can also "plugin" your micro-service into a targeted system for diagnostics. This way dev, testing and product demo teams can work independently. The latter is not micro-service best practice, but in a large, slow-moving organisation, we found it valuable.
We also have a distributed dev team, but decided against using microservices. Instead, we have a monolith with a plugin architecture, so remote teams can just add independent modules to add functionality. Occasionally changes are made to the monolithic application, and these changes are heavily scrutinized. The plugin architecture provides many of the benefits of a microservice, while also allowing for more flexibility on occasion, and eliminates flaky network calls that are inherit to microservices.
I agree with all the issues you've stated, but I'd like to add one more. Microservices arguably make it much easier to build systems with circulate dependencies leading to weird race conditions and deadlocks.
Consider two units of code A and B. If implemented as classes, modules, or libraries, it's relatively easy to spot and prevent A calling into B which in turn calls back to A. Sometimes the compiler and tools can automatically catch that.
With microservices, catching dangerous dependencies like this are much more difficult as each service, outwardly, seems independent of the other and there are few tools to catch these dependencies.
We had a pipeline consisting of microservices and Kafka topics. Simple if/then logic quickly became problematic so I implemented our flow control as a directed acyclic graph, and it helped tremendously.
It's also easy to render your graph out with any number of visualization tools to quickly understand/validate work flows.
I don't see how graph data structure solves this problem. Suppose you've created a photo sharing app. One microservice, A, has the graph database and stores the photos. One service, B, uploads and downloads photos. And a third service, C, applies filters to photos.
It's pretty easy to architect these services such that the download service B uses the filter service C in some situations, and the filter service C uses the download service B in others. This is obviously a bad design, but with microservices it's easier to make these bad design choices because the folks who wrote one service have little information about the other.
A tool to "fix" your example probably doesn't exist, but a graph is an excellent way to represent dependencies, reason about progress in the flow, and enforce constraints in a generic way.
I think people are just spinning up a bunch of unorganized services and calling them micro and then complaining about it.
If you're on a small team that can build a clean monolith, the work to make them microservices is imo pretty trivial.
It's still not comparable with Rails. Let alone some niche bleeding edge technologies which are focused on productivity.
I hear you brother. Google generally puts out the "smartest" stuff, but they don't really care about the day-to-day needs of most of their users. The idea is generally "we've thought about this a lot, and this is the best way, so change your code to handle it." The problem is that every few months, they seem to come out with a new "best way".
Yet, even with this stack, 50% of the spent time was due to internal process. As an example : we called them once to ask for some news, and they told us about doing a kickstart meeting. At this point we already had finished developping everything and wanted to talk about deployment (little did we know we would have to redevelop part of the stack because of guidelines we hadn’t been told about before).
Out of the four modern languages/frameworks that I’m fluent in: C++/TreeFrog (played around with it just because I am a masochist), C#/ASP, Python/Django and JS/Express, I really don’t see any difference in my productivity in any of them.
I think the tech itself has little to do with it. Cash-strapped small startups are hungry for developers, and often ask them to do more than they can. Large companies often have many developers idling or working on fake toy projects that never see the light of day. There are massive inefficiencies in large institutions. Using old tech may be one of them, but it's a drop in the bucket compared to inefficiently utilizing the people you have.
And yet. Despite the danger, that was a billion dollar business. So they were doing something right, weren’t they? ;)
I could use shiny new tech for the startup (which at the time was python on app engine and its nosql datastore, with a backbone.js framework), whereas the other one forced me to use java and deploy the code on the big corp datacenter.
The start up code was built in two months, and maintained by an intern for two years with success. It basically cost nothing to run.
The big corp code was built in a year (note: i did know how to develop in java), people in charge couldn't manage to deploy it (i had to do it myself, despite just discovering their infra on the spot), and i had to maintain the code myself, because despite huge code guidelines and restrictions on the technologies allowed, nobody inside was assigned to maintaining the code.
So, yeah, there's definitely a danger to absolutely wanting to follow the HN hype. But don't feel good if your company relies on 8yo tech and process. It could be loosing a TON of money by not staying up-to-date with the state of the art.