News for hackers. Stuff that matters.
Re:Software engineering topics I changed my mind on by PragmaticPulp on Saturday January 23, 2021 @08:50PM (#25888168)
As an engineer-turned-manager, I spend a lot of time asking engineers how we can simplify their ambitious plans. Often it’s as simple as asking “What would we give up by using a monolith here instead of microservices?”

Forcing people to justify, out loud, why they want to use a specific technology or trendy design pattern is usually sufficient to scuttle complex plans.

Frankly, many engineers want to use the latest trends like microservices or NoSQL because they believe that’s what’s best for their resume, even if it’s not necessarily best for the company. It doesn’t help that some companies screen out resumes that don’t have the right signals (Microservices, ReactJS, NoSQL, ...). There’s a certain amount of FOMO that makes early-career engineers feel like they won’t be able to move up unless they can find a way to use the most advanced and complex architectures, even if their problems don’t warrant those solutions.

Re:Software engineering topics I changed my mind on by reader_mode on Saturday January 23, 2021 @10:13PM (#25888764)
>Forcing people to justify, out loud, why they want to use a specific technology or trendy design pattern is usually sufficient to scuttle complex plans.

Does that really work ?

Usually these guys read the sales pitch from some credible source. Then you need to show them that the argument is X works really well for scenario Y but your scenario Z is not really similar to Y so reasons why X is good for Y don't really apply. To do this you usually rely on experience so you need to expand even further.

And the other side is usually attached to their proposal and starts pushing back and because you're the guy arguing against something and need a deep discussion to prove your point chances are people give up and you end up looking hostile. Even if you win you don't really look good - you just shut someone down and spent a lot of time arguing, unless the rest of the team was already against the idea you'll just look bad.

I just don't bother - if I'm in a situation where someone gives these kind of people decision power they deserve what they get - I get paid either way. And if I have the decision making power I just shut it down without much discussion - I just invoke some version of 'I know this approach works and that's good enough for me'.

Re:Software engineering topics I changed my mind on by ncann on Saturday January 23, 2021 @11:20PM (#25889209)
Yeah, god helps if a higher up is a zealot about a technology. They will try to suggest that at every opportunity and arguing against it makes you stand out like a sore thumb that after a while you wonder why you even bother.
Re:Software engineering topics I changed my mind on by Chris_Newton on Sunday January 24, 2021 @02:33AM (#25890123)
Frankly, many engineers want to use the latest trends like microservices or NoSQL because they believe that’s what’s best for their resume

The sad thing is, they might well be right.

People used to not get hired for a job involving MySQL because their DB experience was with Postgres, but usually more enlightened employers knew better. Today, every major cloud provider offers the basic stuff like VMs and managed databases and scalable storage, and the differences between them are mostly superficial. However, each provider has its own terminology and probably its own dashboard and CLI and config files. Some of them offer additional services that manage more of the infrastructure for you one way or another, too. There is seemingly endless scope for not having some specific combination of buzzwords on an application even for a candidate and a position that are a good fit.

I don’t envy the generation who are applying for relatively junior positions with most big name employers today, and I can hardly blame them for the kind of job-hopping, résumé driven development that seems to have become the norm in some areas.

Re:Software engineering topics I changed my mind on by u678u on Sunday January 24, 2021 @09:13AM (#25892163)
Agreed, I found it really hard to get good roles 5 years ago. Then I worked on some cool shiny stuff - in general I dont like microservices, k8s, React/JS but it opens a whole new world of jobs.
Re:Software engineering topics I changed my mind on by arvinsim on Saturday January 23, 2021 @10:02PM (#25888688)
> As an engineer-turned-manager, I spend a lot of time asking engineers how we can simplify their ambitious plans. Often it’s as simple as asking “What would we give up by using a monolith here instead of microservices?”

Funny you mentioned this. I have the exact opposite problem.

That is, I am an engineer trying to push back against management mandating the use of microservices and microfrontends because they are the new “hot” tech noawadays.

Re:Software engineering topics I changed my mind on by Twisol on Saturday January 23, 2021 @10:20PM (#25888813)
On my reading, this is the exact same problem, not the exact opposite problem. The break-even bar for a reasonable monolith is a lot lower than for microservices, so the GP's question is specifically asking, under a hypothetical where the team simply uses a monolith, what benefits the team would miss out on relative to microservices. If there are none, or they aren't relevant to the project scenario, then microservices probably isn't justifiable.

(I, too, am in the position of pushing back against microservices for hotness' sake.)

Re:Software engineering topics I changed my mind on by pseudalopex on Saturday January 23, 2021 @11:12PM (#25889156)
The point is it can be engineers pushing back against managers. Not just managers pushing back against engineers.
Re:Software engineering topics I changed my mind on by Twisol on Sunday January 24, 2021 @12:35AM (#25889602)
Aha! Opposite in that direction; I misread. (I'm also an engineer pushing back on managers who want microservices.)
Re:Software engineering topics I changed my mind on by Jcampuzano2 on Saturday January 23, 2021 @10:30PM (#25888880)
This. I'm a consultant and 90% of the time the technology has already been decided by our fancy management team who haven't written code in 10+ years before a line of code has been written. But they know the buzzwords like the rest of us and know they sell.

Problem is they no longer have to implement, so they are even more inclined to sell the most complicated tech stack that have marketing pages claiming they scale to basically infinity.

Re:Software engineering topics I changed my mind on by alexb7c4 on Sunday January 24, 2021 @05:40PM (#25896946)
In my company we store financial data for hundreds of thousands of clients in sql db. It's decade okd system and we have hundreds of tables, stored procedures (some touching dozen+ tables) and rely on transactions.

It took me weeks to convince my managers not to migrate to new hot nosql solution because "it's in cloud, it's scalable and it also supports sql queries".

Re:Software engineering topics I changed my mind on by busterarm on Saturday January 23, 2021 @10:50PM (#25889028)
> Frankly, many engineers want to use the latest trends like microservices or NoSQL because they believe that’s what’s best for their resume, even if it’s not necessarily best for the company.

Probably nobody is using NoSQL for their resume. It's because picking a relational database, while usually the correct choice, is HARD when you're operating in an environment that changes quickly and has poorly defined specifications.

When you start seeing engineers have difficulty reasoning about what the data model should be and nobody willing to commit to one, it's the clearest sign that organizationally things are sour and you need to start having very firm and precise conversations with product.

Re:Software engineering topics I changed my mind on by potta_coffee on Saturday January 23, 2021 @10:56PM (#25889055)
I'm facing this issue now. App is supposed to deliver to clients after this sprint - and the data model still isn't locked down. After arguing through about 10 hours worth of meetings this week, I think I need a new job.
Re:Software engineering topics I changed my mind on by theshrike79 on Sunday January 24, 2021 @06:57AM (#25891335)
The best thing about the lockdown is that you can apply, interview and change jobs without leaving your desk at home =)
Re:Software engineering topics I changed my mind on by busterarm on Saturday January 23, 2021 @11:09PM (#25889125)
My condolences.
Re:Software engineering topics I changed my mind on by Shorel on Sunday January 24, 2021 @12:03AM (#25889446)
There are only two options for me: MySQL or Postgres.

And using AWS generally means using Aurora. Then the choice is already made. Not hard at all.

Yep, my work involves heavy use of SQL and I find it better than the NoSQL insanity.

Re:Software engineering topics I changed my mind on by orobinson on Sunday January 24, 2021 @01:59AM (#25889947)
Just curious, for what reasons would you choose MySQL over Postgres?
Re:Software engineering topics I changed my mind on by kenhwang on Sunday January 24, 2021 @02:41AM (#25890162)
Personally, when I want speed or easy upkeep and intend on doing dumb simple things.

Postgres is more featureful, but if you don't intend on using those features, MySQL is consistently faster and historically smoother to update and keep running.

Re:Software engineering topics I changed my mind on by busterarm on Sunday January 24, 2021 @02:43AM (#25890181)
Also in the Enterprise, if you're doing a lot of sharding and replication across networks, Percona MySQL is a very compelling product. I say that as a Postgres diehard.
Re:Software engineering topics I changed my mind on by twic on Sunday January 24, 2021 @01:29PM (#25894373)
> MySQL is consistently faster

Unless you want to do a join.

Re:Software engineering topics I changed my mind on by xorcist on Sunday January 24, 2021 @12:00PM (#25893545)
Traditionally it was because you needed replication or sharding that you didn't have to boil half an ocean for, or at least half decent full text indices. These days however I believe the differences are smaller and in other areas.

Most often, you choose a database because of what you application supports and is tested with, not the other way around. Or what your other applications already use. Complete green fields aren't all that common.

Re:Software engineering topics I changed my mind on by Shorel on Monday January 25, 2021 @10:26AM (#25904214)
Replication.

And our DBAs are already familiar with the gotchas of MySQL.

You just have to write queries in a different way (subqueries are slow, so they are to be rewritten as joins).

Re:Software engineering topics I changed my mind on by malinens on Sunday January 24, 2021 @02:57AM (#25890241)
Probably the same way sometimes people choose sqlite versus mysql- simplicity. There are many cons and pros for both of them!
Re:Software engineering topics I changed my mind on by ithrow on Saturday January 23, 2021 @11:05PM (#25889098)
It's because picking a relational database, while usually the correct choice, is HARD when you're operating in an environment that changes quickly and has poorly defined specifications.

Wouldn't this apply if you are using a static typed language too? what's harder about changing the schema in the DB?

Re:Software engineering topics I changed my mind on by tikhonj on Saturday January 23, 2021 @11:11PM (#25889141)
You (mostly) don't have to deal with data migrations with statically typed languages. Releasing a new version of some code is usually easier than making structural changes to a database that's in active use.
Re:Software engineering topics I changed my mind on by Chris_Newton on Sunday January 24, 2021 @06:13AM (#25891102)
Releasing a new version of some code is usually easier than making structural changes to a database that's in active use.

Yes, and on top of that, code-only changes need to be internally consistent to make sense but DB schema changes almost inevitably require some corresponding code change as well to be useful. Then you have all the fun of trying to deploy both changes while keeping everything in sync and working throughout.

Re:Software engineering topics I changed my mind on by busterarm on Saturday January 23, 2021 @11:09PM (#25889122)
You've hit on something there as well, but essentially it comes down to forced rewrites and flexibility. We tend to choose the more flexible systems to avoid forced upfront work when changes are needed even when it's the wrong choice for the project in the long run.
Re:Software engineering topics I changed my mind on by Falkon1313 on Sunday January 24, 2021 @04:05AM (#25890513)
Eh, not so much databases, but in terms of code, super flexible 'all things to all people' generic abstractions tend to be a lot more work and a lot more difficult to debug than a tight solution tailored to the problem it's solving, written in domain terminology.

If only I had a nickel for every hour I've spent debugging abstractions and indirection that were just there for the sake of adding flexibility that would never be needed.

Re:Software engineering topics I changed my mind on by busterarm on Sunday January 24, 2021 @11:03AM (#25893073)
I'm in agreement with you actually. I was badly suggesting that that kind of flexibility up front bites us in the ass later and it's a bad impulse often followed.
Re:Software engineering topics I changed my mind on by souprock on Sunday January 24, 2021 @05:39AM (#25890937)
Is every NoSQL database non-relational, and every relational database SQL? It sure seems to me that you could have a relational database without SQL. Something non-text could be nice. It might be a binary data format or compiled code.

One might even convert SQL to binary query data at build time, with a code generator. It could work like PIDL, the code generator Samba uses to convert DCE/RPC IDL files to C source with binary data. Binary data goes over the wire. Another way is that both client code and server code could be generated, with the server code getting linked into the database server.

Re:Software engineering topics I changed my mind on by eecks on Sunday January 24, 2021 @06:33AM (#25891212)
Which parts are hard?
Re:Software engineering topics I changed my mind on by blabitty on Saturday January 23, 2021 @09:00PM (#25888242)
I like to half jokingly assert that microservices are a pysop to sell cloud hosting
Re:Software engineering topics I changed my mind on by nitrogen on Saturday January 23, 2021 @09:53PM (#25888639)
If I were an evil tech giant, I would open source a bunch of libraries that require significantly more effort to use than necessary, and pitch them as the One True Solution. Just to slow my competitors down.
Re:Software engineering topics I changed my mind on by specialist on Saturday January 23, 2021 @10:44PM (#25888980)
I had a nemesis who would steal all my ideas.

So I bought all the XP books, dog eared them, left them on my desk. My team nearly mutinied. I asked them to wait and see. Two weeks later, nemesis announced his team was all in for XP, Agile, pair programming, etc.

They never recovered, didn't make another release.

I tossed my copies, unread.

Re:Software engineering topics I changed my mind on by potta_coffee on Saturday January 23, 2021 @10:58PM (#25889068)
I'm praying this story is true, very funny.
Re:Software engineering topics I changed my mind on by pestaa on Sunday January 24, 2021 @05:57AM (#25891019)
With a competent team, XP should work really well, right? So what happened?
Re:Software engineering topics I changed my mind on by kenhwang on Sunday January 24, 2021 @02:43AM (#25890179)
Ah, I see you've used k8s.
Re:Software engineering topics I changed my mind on by twic on Sunday January 24, 2021 @01:30PM (#25894387)
Not a joke. When i worked at Pivotal Labs, sales / executives were very excited about the synergy between helping clients build microservice architectures and selling them Cloud Foundry.
Re:Software engineering topics I changed my mind on by naikrovek on Saturday January 23, 2021 @09:06PM (#25888283)
I bet you're half-right as well.
Re:Software engineering topics I changed my mind on by Gibbon1 on Saturday January 23, 2021 @10:13PM (#25888762)
I've long thought that Java write once, run anywhere, really was a desperate attempt to save Sun's legacy server market from doom.
Re:Software engineering topics I changed my mind on by jiggawatts on Sunday January 24, 2021 @08:03AM (#25891692)
> “What would we give up by using a monolith here instead of microservices?”

I love that.

I think many people are loathe to "turn the argument around", and pretend they're going the other way.

For example, imagine some legacy app used by 10 people out of 10,000 is incompatible with something like a Microsoft Office upgrade.

In many organisations, the argument goes like this: "We can't upgrade to Office 2023 because StupidApp will break!"

Turning that around: "If Office 2023 was already rolled out, would you roll that back to Office 2021 just for StupidApp?"

Re:Software engineering topics I changed my mind on by Aeolun on Saturday January 23, 2021 @10:02PM (#25888683)
> Frankly, many engineers want to use the latest trends like microservices or NoSQL because they believe that’s what’s best for their resume

Then they are bad engineers. It is true that it’s best for their resume, but I also have my professional integrity to maintain.

Re:Software engineering topics I changed my mind on by Ma8ee on Tuesday January 26, 2021 @05:44AM (#25914359)
That integrity isn’t worth much if you can’t get hired.
Re:Software engineering topics I changed my mind on by alkonaut on Sunday January 24, 2021 @11:15AM (#25893150)
The question one should ask is: what $hot_tech can we adopt without the product becoming significantly worse than with $old_tech? That is, which things do we adopt only or mostly to make the project or company attractive to work with or invest in?

Saying “why would you ever do that rather than building the solution the cheapest and with the lowest risk” doesn’t fully appreciate the importance of attractiveness.

I’m selling my hours of work time for a salary, fun, and resume points. My employer pays me in all 3. I’ll always push for $fun_tech or $hot_tech despite it not always being in the short term interest of anyone but myself or my fellow developers. I’ll keep justifying this by “if we do this in $old_tech then me and half the team will leave, and that’s a higher risk than using $new_tech”.

(By tech I here mean things like languages and frameworks not buzzwords like microservices or blockchain, ai... )

Re:Software engineering topics I changed my mind on by mrisoli on Sunday January 24, 2021 @11:22AM (#25893204)
IMO it's a no-brainer to choose complex technology, the incentives are much more attractive.

Go the simple way and it works, you get paid and off to the next project, if it doesn't work, its your fault.

But on the job experience with scaling tech is way more valued than doing some online course, you get paid to learn by doing on company time, and you don't lose anything by possibly wasting company resources. So you tick the box that often job postings have "proven track record of <insert scale buzz here>" which could possibly lead to a much better salary, its all incentives.

Re:Software engineering topics I changed my mind on by KaoruAoiShiho on Sunday January 24, 2021 @09:23AM (#25892227)
To me this is a little bit weird, because while OP is totally correct in that monoliths are totally fine too when it's the best tool for the job, the default should still be microservices. It's not really harder to use once you have the practice in place and advantages will usually be quite visible in time. But of course there are times when there are great monoliths you can just use and you should use them.
Re:Software engineering topics I changed my mind on by hu3 on Sunday January 24, 2021 @10:02AM (#25892576)
There are challenges with microservices that hints me to build monoliths by default unless not viable.

Things that are trivial in monoliths are hard in microservices like error propagation, profilling, line by line debbugging, log aggregation, orquestration, load balancing, health checking and ACID transactions.

It can be done but requires more complex machinery and larger teams.

Re:Software engineering topics I changed my mind on by closeparen on Saturday January 23, 2021 @10:49PM (#25889016)
Do you mean “a monolith” or “the monolith?” The essential characteristic of monoliths is that you don’t get to start new ones for new projects.

The real skill of architecture is understanding everything your company has built before and finding the most graceful way to graft your new use case onto that. We get microservices proliferation because people don’t want to do this hard work.