News for hackers. Stuff that matters.
Re:Microservices: Why Are We Doing This? by glouwbug on Monday March 21, 2022 @05:06PM (#30759721)
The last microservice architecture I worked on consisted of 7 python repositories that shared one "standard library" repository. Something as simple as adding an additional argument to a function required a PR to 7 repos and sign off on each. When it came to release we had to do a mass docker swarm restart and spin up because the giant entanglement of micro-services was really just a monolithic uncontrolled Cthulhu in disguise.

The business revolved around filling out a form and PDF generations of said form. I felt like I got no work done in a year and so I left

Re:Microservices: Why Are We Doing This? by pkulak on Monday March 21, 2022 @05:20PM (#30759834)
So... you worked on something terrible that people called a "microservice architecture"? Once a pattern gets popular, people start writing nasty code in the style, and then the pattern takes the reputation hit and people move on to the next thing (or just back to the last thing). Rinse, repeat.

My company uses microservices; deploys restart one service and PRs are one repo at a time. There's a shared library, but it's versioned and there's nothing compelling you to keep on the bleeding edge.

Re:Microservices: Why Are We Doing This? by jmchuster on Monday March 21, 2022 @06:54PM (#30760814)
Is this something a lot of people are missing with their microservice implementation? You need to be able to deploy each microservice independently of everyone else. If you have a change that's spread across multiple services, you should be able to just do them one-by-one in order. If you want to rollout a shared library, you should be able to just update your services one-by-one. If a coordinated rollout is required, then doesn't that kind of defeat the whole point of doing microservices?
Re:Microservices: Why Are We Doing This? by dragonwriter on Monday March 21, 2022 @07:10PM (#30760932)
> If a coordinated rollout is required, then doesn't that kind of defeat the whole point of doing microservices?

If a coordinated rollout is requires for anything but a change of service API (and then only the service and it's direct clients should be impacted, and even then a decent deprecation policy should eliminate the need for close coordination), you aren't doing microservices, because loose coupling is part of the definition of the pattern.

Re:Microservices: Why Are We Doing This? by Gigachad on Monday March 21, 2022 @07:53PM (#30761256)
Its a big anti pattern I have seen that a change requires simultaneous merges on multiple repos and there is basically no way to avoid possible outages/broken systems in the middle period while its deploying.
Re:Microservices: Why Are We Doing This? by dragonwriter on Monday March 21, 2022 @07:07PM (#30760907)
> Once a pattern gets popular, people start writing nasty code in the style,

Once it gets popular, people start “implementing the pattern” based on rumor, bad descriptions from unqualified (and sometimes ill-motivated) intermediaries, and some boss and/or tech lead’s fever dreams, and blaming it on the pattern.

(If the pattern is useful enough, it will be later invented under a new name to escape from the accumulated cruft, which will briefly succeed only to succumb to the same process until another cycle passes.)

Re:Microservices: Why Are We Doing This? by throwaway6532 on Monday March 21, 2022 @11:18PM (#30762372)
It there some sort of technical name for this phenomenon? Maybe like "pattern decay" or "technical decay"? From what I have observed in all throughout my career this happens to every pattern eventually. People that come up with it obviously understand it themselves, but they have to serialize their understanding into some format like a conference talk or a book, which is inherently lossy as it's not an implementation, people in the wider community then deserialize the original understanding and (try) build an implementation out of it thus providing a myriad of serialized (but wrong) understandings out in the wild that are then encountered by subsequent waves of engineers who then take the pattern to have a new definition than the originally intended one.
Re:Microservices: Why Are We Doing This? by drwiggly on Tuesday March 22, 2022 @11:12AM (#30767036)
Its just cargo culting. A lot of programming advice/processes are fuzzy in a wider context. Generally they came to be in a specific context. Relating that context is hard. Eventually its picked up as a fad and spread around, just like a game of telephone we lose nuance and context. Pair this with imposter syndrome and low confidence, everyone tries to cover their ass and do what the current hype in the industry is doing.

What if I suggested an architecture where every 5th call had the overhead of serialization and network time. Also the reliability and concern issues that go with multi machine calls. Data syncing issues and network partition concerns.

Pretty sure most people couldn't reason about this system. They'd suggest instead of having network calls on every 5th call, lets really look at our use case and only insert them where we may have load issues and have to scale. Data state becomes a big concern, make sure we know where our data is at all times and is only passed when needed to be correct and not over pass it to be efficient.

Re:Microservices: Why Are We Doing This? by throwaway6532 on Wednesday March 23, 2022 @04:51PM (#30783183)
Cargo culting. Of course. Derp! Why did that not immediately spring to mind???
Re:Microservices: Why Are We Doing This? by root_axis on Monday March 21, 2022 @06:07PM (#30760344)
> I worked on consisted of 7 python repositories that shared one "standard library" repository. Something as simple as adding an additional argument to a function required a PR to 7 repos and sign off on each

This is an engineering process failure, not a failure of microservices or shared dependencies. You should be versioning your shared library, that way you only need to make a deployment to the service that requires the update, leaving the others pegged at the previous version until a business or engineering need motivates the upgrade.

Re:Microservices: Why Are We Doing This? by paskozdilar on Monday March 21, 2022 @06:22PM (#30760476)
While we're at the topic of engineering process failures - is there any research/literature about software engineering in general that describes all the hows and whys of large-scale programming like that?
Re:Microservices: Why Are We Doing This? by paskozdilar on Monday March 21, 2022 @06:17PM (#30760437)
I have had similar experience building microservices that used shared repositories. The PR paperwork was so bad that at one point I've made all my services self-contained, just to avoid having to modify my own code in two different places and synchronize the changes.

The whole problem, I think, comes from the "split the code" cargo-cult. We need to think about why we're splitting the code, and use that why to figure out when to split code.

IMHO, code separation arises naturally from modular programming - once your code is mature enough, it becomes just a piece of glue around a set of libraries that you can just rip out and put in their own repos, provided that they're useful enough.

Re:Microservices: Why Are We Doing This? by AndyPa32 on Monday March 21, 2022 @07:28PM (#30761056)
So, then you have been working on a distributed monolith and not a proper microservice architecture. Just saying.
Re:Microservices: Why Are We Doing This? by abledon on Monday March 21, 2022 @05:53PM (#30760193)
but boy could your pdf generator system scale. Boy could it scale...
Re:Microservices: Why Are We Doing This? by ris on Monday March 21, 2022 @08:25PM (#30761483)
Except in my experience most attempts at microservices scale extremely poorly because they have nowhere near enough precision in their APIs to get just the information you need and so most applications end up fetching way, way more data than they need and whittling it down manually - and often having to stitch together the results from multiple calls.

That of course leads you down the path of creating an "oddly specific" API that caters exactly to the needs of the clients, down to the point that small changes in the client's needs require changes in the upstream service(s) - so your "separation of concerns" has become a joke.

Re:Microservices: Why Are We Doing This? by lIIIllllIIII on Monday March 21, 2022 @10:51PM (#30762251)
Is it a common thing for "proper" microservices to have microservices retrieve data from other services via APIs?

I've worked on "microservices" systems before that do this. They were mostly shit.

The system I'm working on at the moment has each service subscribe to events and maintain its own database. The only comms is via events. It works pretty well and everything really is nicely separated, but it feels a bit haphazard.

Re:Microservices: Why Are We Doing This? by abledon on Monday March 21, 2022 @05:10PM (#30759753)
One upside in dealing with this mess(and understanding all the intricacies, pains etc..) were in is being able to laugh at the kafkaesque situations people find themselves in now, like your story or other blogposts.
Re:Microservices: Why Are We Doing This? by willhoyle on Tuesday March 22, 2022 @08:30AM (#30765228)
This is blowing my mind. I felt like you were specifically talking about my place of work. Turns out (based on your github link in your profile), you did work there haha!

The 7 repos are gone now and a k8s cluster has replaced the swarm service. Deployments are a bit easier to manage now. It's still such slow development process to add a simple API endpoint, especially if that API has to call other DAOs. It's crazy because I feel like it's completely normalized here that dev work takes 300% longer than it should for simple features.

Re:Microservices: Why Are We Doing This? by umpalumpaaa on Monday March 21, 2022 @05:10PM (#30759751)
Have you considered not sharing code between services?
Re:Microservices: Why Are We Doing This? by glouwbug on Monday March 21, 2022 @05:17PM (#30759800)
Yes, I came to learn that microservices would probably greatly excel at sending JSON strings back and forth. Then I wondered why we weren't just all using Erlang
Re:Microservices: Why Are We Doing This? by umpalumpaaa on Tuesday March 22, 2022 @03:19PM (#30770273)
I don’t understand why I got the downvotes…

I was taught that sharing code between services should only be done in circumstances in which there is already a strong connection between them - for example by being owned by the same team.

In the organization where I initial learned how to deal with microservices even direct calls via HTTP between services was a no-go and only used in rare circumstances and if then only temporary.

I am not sure if this is the way to do it since I have a sample size of one - but we had about 500 services to manage and in the end it worked okay and I never saw something you described. That’s why I wanted to know if you have considered just not sharing code.

Re:Microservices: Why Are We Doing This? by archduck on Wednesday March 23, 2022 @02:43AM (#30775673)
Regarding why you got the downvotes, I have a guess: "Have you (maybe) considered not ..." usually gets followed by something that's transparently stupid or incompetent or otherwise undesirable in the given context. For example: "Have you considered NOT publicly yelling at your coworkers?" Obviously that's not what you were saying, but it may have read like it.

It's a fantastic INDIRECT way of telling someone they did something stupid. And because stupid actions map so well onto their doers, it's also a fantastic way to insult people.

As an aside: this sort of indirection - asking a question to assert something - it's not peculiar to English, but it's suuper common in English-language expressions (which is, in fact, due to the habitual indirection of the English - at least according to a linguistics professor I once had.)

The kicker is: because it's an ironic construction, it’s likely going to require more effort to process: the presence of irony means that what is actually meant by the speaker is not encoded literally, but rather must be interpreted - derived from what WAS said - and irony (and other ways of flouting literal language) yields a disjunction (x || y || z). It puts the listener into a role where their recognition of intent and ability to joke around are tested - and the correct interpretation is up in the air. So while it's possible to say this expression to a friend when they screw up ("Have you considered NOT showing up to work drunk?"), it's also a great way to make an enemy ("Have you considered NOT being a fuckup that nobody likes?"). Same construction. (In fact, this later example is especially sinister, because any answer you give just makes YOU assert what you're being slandered with. "No" - I haven't considered not being a fuckup? / "Yes" - I did in fact consider not being a fuckup? You get the idea. It can be wielded in a very mean way.)

And your comment wasn't intended this way at all, and in fact it's obvious when you read it that straight talk and no irony is meant, but it shares enough of the signification with the ironic phrase that it sets off alarm bells. Especially in an online reply to someone, where trolling is sport! The context is working against you here.

And the context is the first tool we have when interpreting communication, because it's already available to our cognition before we receive any given message. And brains are energy misers, using heuristics to filter shit out that doesn't look like a good reward for the necessary energy expenditure.

So I bet that the downvoters saw 1) a short comment, and short is by the way much more likely to be low effort and unconstructive; and 2) all the native speakers' downvoting brains instantly recognized the same construction as the ironic idiom talked about above, and then they instantly said fuck it, this person is being a dick, without even really reading it. Since irony (unless it’s expected) requires more processing effort (both in detecting WHETHER irony is present AND what the ironic speaker means, just to get to the point where you can start assessing the semantic content), these two things were enough to justify jumping the interpretive gun. I wouldn't be surprised if your question was flagged by MOST native speakers’ brains’ asshole detection systems. And so, the actual substance of your question, which, processed in its entirety, have indicated that you were asking a bona fide, relevant, even microservices-essential question - got booted out. And then the overzealous downvoters succumbed to the urge to smash the downvote button without a second thought and never looked back.

I found myself in the same place, actually - thinking "they're being a dick," then moving on. Probably took less than a second. Often when people in online conversations say why the downvotes? they're trolls acting in bad faith, which pisses me off because I always go reread what they wrote. So I did. You weren't calling anyone stupid! Still, I had to stare at it a bit before I realized it was an unintentional collision with the syntax of sarcasm. Maybe like if I wrote "Was hast du denn?" and really truly meant "what do you have?" instead of "what's wrong with you?" If you don't recognize it for what it is, it might look like an honest question which expects an honest answer.

Oh. Another thing to consider: after a while spent in any given semiotic context, our cognition adjusts and those cognitive heuristics that reject hard things become even lazier. Our pattern of behavior when reading on screens starts with reading, but then it turns to skimming and scanning if given long enough. I'd bet money that if your brief, but bona fide contribution had been one of the first comments, your downvote percentage would be a LOT lower. Halfway down a long page, the skippers are skipping and the downvoters downvoting. That doesn't mean that they intended to or are even aware of the switch in reading modes. At the top of the page, information is still shiny and new, and worth interpreting, speculating on the speaker and their intentions, etc. After a while, nobody gives any comment a second chance.

Anyway, I don't get to revisit all the shit I learned at university enough, so it felt good to write all that. Are you still here? Maybe everyone's already moved the fuck on. Brain got bored? Would serve me right, ha! So, here I go, back to my job, where instead of working in cognitive linguistics, text semiotics, pragmatics, philosophy of language, etc., I'm writing boring-ass JavaScript all day, for an application ... being built on microservices.

Re:Microservices: Why Are We Doing This? by umpalumpaaa on Wednesday March 23, 2022 @05:29PM (#30783664)
Such a nice and enlightening explanation. Thank you.