News for hackers. Stuff that matters.
Re:Etcd, or, why modern software makes me sad by pm90 on Tuesday July 14, 2020 @12:50PM (#23834360)
The apprehensiveness of sysadmins May have been justified in the world 5 years ago but today it sounds somewhat out of place. Note that critically evaluating new technologies is still an important skill and many infrastructure people I work with are extremely cautious about adopting new tech without spiking/getting to know it. But the sysadmins with the penchant for saying no is probably one of the reasons the devops movement actually kicked off: developers and infrastructure folks could see the immense productivity gains from being able to ship code quickly to production and got onboard the technologies that enabled this after being frustrated with the amount of time that traditional software deployment processes took.

So in today’s world, that kind of attitude is hardly productive. Skeptical? Absolutely. But open minded.

Re:Etcd, or, why modern software makes me sad by Nextgrid on Tuesday July 14, 2020 @01:08PM (#23834635)
> the devops movement actually kicked off

And this is why every single project out there is now a house of cards (or should I say house of YAML files) using insanely complicated technologies (like Kubernetes) with very "interesting" failure modes to say the least.

This attitude works today because of engineering-driven-development; the whole purpose of engineering is engineering and business priorities took a backseat in favor of buzzwords on the careers page and an obligatory "engineering blog" (describing how they solve self-inflicted problems), however when it comes to reliability and solving problems a large majority of projects can get away with much simpler, old-school technologies.

Re:Etcd, or, why modern software makes me sad by pm90 on Tuesday July 14, 2020 @05:34PM (#23838229)
Almost everything in your comment is wrong. Kubernetes has enabled a whole host of observability tooling (eg opentracing), promoted a culture where app logs are easily and always accessible, enabled 0 downtime deployments for teams without dedicate infra specialists and so much more. It has made deploying reliable applications a lot more easier than ever before.

Services today scale to handle a lot more users and traffic than they did not so long ago; and these reliability guarantees are the norm rather than an exception.

Re:Etcd, or, why modern software makes me sad by Nextgrid on Tuesday July 14, 2020 @06:01PM (#23838537)
Kubernetes does indeed have advantages but also brings a whole layer of complexity, overhead and moving parts. From my experience, in many cases the theoretical advantages don't end up being worth the tradeoff and/or don't even end up being implemented. Furthermore the particular things you mention (tracing, centralized logging & no-downtime deploys) can be done just as easily without Kubernetes.

I disagree about not needing dedicated infrastructure specialists. Kubernetes' complexity, learning curve and failure modes would make me uncomfortable operating without having a dedicated "devops" person (or sysadmin as we used to call them) while I am perfectly comfortable managing a few virtual machines (or even bare metal hosts) with a load-balancer in front of it. I recommend building systems in a way that can easily fit in your mind, and there's only so many abstraction layers and moving parts you can fit in there before you overload.

When it comes to scaling, not every application needs to scale and even when it needs to, it's trivial to scale stateless app servers without Kubernetes. You can scale quite far without Kubernetes, and when you're past that point you'll realize your main bottleneck is your data store and Kubernetes (or similar) can't magically solve that.

Re:Etcd, or, why modern software makes me sad by pm90 on Tuesday July 14, 2020 @07:08PM (#23839210)
Your answer to everything I said is “actually, it’s not that hard without kubernetes”. Maybe it’s not for you. For most developers, it absolutely is. And that’s why kubernetes is popular.

Fads don’t form out of thin air. There’s always some value that they provide. To someone experienced with setting up infrastructure, the tasks may seem trivial, and the value add is low. For others who don’t, having a dead simple way of easily adding tooling around their applications is a godsend. Why is this so fucking hard for you to understand?

Re:Etcd, or, why modern software makes me sad by Nextgrid on Tuesday July 14, 2020 @08:55PM (#23840009)
I am approaching this from a developer's perspective. Kubernetes and similar (even local Docker) introduces an extra layer of indirection that you often have to fight with. Sometimes it's worth it, sometimes it's not. For me, if my application is misbehaving in production, I prefer being able to just SSH into the machine and figure out what's wrong than fight with the container layer, its authentication system, command-line syntax just to obtain a shell inside the container. When I am developing locally, I prefer having all my files on the local filesystem instead of having to worry about volume redirection and "docker exec".

I am not an expert in setting up infrastructure by any means. In fact if I were I would probably use and promote these technologies. But in my opinion, adding another layer of abstraction doesn't magically solve the problems of the underlying stack (it won't protect you against obscure Linux kernel behavior, but now you have yet another moving part and potential variable which you need to account for when troubleshooting) but still gets in the way when you're trying to do something simple that doesn't even require any of the advantages the container technology is offering.

When it comes to "adding tooling around their applications", I am not sure what you mean but I will assume you refer to your previous examples, in which case I do not see how container technologies change the game at all. Tracing and centralized logging require your application to talk to a centralized log server (for logs, you can also output to stdout and have systemd/syslog collect and send them to the logging server) and container technologies don't change anything here.

I am not saying that container orchestration technologies provide no value. I am saying that they are often overkill for the task at hand and introduce extra complexity, moving parts and management overhead.

Re:Etcd, or, why modern software makes me sad by dijit on Tuesday July 14, 2020 @09:17PM (#23840156)
I'm an infrastructure type and everything you said rings true for me.

Kubernetes and abstractions of its ilk (shipping containers for example) have a place, but every abstraction comes with some form of trade-offs, be that performance or transparency.

Dealing with a node brown-out in kubernetes is much worse than dealing with a network, host or service outage because the troubleshooting steps involved evolve fractally.

That said, obviously there is value- but it's good to critically assess the value instead of just jumping in.

Re:Etcd, or, why modern software makes me sad by bandrami on Tuesday July 14, 2020 @12:51PM (#23834383)
Revisit this comment of yours in a decade.
Re:Etcd, or, why modern software makes me sad by pm90 on Tuesday July 14, 2020 @05:31PM (#23838190)
A decade is a long time in software engineering. Kubernetes would be obsolete by then. What do I need to revisit?
Re:Etcd, or, why modern software makes me sad by bandrami on Tuesday July 14, 2020 @06:21PM (#23838757)
It is and it isn't. I remember the first time I thought "this software is just X that everybody forgot about, reinvented again". That was 2005. Not coincidentally that was a decade after I started working as a sysadmin.

Venkatesh Rao says that there comes a point in your life when you realize things you thought were permanent are temporary, and things you thought were temporary are permanent; he uses "40" as a good rubric for that developmental stage. IT goes through many pendula, whether it be containerization vs. amalgamation, or thin clients vs. thick. What, over time, you learn to hold on to is the tools that have lasted decades and will probably continue to. Right now the pendulum is starting to swing back towards amalgamation, and it will probably swing back towards containerization again in another decade. Whatever the tech is at the time, it can be good to reconsider whether your views will have changed not on the technology, but on the larger pendulum it's riding.

Re:Etcd, or, why modern software makes me sad by pm90 on Tuesday July 14, 2020 @07:01PM (#23839150)
I find it hard to parse anything meaningful from your comment, sorry. I don’t mean to be rude, perhaps blunt.

Please name what infrastructure technology has lasted decades, I am curious to know. If you try hard enough yes you can see it’s all cyclical, you can say, MULTICS was the OG cloud. OK, fine, but what use is that to be as a practical software developer building things? Perhaps with age you start seeing everything as being like something you’ve already seen so it’s not that exciting anymore?

Re:Etcd, or, why modern software makes me sad by efficax on Tuesday July 14, 2020 @11:17PM (#23840836)
> A decade is a long time in software engineering.

Is it?

Code I wrote literally 20 years ago is still running in production, and I get paid to work on an app with plenty of code around that was first committed in 2009.

If you build a system in k8s today, and it's a success, there's good reason to believe you'll be on k8s in 2030.

Re:Etcd, or, why modern software makes me sad by pm90 on Wednesday July 15, 2020 @12:39AM (#23841379)
That’s pretty cool, really. I can’t really find studies on average software life time, so all I really have is anecdotes. I can’t imagine software running for that long on eg public clouds without any kind of major refactor considering how quickly technology changes today, and how aggressively cloud costs push companies to change. Specifically, I can see something like “Serverless” coming to maturity and completely changing the paradigm (again!) on how software is architected. But it’s possible I’m wrong and like you say, k8s will be around for much longer than I anticipate.