He ponders why it is that big websites inevitably have kernel developers. Way out of their domain of expertise, right? If you adopt a technology, you're responsible for it.
When Kubernetes inevitably has an issue that is a blocker for us, I don't have confidence in my ability to fix it. When an internal python or shell program has an issue that is a blocker for us, I change it.
PostgreSQL is used by probably millions of people, but we've had to patch it on multiple occasions and run our fork in production. Nobody wants to do that, but sometimes you have to.
The point is, you can't just say "oh, we use kubernetes so we don't have to think about it". No. You added it to your stack, and now you're responsible for it. Pick technologies that you're able to support if they're abandoned, unresponsive to your feature requests and bug reports, or not interested in your use case. Pick open source, obviously.
This is another reason I like Nix. It's a one-line change to add a patch file to a build to fix an issue. So I can contribute a fix upstream to some project, and then I don't have to wait for their release process, I can use the patch immediately and let go of it whenever the upstream eventually integrates it. It lowers the cost of being responsible for third party software that we depend on.
I doubt that this happen in reality because Kubernetes cover uses cases for pretty much everyone, now I would doubt the knowledge of a regular dev to try to mimic a solution that k8s already does.
Does that matter if the current stack works just fine?
Imagine someone who built a calculator app with jQuery javascript 10 years ago, and all it does is add and subtract numbers. You could spend time porting it to Ember, and then migrating to Angular, and then porting it to React, and then porting it to React with SSR and hooks.
If the calculator app worked with 15 year old code, you can either leave it be, or try to keep up with latest tech trends and continuously refactor and port the code.
I think there are a lot of "calculator app" type components of many systems that, at a certain point, can be considered "done, complete" without ever needing a rebuild or rewrite. Even if they were built on technologies that might now be considered antiquated.
Which they don't, because nothing has convinced me of the crappiness of some of my code more thoroughly than watching other people try and fail to use it. They are a mirror and people don't always like what they see in it. That guy who keeps talking trash about how stupid everyone else is for not understanding his beautiful code is deflecting. If they're right then my code does not have value/isn't smart which means I don't have value/am not smart, so clearly the problem is that I'm surrounded by assholes.
I don't think we would be using as much 3rd party software without StackOverflow, and almost nobody creates a StackOverflow for internal tools. Nobody writes books for internal tools. Nobody writes competitors for internal tools which either prove or disprove the utility of the original tool. All of those options are how some people learn, and in some cases how they debug. You're cutting off your own nose when you reinvent something and don't invest the time to do all of these other things. If I lose someone important or get a big contract I didn't think I could win, I can't go out and hire anyone with 3 years experience in your internal tool, trading money for time. I have to hire noobs and sweat it out while we find out if they will ever learn the internal tools or not.
The older I get the more I see NIH people as working a protection racket, because it amplifies the 'value' of tenure at the company. The externalities created really only affect new team members, delaying the date when you can no longer dismiss their ideas out of hand. You have in effect created an oligarchy, whereas most of the rest of us prefer a representative republic (actual democracy is too many meetings).
Some of what I said is a performance piece. If I insist that I am human and thus make mistakes, then it makes space both for others to make mistakes and for us to have a frank conversation of how, in our more lucid moments, we can do things that fight/counteract our basic nature.
It's morally equivalent to putting a plastic cover over the Big Red Button so that pushing it becomes a conscious and considered act. Deciding to push the button is step 1, moving the cover is step 2, and step 1.5 or 2.5 is realizing this is in fact not what you want to do after all, instead of a momentary bit of inattention, bad judgement, or the Imp of the Perverse.. It's not that I won't make mistakes, but that (hopefully) I a make them less often. But making them less often, I'm entrusted with bigger potential for mistakes, and so my mistakes tend to cause more harm. In a Precautionary Principle sense, we end up roughly within the same order of magnitude. When I make 5x fewer mistakes I'm allowed to be responsible for problems that are twice as risky.
If you want candor about mistakes you have to start by giving people permission to make them (or at least, to make new ones).
If I gave you a Dockerfile for a node.js service that installed its dependencies using npm, and the service used non-trivial dependencies, and I gave you this file 10 years after it was first written, chances are you would not be able to build a new image using that Dockerfile.
This would be a problem for you if you had to make some small change to that node.js service (without changing any dependencies).
For example Crunchy Postgres offers an enterprise supported Kubernetes Operator that leverages stateful sets. Yet I am reading in these comments that it is an unsolved issue.
It's very easy to do in Go.
You pin the version used to build your app ( the runtime version ) In your git repo you have dependcies
That's it.
People have been building this infrastructure since long before containers, thousands of servers in single operational clusters. Highly automated data infrastructure without containers is pretty simple and straightforward if this is your business.
https://www.redhat.com/en/resources/crunchy-data-and-openshi...
The great majority of Kubernetes experts I have met use a managed service and have not been exposed to the internals or the control plane back end. An abstracted version of Kubernetes that works well for most use cases but makes it more difficult to solve some of the problems listed in this thread.
Most folks dont really know Kubernetes. They know how to deploy X application using X abstraction (Helm, Operators, CD tooling) to some worker node in a public cloud using public sources and thats basically it. It's no wonder they cannot solve some of the common problems because they didnt really deploy the apps. They filled out a form and pushed a button.
The original Linux container design was never intended with this type of software in mind, and tacitly reduces the control a process has over its resources. Invariants required for performance are violated, and therefore performance suffers relative to bare metal (or conditionally VMs). The v2 container implementation, which isn't widely available yet, recognizes these problems and attempts to remedy some of these poor behaviors of the v1 container implementation.
If your database does not assume strict control of its underlying hardware resources then this does not affect you. However, this kind of strict resource control is idiomatic in all the highest performing databases architectures, to great effect, so it is the most performant databases that are most adversely affected by containerization.
FWIW, virtual machines used to have similar terrible performance for high-performance databases for similar reasons. Now there are mechanisms for the database to effectively bypass the VM similar to the kernel, and the loss of performance is minimal. Container environments typically don't offer similar bypass mechanisms.
> Containers only give reproducible deployments, not builds, so they would be a step down.
This is not true, if you use a docker image A with specific version to build then the result is reproducible.
That's what most people should be doing, you pin versions in your build image that's it.
> Kubernetes has mostly focused on stateless applications so far. That's the easy part
Kubernetes can run database, and stateless application is not a solved problem, I mean how does your services get restarted on your system, is it something that you had to re-create as well?