Re:Kubernetes Failure Stories
by wildermuthn on Friday February 12, 2021 @09:41AM
(#26114091)
I think you inadvertently explained why Tolstoy is right: while there are many categories of failure, within those categories each failure is it’s own big, and even for the same bug there are often contextual reasons why the bug manifests in one scenario but not another. Yes, there are alcoholics, but every alcoholic has their own lonely story of descent into addiction.
There are infinite ways to fail, and only finite ways to succeed. That relative difference between infinite and finite is what Tolstoy is getting at. And this is a helpful perspective, because then success becomes less about trying to prevent infinite failure-points and more about doing a few finite things very well.
This is why you see advice from YC like — “don’t worry about competition”, and “talk to users, talk to users, talk to users.”
The problem is that everyone needs to learn those lessons on their own. (But I'm biased: I'm working on a troubleshooting tool for K8s which has remediation rules for all those common cases.)
The tool sounds interesting — can you share any more details?
Re:Kubernetes Failure Stories
by MichaelMoser123 on Thursday February 11, 2021 @07:46PM
(#26109550)
it's the too many moving parts and black boxes blues.
I guess it depends on your definition of "at scale" but IMO with Hashicorp's stuff and maybe something like Backstage you get all the benefits k8s but in a package that's much simpler to reason about and manage.
By building a neat box with few buttons on the outside and still the same or more complexity on the inside. If something goes wrong, you still have to deal with all the inside parts. The only hope is that the few outside buttons only trigger happy paths without complications. "At scale" this seems unlikely, simplifications usually only work in smaller, simpler use cases.
To tie it back to Kubernetes, you have the scheduler-does-not-schedule lessons, the ingress-not-consistent lessons, the autoscaler-killed-all-my-pods lessons, the moving-stateful-services-with-60TB-storage-size-took-20-minutes lessons and probably many more. It's like Churchills explanation of democracy: terrible, but better than all the alternatives. (at scale, at least)