If there are several services all with the same share of CPU resources and with no configured limits, and they are all runnable, then none of them will be able to starve the others. The kernel will schedule them each in turn.

If you configure a static limit what you get is services that don't run even when there is CPU time available, which is bad.

Exactly! Most commenters piling on the idea clearly don’t know about cpu shares and how they are used in container runtimes.

The number of times I’ve seen CPU limits kill off pods during even mild spikes and causing pretty much downtime and “disaster” is just as surprising. Work on autoscaling nodes instead, don’t use cpu limits.

This advice is confusing. CPU is a "compressible" resource -- pods don't get killed for (trying to) exceed it. Pods don't get evicted from nodes based on CPU starvation, so autoscaling your node count won't help if you end up with a set of pods on a node that need more CPU than the node can provide. They'll just starve each other.

If your service allows horizontal scalability, you can use autoscaling of pods with Horizontal Pod Autoscaler (ideally also with a cluster autoscaler) to increase pod count for a given service when some percentage of the requested CPU is exceeded, whether or not you set a CPU limit. Setting the cpu_request appropriately for your pods is critical to ensure that node CPU is not oversubscribed by the Kubernetes pod scheduler.

Pods where mem & CPU requests = limits are given the highest class of service ("guaranteed"). For your most critical and latency sensitive services, this is the best approach when also coupled with HPA. Assuming a 4.19 kernel or later, I suppose.

https://medium.com/better-programming/the-kubernetes-quality...

Maybe I'm misunderstanding you, but I'm pretty sure CPU limits will only limit the amount of CPU used even if there is more available. It will not kill off the pod.

Memory limits however will kill the pod if the pod uses more than the limit.

It is possible to get into this state. CPU starvation can be so severe that containers start failing their liveness probes and are killed. This is obviously very different than things like memory limits where the kernel OOMKills you, but will look similar to the untrained observer. Their app is serving 503s and the containers are in a restart loop -- looks like a Kubernetes problem.

In general, the problem is that people don't understand how these complex systems interact -- what do limits do, what are the consequences of limits, how do you decide on correct limits, what do liveness and readiness probes do, what is the kubelet's role in the probes, wait what's a kubelet, etc.

That may be more likely with limits, but it doesn’t require a limit. I’ve had lots of fun with that in Elasticsearch pods with no limit. And then you get to enjoy a nice cascading failure.

This isn't exclusive to containers, you had this problem with VMs and even hardware. Back in the physical server days, when you didn't have enough CPU to service the current rate of inbound requests, eventually your healthchecks start to fail and load balancers would evict you from the pool. Now instead of a physical CPU, it sounds like you're running into a CPU limit implemented in the kernel but the same basic principles apply.

Kubernetes changes how you create the infrastructure but you still have the same problems you had in a distributed system before.

Been there, seen that, can confirm.

Of course it’s a horrible idea. I support kubernetes clusters, and I would fix the underlying issue not paste it over.