Re:Ask HN: Who operates at scale without containers?
by maxk42 on Tuesday March 22, 2022 @12:26PM
(#30768110)
Back in 2010 I built and operated MySpace' analytics system on 14 EC2 instances. Handled 30 billion writes per day. Later I was involved in ESPN's streaming service which handled several million concurrent connections with VMs but no containers. More recently I ran an Alexa top 2k website (45 million visitors per month) off of a single container-free EC2 insurance. Then I spent two years working for a streaming company that used k8s + containers and would fall over of it had more than about 60 concurrent connections per EC2 instance. K8s + docker is much heavier than advertised.
[ Parent
| on HN ]
Re:Ask HN: Who operates at scale without containers?
by danielrhodes on Tuesday March 22, 2022 @01:10PM
(#30768729)
Docker is far heavier - the overhead is the flexibility and process isolation you get. I imagine that's really useful for certain types of workloads (e.g. an ETL pipeline), but is crazy inefficient for something single purpose like a web app.
Re:Ask HN: Who operates at scale without containers?
by ryanjkirk on Tuesday March 22, 2022 @02:14PM
(#30769465)
Docker is heavier (and more dangerous) because of dockerd, the management and api daemon that runs as root. Actual process isolation is handled by cgroup controls which are already built into the kernel and have been for years. You can apply them to any process, not just docker ones.
However, Docker is essentially dead; the future is CRI-O or something similar which has no daemon and runs as an unprivileged user. And you still get the flexibility and process isolation, but with more security.
Re:Ask HN: Who operates at scale without containers?
by freebuju on Tuesday March 22, 2022 @02:56PM
(#30770002)
All the so-called "docker killers" are essentially unfinished products. They don't compare 1:1 to docker in feature set and even if they run as rootless, they still are vulnerable to namespace exploits in the Linux kernel. Though docker runs as root, it's still well protected out-of-the-box for the average user and is a very mature technology.
Re:Ask HN: Who operates at scale without containers?
by ryanjkirk on Tuesday March 22, 2022 @04:00PM
(#30770756)
Are you from 2018? Everyone running OpenShift is using CRI-O and that footprint is not small. We made the switch in our EKS and vanilla k8s clusters in 2021. Docker has now even made their API OCI-compliant in order to not be left behind. And the point is that most people don't want a docker feature-for-feature running in prod. The attack surface is simply too large. I don't need an API server running as root on all my container hosts.
Use docker on your laptop, sure. Its time in prod is over.
Re:Ask HN: Who operates at scale without containers?
by Art9681 on Tuesday March 22, 2022 @05:41PM
(#30771944)
Agreed. Tons of obsolete assumptions in this thread. We have been using Podman / OpenShift in production and never ran into a use case where Docker was needed.
Re:Ask HN: Who operates at scale without containers?
by p_l on Tuesday March 22, 2022 @04:33PM
(#30771143)
One of the biggest benefits of k8s for me, back in 2016 when I first used it in prod, was that it threw away all the extra features of Docker and implemented them directly by itself - better. Writing was already in the wall that docker will face stern competition that doesn't have all of its accidental complexity (rktnetes and hypernetes were a thing already)
Re:Ask HN: Who operates at scale without containers?
by richardwhiuk on Tuesday March 22, 2022 @04:48PM
(#30771332)
Kubernetes used to (tediously) pass everything through to Docker, but since 1.20, that's resolved, and it now uses containerd.
Re:Ask HN: Who operates at scale without containers?
by p_l on Tuesday March 22, 2022 @05:13PM
(#30771643)
Not everything - for a bunch of things, the actual setup increasingly happened outside docker then docker was just informed how to access it, bypassing all the higher level logic in Docker.
1.20 is when docker mode got deprecated, IIRC, but many of us were already happily running in containerd for some time.
Re:Ask HN: Who operates at scale without containers?
by qwertywert_ on Tuesday March 22, 2022 @09:20PM
(#30773763)
I thought its moving to CRI-o as well instead of containerd, or is k8s just not using docker, and containerd still supported in the future?
Re:Ask HN: Who operates at scale without containers?
by p_l on Wednesday March 23, 2022 @06:07AM
(#30776851)
CRI-O is the target and containerd is the most common runtime implementing it at the moment.
Re:Ask HN: Who operates at scale without containers?
by qwertywert_ on Wednesday March 23, 2022 @09:10PM
(#30785440)
Are you sure? This isn't my subject area but CRI-O looks like an alternative to containerd and implements the OCI compliant runtime like containerd does. And then there is a 3rd which is docker engine which is the one being dropped.
Re:Ask HN: Who operates at scale without containers?
by p_l on Thursday March 24, 2022 @04:22AM
(#30787380)
Sorry, I mixed up CRI and CRI-O. The roadmap is to remove dockershim (the interface to docker-compatible container runtime) and use only CRI - of which containerd and CRI-O are two compatible implementations.
Re:Ask HN: Who operates at scale without containers?
by richardwhiuk on Tuesday March 22, 2022 @04:47PM
(#30771321)
Kubernetes has removed docker, so I think that's basically it from a large scale perspective.
Re:Ask HN: Who operates at scale without containers?
by freedomben on Wednesday March 23, 2022 @11:37AM
(#30779573)
Yep. Mantis has announced intent to continue maintaining the docker shim (which allows k8s to talk to docker programmatically, but I can't imagine many people switching the default to docker unless they are manually installing k8s on their nodes, which used to be common but no longer is.
Re:Ask HN: Who operates at scale without containers?
by stickyricky on Tuesday March 22, 2022 @11:22PM
(#30774518)
> Handled 30 billion writes per day.
Writes to what?