Now I wonder.. how many projects actually needs these kind of solution when even StackOverflow can do without it (they are in the range of few servers)? I would imagine it would be only few top popular web apps/services, but by popularity of these posts it looks like it is probably a lot more...
Does it make things better, though? Yes. Yes it does.
Kubernetes was designed at Google where they reaaaaally feel their scale problems day to day.
I see Docker/Kube/CoreOS/ etc as the natural evolution of where we were already going. Bash -> puppet -> vagrant -> docker -> kubernetes. Less abstract to more abstract.
So it's actually "only" an incremental evolution in terms of managing the server ecosystem. But it's a revolutionary improvement in how we think about server ecosystems, which is why many people struggle with Docker et al at first; it's a brand new mental model.
Sometimes you need to have the ability to run more services when your "web-scale", but even if you're not at that point, scaling how you deploy new versions, and new services is still important.
Tools like docker, and Kubernetes really help with the application delivery aspect, and really enable you to rapidly iterate on your project.
You might have a site with 100-1000 users, not huge by any means, and 1 server could probably handle all of your needs. But once you start adding other components, perhaps redis, or a runtime like nodejs, those can all be managed, but if you need to rapidly iterate, something like Kubernetes/docker can make updating or deploying these easier in the long run.
Really, why should cost anything to have basic, low-traffic, site on the internet today? That's the quetion I ask myself.
There's also other aspects, like being able to spin up the newest PostgreSQL within minutes, and when you're done you rm the container and image and have no trace and/or side effects.
There's some pretty useful multi-container configurations out there too, like a dockerized gitlab environment: https://github.com/sameersbn/docker-gitlab
stackoverflow is just a nice example that you don't need dozens of servers, even when you reach their scale.
I do believe Brook's The Mythical Man-Month does also applies to computers. In that communication overhead soon outweighs the benefits of adding more computer nodes.
Something I never understood with containers: where do they store persistent data, e.g. MySQL's /var/lib/mysql - and how does upgrading work, i.e. when the apt-get postinstall script runs transformation on the persistent data, how is the transformation applied to the "clones"?
You don't upgrade a running container. Imagine that containers are immutable; to launch a new, upgraded version, you re-build the image in your build/dev environment, and re-launch the image into production. If you're using a volume, you get to use the same backing data.
It's rare that apt-get postinstall will affect any data that you would persist -- app-specific data you'd keep in the image/container, and mysql data for instance you don't really want apt touching anyway. But if a data migration is necessary you'd either manage it with a "utility container" (image that's designed to run a script then stop, rather than run and keep running).
Load balancing two separate servers you can consider as active-active for the terminology.
Right now the first several chapters serve as a "I'm a programmer but know nothing about Docker" guide, the mid of the book dives into Docker-specifics (how exactly volumes work, how Docker networking works, etc), and then it graduates into the Docker Ecosystem, dedicating some time to covering tools like Kubernetes, CoreOS, Fleet, Amazon ECS.
The latter portion of the book is a number of recipes: how to get a Ghost blog running behind an nginx proxy; how to launch WordPress with MariaDB; how to launch an ELK stack, etc.
And hopefully your book will mention SmartOS and Triton and zones providing full isolation for Docker.
"At the scale of "a few servers", Kubernetes doesn't help much. At the scale of dozens or hundreds, it definitely does."
So now I know I don't need to learn much about Kubernetes unless I have "dozens or hundreds" of servers to manage.
One thing Kubernetes IS helpful with at small scales: portability. If you're fully Kubernetes/Docker, then you won't get locked into (eg) AWS's ecosystem. It's relatively easy to pick up an entire Kubernetes cluster and move it from AWS to DigitalOcean to private hardware.
So even if you're at small scale today but want to design for portability, I'd definitely look into Kubernetes.
Now, think about it from a reductionistic engineering perspective. What do I really want this server to do? Well, it accepts TCP connections, parses a request to figure out which file (at the time, it was all files) to server, sticks an HTTP header on the file, and shoves it down the socket.
This task is so simple that a skilled network programmer can nowadays literally bash together a 1990s-level static HTTP server in an day, with nothing but a socket library and some basic string handling. (It may not be great and it probably is insecure, but, well... see also "1990s web server"....) The code to do this is perhaps in the dozens or hundreds of kilobytes.
But that's not what I had. I have a full computer that physically needs to live somewhere. It has hardware ethernet and hardware graphics cards and a physical monitor and a power supply and RAM and, basically, hardware hardware hardware, the failure of any one of which means the system is either difficult to change or outright down. I have an entire Windows operating system, which even in the 1990s was hundreds of megabytes of code, endless code. Code for a windowing system, for pete's sake. Code for the audio subsystem. Code for accessing the hard drive. Code for access code that accesses code. Code code code code code, a bug in any one of which means the system may be down or insecure. My website, which at the time was quite likely in single-digit megabytes in size, was a tiny directory lost in a sea of files on the hard drive.
Over the past 20 years, the commodity hardware world [1] has been slicing away at the fact that several dozen kilobytes of code are being accompanied by hundreds of megabytes of support and literal pounds of physical hardware. Hardware went first with VMs. VMs got lighter and lighter. Lightweight hypervisor solutions sliced away at the heaviness of the VM. Containers slice away at the OS. Things like Kubernates slice away at the idea of a container living somewhere physically.
We're trying to free that several dozen kilobytes of code to be just several dozen kilobytes of code, as flexible and easy-to-deploy as several dozen kilobytes should be, if you weren't mired in the world of hardware and OSes and code and strong physical connections.
(Data storage is more complicated, but in a lot of ways, the same principles are in play.)
Operationally, containers are very exciting. However, in terms of "magic technologies I don't understand", I don't think they're worth stressing about "getting old" or anything. It's mostly "just" a big pile of practical considerations in trying to not just build that world, but in some sense also undo decades of grinding-in of the physical world to our operational considerations. If you want worry about getting old and out-of-date, worry about that let you do something that you couldn't before, like GPU programming or deep learning.
[1]: Which must be specified because mainframes beat us all here decades ago, in a lot of ways.
In the ancient past, like when I'm from, you'd write up a few different bash scripts to help you provision each server type. But setting this all up, you'd still have to run around and create 20 servers and provision them into one of 5 different types, etc.
Then there's chef/puppet, which takes your bash script and makes it a little more maintainable. But there are still issues: huge divide between dev/prod environments, and adding 5 new nodes ASAP is still tedious.
Now you have cloud and container orchestration. Containers are like the git repos of the server world. You build a container to run each of your apps (nginx, redis, etc), configure each once (like coding and committing), and then they work identically on dev and prod after you launch them (you can clone/pull onto hardware). And what's more, since a container image is pre-built, it launches on metal in a matter of seconds, not minutes. All the apt-get install crap was done at image build time, not container launch time.
Things are a lot easier now, but you still have a problem. You're scaling to 30, maybe 50 different servers running 6 or 7 different services. More and more you want to treat your hardware as a generic compute cloud, but you can't escape that, even with docker, your servers have identities and personalities. You still need to sit and think about which of your 50 servers to launch a container on, and make sure it's under the correct load balancer, etc.
That's where Kubernetes steps in; it's a level of abstraction higher than docker, and works at the cluster level. You define services around your docker containers, and let Kubernetes initialize the hardware, and abstract it away into a giant compute cloud, and then all you have to do is tell kubernetes to scale a certain service up and down, and it automatically figures out which servers to take the action on and modifies your load balancer for that service accordingly.
At the scale of "a few servers", Kubernetes doesn't help much. At the scale of dozens or hundreds, it definitely does. "Orchestration" isn't just a buzzword, it's the correct term here; all those containers and services and pieces of hardware DO need to be wrangled. In the past it was a full time sysadmin job, now it's just a Kubernetes or Fleet config file.
Disclosure: I'm currently writing a book on Docker. Disclaimer: I have not had my coffee yet.
Edit: Since someone asked, I'm writing a book called "Complete Docker" which will be published by Apress. I don't know the exact pub date that Apress will launch it on, but I expect it'll be available in October.