I have one server with 1 GB RAM. Can I start with that? And expand on more servers later if need arises.
For small scale development, plain docker/podman is better in my opinion.
I used k3sup to setup my cluster, but that was just for convenience.
I have been running it on those for a while, but also on other SBCs. You can grow the single master by attaching one or more workers to it. The "pain" is mainly that you can't expand the single-master to a multi-master setup, so once you want to go beyond a single master, you have to re-install it all. On the other hand, with k8s you have all deployments, etc. in yaml files and you simply redeploy these on the new cluster.
My remaining pain-point is persistent storage - it's simple to run complex setups with stateless containers or that can retrieve there data at initialization (i.e. a secondary name server that populates itself from the master at startup), but for classic databases I currently still have a manual step to import the DB dump once the service got (re)started/migrated.
Sources: - https://k3s.io/ - https://www.jeffgeerling.com/blog/2020/installing-k3s-kubern...
K8s operates just fine on 1GB nodes. It's very low overhead.
You can operate it on a single node cluster, but that's generally not recommended, not for performance reasons, but for redundancy.
The reason larger nodes are recommended is that k8s is for fleets of applications. If you are only talking about a single service, then a k8s cluster is overkill.
K8s is the cheapest way to run a fleet of microservices.
If you'd like to use a management UI instead of just the CLI, Portainer is also a wonderfully lightweight option: https://www.portainer.io/ (and also supports Docker Swarm, if you'd like even more lightweight container orchestration)
That said, in my eyes the biggest problem with Kubernetes is that some distros by default reserve resources and thus disallow overcommit, which is fine from a stability perspective, but doesn't really work for overcommit scenarios.
For example, with Docker Swarm, it's exceedingly easy to say: "Okay, this PostgreSQL instance can use anywhere from 0 to 256 MB of RAM, but no more than that" and with 1 GB of RAM i could easily run 8 of those instances if the average usage was half or less of this maximum limit - a risk that i'd sometimes like to take, when i know that the limits in place are more along the lines of controls so that the whole OS doesn't run into OOM errors.
Of course, if you mess around with the YAML, i guess that's doable on some level as well.
As a sidenote: most of the software that i run (Java, .NET, Ruby, Python, Node, Go, PHP) all is memory constrained most of the times, so when looking for server specs, i usually prioritized the memory the highest, CPU, storage and network speed all usually being a secondary concern. Sadly, in the current day and age, i'm not sure how far you could actually get with 1 GB or why even use container orchestration at that point. The smallest VPSes that i have come with 2 GB of RAM and even those are for smaller pages/proxies and such, whereas the majority have at least 4 GB (since GitLab and Nexus won't launch and work with much less) and my homelab servers have 16 GB, whereas my workstation has 24 GB at this point, because that's all that i can afford.
Wirth's law be damned, i doubt that even OSes will work well with that much in the future, especially because of desktop software being infected with Electron and server software having JVM eat as much memory as you'll give it (though that also happens with MySQL or MariaDB). I don't care about "Unused RAM is wasted RAM" as much as others do, i just want to run more stuff on my servers and to have software that doesn't NEED the memory not take it.
There is mini-Kubernetes implementation, but I don't know if they're any good for production.
There seems to be a limit to how small you can go with Kubernetes in my mind, and most people never need to cross the line where they're large enough that Kubernetes makes sense. It's honestly a niche product, but right now it's being viewed as a catch all solution.
You need to connect your cluster to public Internet? No problem that's $x for each load balancer (whether its needed or not).
Ohh you are feeling the whole thing feels like a blackbox and difficult to debug or observe? No problem that is $x for data dog.
Gcp is the worst when it comes to setting up k8s with preemptive nodes. Thought you could get away with preemptive nodes? Not so fast, we restart all the nodes together at end of 24 hours period so that your multi node cluster will have zero availability for 5 minutes everyday. Or jump through hoops to killing your own nodes periodically to keep them all restarting at the same time.
If you like you can install a free ingress such as NginX and route traffic to it.
If you like you can just have logs on your physical nodes and go look one by one, just as you would have to before things like k8s came along. Datadog is a value add. It's not essential.
GCP preemptible VMs are nothing to do with k8s - they are literally designed to be short-term (up to 24 hours) VMs to do things with. Yes, GKE can use them, but not as persistent resources. That's not what they're for.
Here's what they're for:
> Preemptible VMs are Compute Engine VM instances that last a maximum of 24 hours, and provide no availability guarantees.
I.e. don't try and "get away with" them. There are plenty of options for cheap K8s.
I think k8s actually saves you money, because it allows you to use your resources better.
I've worked with it for a bunch of years now and run a cluster at home, but I do hate that it scales down so poorly. Is so much work to run a bare minimum, if not for SME-purposes but for local development. It's just starting to get acceptable.
doubt it as there was JVM back then.
>This is why I'd love WASM to win. Right now your browser can run 100s of tabs, with inter-tab communications and all the security sophistications on a single machine.
Now just add K8s on top to orchestrate it :) Somebody will just develop new WASM based container ...
>[...] WASM is Java for the haters, so it sells much better at the bytecode shop.
Which means WASM was advertised as a new bytecode VM to replace Java.
What I'm saying is that, for these supposed Java haters, why did they choose to completely reinvent the bytecode VM when they could have just chosen Kotlin and not have to reinvent the wheel? It would fix most if not all of their concerns about Java.
Just like "if we had internet in the 19th centry", it implies so much more.
Anyway, to make WASM useful as a server platform it will end up re-implementing the entire Linux kernel API.
If your service can run on few nodes, do you really need a scheduler like K8s? Why not go serverless like Fargate or CloudRun?
My biggest concern with K8s is how the entire ecosystem is a costly affair. Can I run a service with multiple nodes for 10-50$ per month?
This is why I'd love WASM to win. Right now your browser can run 100s of tabs, with inter-tab communications and all the security sophistications on a single machine. For me, this symbolises a future where WASM can bring cheaper computing to the cloud as well. I don't want to pay a fortune as AWS/Azure bills just because k8s is a costly ecosystem.