The API is the main drawcard of k8s in the first place, if you are off in ECS land all you are doing is wasting a bunch of time on a dead-end.
I would instead focus on getting to understand the basics of the API by using a hosted k8s service like GKE or EKS. Stick with some basic manifests, i.e deployments, services, ingress.
Once you have some stuff running you can start learning how it really works and goes together, i.e what are pods, why are pods immutable, why is a replicaset, how does a deployment orchestrate multiple replicasets, what are endpoints, what is the difference between pod readiness/liveliness.
Don't cheat yourself this early in the game, just learn things the right way from the start and save yourself a bunch of work.
If you want to stay on the Kubernetes route check out k3s. Super easy to setup and usable for small production workloads
I always thought that one of the advantages of going serverless was that you didn't have to worry about keeping the underlying operating system up-to-date. No needing to do a monthly "sudo apt update && apt upgrade" or whatever. But containers throw that all away when container images enter the world.
Instead of updating your operating system, you're updating your images...and it's basically the same thing.
Most would say even a server-ful system (k8s, or whatever) should be considered 'cattle not pets' with immutable nodes replaced as needed anyway. No update, just replace. Just like building a new image and having new pods (or serverless whatevers) pull it.
I’ve never cordoned off an individual head of cattle and lobotomized it, which is kinda what we do when debugging issues. We take the pod out of rotation, flip a bunch of configs, then give it some traffic to see the new debugging statements.
If you're truly "serverless" by my interpretation of it, then you wouldn't care. Your cloud provider will have updated their infrastructure, and that's all that matters.
In fact almost nothing is serverless (well, the truth comes out! ;)) by that definition, since even Lambda has runtime versions to choose/upgrade, Managed-Acme has Acme versions, etc.
SES, SNS, SQS, etc. sure, but I suppose no compute, since you need libraries, and libraries have versions, and you can't have them (significantly/major versions) changing under your feet. (Or if they don't have versions they're of course destined to have known security holes.)
(Or it's not even about libraries if you want to say no you don't need libraries - it's just about having to interface with anything.)
Before I used Kubernetes for my side projects and only at work I always thought it's hard to operate and very tricky. If you start with an empty "default" cluster and then just add bits when you need them it's actually not that complicated and doesn't feel too heavyweight. I'd suggest to just play around with a simple example and then see how it goes.
There's things that are used in "production" clusters that you don't need at the beginning, like rbac rules, Prometheus annotations etc.
If you want to learn about containers, distributed workloads, etc, then Nomad is a great option that is easy to learn/adopt piecemeal.
There's now even onprem ECS variants which means not having to pay aws very much and still get the benefit of them running and maintaining the control plane
It's ridiculous overkill, but I'm looking at a NextCloud server on one of their PiBox hardware servers for the house. (You don't need a PiBox - their stuff will run fine on little instances from AWS/DigitalOcean/Hertzner, etc., or a spare PC you have lying around...)
Regarding PaaS stuff like Heroku, the only people I know that are still using that are solo hackers.