News for hackers. Stuff that matters.
Re:How we migrated onto K8s in less than 12 months by maccard on Thursday August 08, 2024 @06:59PM (#41197297)
Figma were running on ECS before, so they weren't just running dedicated servers.

I'm a K8S believer, but it _is_ complicated. It solves hard problems. If you're multi-cloud, it's a no brainer. If you're doing complex infra that you want a 1:1 mapping of locally, it works great.

But if you're less than 100 developers and are deploying containers to just AWS, I think you'd be insane to use EKS over ECS + Fargate in 2024.

Re:How we migrated onto K8s in less than 12 months by epgui on Thursday August 08, 2024 @08:30PM (#41197777)
I don’t know if it’s just me, but I really don’t see how kubernetes is more complex than ECS. Even for a one-man show.
Re:How we migrated onto K8s in less than 12 months by mrgaro on Friday August 09, 2024 @02:23PM (#41204178)
Kubernetes needs regular updates, just as everything else (unless you carefully freeze your environment and somehow manage the vulnerability risks) and that requires manual work.

ECS+Fargate however does not. If you are a small team managing the entire stack, you need to factor this into accounts. For example EKS forces you to upgrade the cluster to keep in the main kubernetes release cycle, albeit you can delay it somewhat.

I personally run k8s at home and another two at work and I recommend our teams to use ECS+Fargate+ALB if it is enough for them.

Re:How we migrated onto K8s in less than 12 months by metaltyphoon on Friday August 09, 2024 @05:28PM (#41205520)
> Kubernetes needs regular updates, just as everything else (unless you carefully freeze your environment and somehow manage the vulnerability risks) and that requires manual work

Just use a managed K8s solution that deals with this? AKS, EKS and GKE all do this for you.

Re:How we migrated onto K8s in less than 12 months by ttymck on Friday August 09, 2024 @05:51PM (#41205647)
It doesn't do everything for you. You still need to update applications that use deprecated APIs.

This sort of "just" thinking is a great way for teams to drown in ops toil.

Re:How we migrated onto K8s in less than 12 months by metaltyphoon on Friday August 09, 2024 @06:40PM (#41205960)
Are you assuming the workloads have to use K8s APIs? Where is this coming from? If that’s not the case can you actually explain with a concrete example?
Re:How we migrated onto K8s in less than 12 months by ttymck on Friday August 09, 2024 @09:10PM (#41206647)
Any cluster extension. Helm is a good example.

https://helm.sh/docs/topics/version_skew/

Istio: https://istio.io/latest/docs/releases/supported-releases/#su...

Literally every kubernetes manifest that hits the server uses a k8s api:

    apiVersion: apps/v1
Re:How we migrated onto K8s in less than 12 months by epgui on Friday August 09, 2024 @09:57PM (#41206778)
You mean operators?

(genuine tone, not rhetorical)

Re:How we migrated onto K8s in less than 12 months by ttymck on Friday August 09, 2024 @10:47PM (#41206949)
Sure, an operator is likely to use a wide array of APIs.

But, to reiterate, everything uses APIs. The *betavX APIs are of course likely to be deprecated and replaced with stable APIs after a few versions.

Re:How we migrated onto K8s in less than 12 months by Too on Saturday August 10, 2024 @01:35AM (#41207559)
Man, you don't need to use service mesh just because you use k8s. Istio is a very advanced component that 99% of users don't need.

So if you are going to compare with a managed solution, compare with something equivalent. Take a bare managed cluster and add a single Deployment to it, it will be no more complex than ECS, while giving you much better developer ergonomics.

Re:How we migrated onto K8s in less than 12 months by ttymck on Saturday August 10, 2024 @02:03AM (#41207648)
99% of users don't need kubernetes. Just deploy to heroku, and you'll have a much better developer experience.
Re:How we migrated onto K8s in less than 12 months by 5Qn8mNbc2FNCiVV on Monday August 12, 2024 @11:09AM (#41225342)
My wallet says otherwise
Re:How we migrated onto K8s in less than 12 months by Ramiro on Friday August 09, 2024 @06:47PM (#41205997)
I agree with @metaltyphoon on this. Even for small teams, a managed version of Kubernetes takes away most of the pain. I've used both ECS+Fargate and Kubernetes, but these days, I prefer Kubernetes mainly because the ecosystem is way bigger, both vendor and open source. Most of the problems we run into are always one search or open source project away.
Re:How we migrated onto K8s in less than 12 months by epgui on Friday August 09, 2024 @08:35PM (#41206522)
My experience with k8s has been very much “just”, and I’ve never really had issues or experienced any real friction with updates. shrugs
Re:How we migrated onto K8s in less than 12 months by ttymck on Friday August 09, 2024 @09:21PM (#41206670)
That's great. I guess I've somehow been making things harder than they need to be.
Re:How we migrated onto K8s in less than 12 months by butshouldyou on Saturday August 10, 2024 @07:13PM (#41212915)
Looks like you were using k8s APIs directly in your applications, which is a more complex set-up.

In my experience, most k8s deployments are just "dumb" docker images, they're not very "k8s native".

Your use case may be more complex, which is why you have had more difficulty keeping things up-to-date.

Re:How we migrated onto K8s in less than 12 months by anonfordays on Thursday August 22, 2024 @09:14PM (#41325525)
There's still Helm oddities, "annotations", CRDs, mutating web hooks, operators, etc. to comprehend before you have any idea of what the system is doing. All it takes is a random annotation to throw all your assumptions away.

It's a complicated mess compared to something like a Nomad jobspec. That's one of the reasons we decided on Nomad while I was at Cloudflare.

Re:How we migrated onto K8s in less than 12 months by mountainriver on Friday August 09, 2024 @01:43PM (#41203880)
This just feels like a myth to me at this point. Kubernetes isn’t hard, the clouds have made is so simple now that it’s in no way more difficult than ECS and is way more flexible
Re:How we migrated onto K8s in less than 12 months by davewritescode on Friday August 09, 2024 @01:54PM (#41203989)
I’m not saying I agree with the comment above you but Kubernetes upgrades and keeping all your addons/vpc stuff up to date can be a never ending slog of one-way upgrades that, when they go wrong, can cause big issues.
Re:How we migrated onto K8s in less than 12 months by organsnyder on Friday August 09, 2024 @02:30PM (#41204224)
Those are all issues that should be solved by the managed provider.

It's been a while since I spun up a k8s instance on AWS, Azure, or the like, but when I did I was astounded at how many implementation decisions and toil I had to do myself. Hosted k8s should be plug-and-play unless you have a very specialized use-case.

Re:How we migrated onto K8s in less than 12 months by adammarples on Saturday August 10, 2024 @02:35PM (#41211352)
Yeah if you pay someone to manage something for you, it's not hard. That's because you're not doing it. Like air travel and taxes, it's easy.
Re:How we migrated onto K8s in less than 12 months by organsnyder on Monday August 12, 2024 @10:32AM (#41224829)
If you're not paying someone to manage this for you, either you're a hobbyist (perhaps masquerading as a professional) or you have the scale or special use-case that makes you outside 95% of use-cases.

Last I checked, managed k8s clusters weren't much more expensive than the compute they ran on.

Re:How we migrated onto K8s in less than 12 months by anonfordays on Thursday August 22, 2024 @09:15PM (#41325533)
It is by no means "a myth". There's still Helm oddities, "annotations", CRDs, mutating web hooks, operators, etc. to comprehend before you have any idea of what the system is doing. All it takes is a random annotation to throw all your assumptions away.

It's a complicated mess compared to something like a Nomad jobspec. That's one of the reasons we decided on Nomad while I was at Cloudflare.