News for hackers. Stuff that matters.
Re:So many bad takes. Learn from the Prime Video microservic... by romanhn on Sunday May 07, 2023 @01:40PM (#35853740)
As someone who went with Lambda for my current project, I don't agree with this take at all. In fact, it's as boring of an implementation as you can imagine - a single "fat function" developed in .NET serving the entire web application. Can be easily transplanted onto any other hosting paradigm, but Lambda is incredibly cheap (free at my current usage) and incredibly scalable should I hit some inflection point. Learning K8s would have been quite an overkill.
Re:So many bad takes. Learn from the Prime Video microservic... by bob1029 on Sunday May 07, 2023 @01:48PM (#35853816)
We are thinking the exact same thing (w/ Az Functions). Keeping it simple as possible means you can forklift the whole thing in one afternoon to a different provider. It also means you could potentially even go back to an on-prem stack or whatever.

We are looking at taking our .NET6 monolith and "inverting" it so the entry point is our logical HTTP resources instead of the various DI monstrosities that emerge from main.

The reasoning for us is that we don't want ownership over all of the horrible things we could get wrong. If a <TLS1.2 connection some how gets established and something naughty happens because we forgot to update some startup.cs boilerplate, it's currently our pain today. Like legally. If we are just eating pre-approved/filtered/etc HTTP requests from inside our hermetically-sealed function environment, we get to cast a lot more blame than we otherwise would be able to.

I really don't get why you'd go to serverless and then shit it up with a lot of extra complexity. The whole point in my view is to inherit the compliance packages of your hyperscalar by way of directly using their native primitives in as standard a way as possible. This means you get to 'sail' through audits and engage up-market clients that are typically out of reach for everyone else.

Re:So many bad takes. Learn from the Prime Video microservic... by ricardobeat on Sunday May 07, 2023 @02:53PM (#35854402)
If you have a monolithic entrypoint, is it really serverless? You're simply using the Lambda infra as a run-on-demand server, something Heroku has been offering for 10+ years. Why would you pay extra when you can deploy to a simple VM or managed platform costing 100x less?
Re:So many bad takes. Learn from the Prime Video microservic... by rhodysurf on Sunday May 07, 2023 @04:21PM (#35855259)
Because it doesn’t actually cost less? You have to have a LOT of invocations to match the cost of the entry level 6 dollar digital ocean box. And with that you have to manage that box vs serverless you don’t have to spend engineering time (which costs an order of magnitude more) on maintenance, a big deal for small shops without dedicated/capable ops teams.
Re:So many bad takes. Learn from the Prime Video microservic... by lukevp on Sunday May 07, 2023 @04:23PM (#35855285)
What you described would be more akin to using ECS Fargate to run the workload. What they described sounds like a single .NET lambda behind API Gateway, so the infrastructure of ingesting requests (and handling TLS termination, DDoS mitigation, WAF, etc.) all happens on the AWS side. All you have to do is handle the routing inside of the lambda using regular ASP.NET Core. It would also scale up infinitely and down to zero with no work on their end. This isn’t similar to running something on Heroku on an allocated server with fixed overhead at all.
Re:So many bad takes. Learn from the Prime Video microservic... by ricardobeat on Sunday May 07, 2023 @11:23PM (#35857730)
Indeed it would be more like Fargate in terms of container runtime vs lambda runtime, but the line between those is pretty blurry. Fly.io for example does containers but they spawn faster than some cloud function platforms. It's not really relevant for the discussion.

> Heroku on an allocated server with fixed overhead at all

I don't think you're familiar with Heroku then? I'm not talking about dedicated or managed servers. That's exactly what their 'dynos' platform has been doing since ~2008, though less granular: https://www.heroku.com/dynos/scaling

"All you have to do is handle the routing inside of the lambda using regular ASP.NET Core" also applies to both fly.io and Heroku. That's what I'm getting at. If you don't have a collection of independent cloud functions to execute, and you are spinning up a little monolith for every request, the platform seems like overkill.

Re:So many bad takes. Learn from the Prime Video microservic... by romanhn on Sunday May 07, 2023 @04:35PM (#35855381)
What makes it serverless is the ability to spin up instances on demand (including down to zero when not in use), while paying only when the resources are in use. Given Lambda's generous free tier, I'm literally paying nothing for it. That would not be the case for any other platform.
Re:So many bad takes. Learn from the Prime Video microservic... by fallat on Sunday May 07, 2023 @01:49PM (#35853831)
k8s is not as hard as its advertised
Re:So many bad takes. Learn from the Prime Video microservic... by ghaff on Sunday May 07, 2023 @02:04PM (#35853952)
One of my recent conclusions is that it's not so much that Kubernetes on its own is so very complicated; it's that if someone tries to glue their own container platform together from Kubernetes plus another dozen or so cloud-native projects and integrate the whole thing, it can get wickedly complex very quickly.
Re:So many bad takes. Learn from the Prime Video microservic... by derefr on Sunday May 07, 2023 @02:23PM (#35854109)
Right — Kubernetes was evolved in the context of being a Google Cloud service, and despite now having other deployments, still is mostly architected around the assumption that your Kubernetes cluster is surrounded by a featureful IaaS platform providing things like managed load balancers, VM auto-scaling pools, iSCSI volumes on a dedicated SAN network, non-throughput-bottlenecked object storage (and a container image repo on top of that!), a centralized logs and metrics sink, etc. A k8s cluster expects these like a computer expects a bootable storage device.

If you're using a managed Kubernetes (in GCP, or AWS, or Azure, etc) then the owner of the platform presumably has all this stuff going on somewhere (whether they expose it to you or not), and so is solving all these problems for you. But if you're not using managed Kubernetes — i.e. if you're ever typing `k3s deploy` or whatever equivalent — then you'd better realize that you're signing up to either also deploy and manage your own bare-metal OpenStack deployment for your k8s cluster to plug into; or to tediously glue together a grab-bag of services from your hosting provider + third-parties to achieve the same effects.

(And this is why Google's "Anthos" is an interesting offering to many enterprises: rather than running your own on-prem k8s cluster implying the need to own+manage on-prem supporting infra, you can instead have a GCP project that has all the managed supporting infra, and maybe some hosted k8s clusters; and then your on-prem k8s nodes can just think of themselves as part of that GCP project, relying on the same cloud infra that the hosted clusters are relying on — while still being on-prem nodes, and so still having data security guarantees, low-latency links to your other on-prem systems, etc.)

Re:So many bad takes. Learn from the Prime Video microservic... by ghaff on Sunday May 07, 2023 @02:34PM (#35854213)
There are other container platforms like Red Hat OpenShift as well (disclaimer: I work there) either operated for you or on-prem (doesn't need to be on top of OpenStack and typically isn't these days). I'm biased but Kubernetes complexity tends to be conflated with the complexity of a DIY container platform which can be significant, especially given that Kubernetes itself has generally made the decision to keep its scope relatively narrow.
Re:So many bad takes. Learn from the Prime Video microservic... by vbezhenar on Sunday May 07, 2023 @02:54PM (#35854413)
I implemented self-hosted k8s in my current company and had to learn it from the ground. I'm not a professional, but I learned a thing or two.

My opinion is as follows:

Installing kubernetes with kubeadm is easy. You can do it in few hours if you're fluent with Linux.

Kubernetes storage is where it starts to be hard. We were lucky to have OpenStack hosting which allows to use network block storage (ceph) without any additional work. Using kubernetes without any storage might severely limits apps you can run there, although it's possible. Using local storage is an option but also with many "but". Rolling out ceph: I've heard it's hard.

Load balancing: again I was lucky to have OpenStack load balancer. Otherwise, I think that it's possible, but definitely not within a few hours, unless you did it before.

Kubernetes networking might be tricky. I wasn't able to implement it in a way I like. I used calico with overlay mode in the end, but I'd prefer "flat" network. Calico overlay mode was easy.

Upgrading kubernetes manually is easy. At least until it breaks, it didn't break for me yet. kubeadm just works.

Node autoscaling is hard. There're no simple solutions, at least I didn't find any. Our cluster does not autoscale. Manual scaling is not hard, though. I can implement autoscaling with loads of bash/terraform/cloudinit scripts but I'd rather not to. I think that node autoscaling is what managed kubernetes offering do well.

Now that we have kubernetes, story is not really over. We need to collect logs for nodes, for services, for pods. We need to collect metrics. We need to set up alerts. We need to have traces. Service mesh looks really nice so you can see how services interact with each other. Those things are not for given. They take lot of time and expertise. And while they're not strictly about kubernetes, usually you need them.

Re:So many bad takes. Learn from the Prime Video microservic... by __turbobrew__ on Sunday May 07, 2023 @06:27PM (#35856217)
Agree on storage being the hard part. You either need a robust SAN to store persistent volumes or you need to store persistent volumes on the local storage of kubelets which is a major pain. Once a pod binds a local pvc to a kubelet that pod must now run on that kubelet for the rest of eternity. If you want to get rid of a kubelet with a local pvc you either need the application be durable to data loss (which many are not) or you need to implement some sort of stateful pod draining logic.

When rolling your own, I think you need to invest in a battle tested SAN or just not do stateful on k8s. For the handful of stateful services I have run on k8s it would have been better to just use a managed stateful solution.