News for hackers. Stuff that matters.
Re:Kubernetes Is Your Private Cloud by nostrebored on Wednesday November 12, 2025 @11:40AM (#45902278)
“Everything You Expect from a Cloud, Running on Your Terms“*

Except you own ops, management, extension, interoperability, access, security, scalability, redundancy… words cannot express how ridiculous all of the koober propaganda is

Re:Kubernetes Is Your Private Cloud by bakies on Wednesday November 12, 2025 @12:00PM (#45902571)
You have to pick your battles. Most of this stuff isn't necessary to babysit until you're scaling your app tremendously. And by the time you're doing that I'm sure you've got the people to do these things.
Re:Kubernetes Is Your Private Cloud by hobs on Wednesday November 12, 2025 @12:03PM (#45902620)
And then why would you need koob at all? All that setup and learning on a platform you don't understand and won't need to manage and you will do it wrong, so a completely wasted set of time and money afaict.
Re:Kubernetes Is Your Private Cloud by bakies on Wednesday November 12, 2025 @12:10PM (#45902691)
IMO, there's no alternative private cloud.
Re:Kubernetes Is Your Private Cloud by pyrale on Wednesday November 12, 2025 @11:51AM (#45902436)
All things people used to own 10 years ago. It’s not like the people doing that stuff have vanished.

Cloud’s big promise was speed to market and price, and let’s be honest, price is no longer there compared to a decent operation.

The one thing where clouds remain kings is speed for small teams. Any large enough company should probably Ask themselves whether running their own operation using ias would be a better choice.

Re:Kubernetes Is Your Private Cloud by bushbaba on Wednesday November 12, 2025 @12:01PM (#45902585)
My company is on prem, spending north of 1 billion per year. Cloud is actually cheaper when considering total cost of ownership. Thats salaries, opex, capex costs. Worse, our speed to delivery is generally worse.

Because on prem is inelastic, we are at sub 10% peak utilization of compute resources. If we added in the likely higher cloud utilization rate we are talking of 30%+ savings from on prem.

Re:Kubernetes Is Your Private Cloud by bakies on Wednesday November 12, 2025 @12:04PM (#45902634)
> we are at sub 10% peak utilization of compute resources

so... you bought way too much hardware?

Re:Kubernetes Is Your Private Cloud by stackskipton on Wednesday November 12, 2025 @12:36PM (#45903068)
That's not unusual. First off, sometimes 1000 extra dollars will get you a ton more compute you need so why not and second, on prem tends to be extremely inelastic so you buy a ton of compute because you never know when compute requirements change.
Re:Kubernetes Is Your Private Cloud by bakies on Wednesday November 12, 2025 @01:08PM (#45903528)
If we're talking on the scale of $1,000s then it's cheaper to run on-prem than in the cloud. It's really easy to spend $1,000 on managed kubernetes and have very little actual compute.
Re:Kubernetes Is Your Private Cloud by ecshafer on Wednesday November 12, 2025 @12:13PM (#45902726)
Peak Utilization is a tough one for on prem and is a decent argument for cloud. I was at a company that was also at <10% peak utilization most of the time. It was finance, so it was mostly doing nothing, except for the couple days a year where we shot up 10000x, so we had to build for that case. So yeah losing the data centers, and cloud was a cost savings.
Re:Kubernetes Is Your Private Cloud by senderista on Wednesday November 12, 2025 @12:35PM (#45903049)
So the "obvious" (but complex!) solution is a "hybrid cloud": use on-prem for the predictable, constant "DC component" of your demand, and use cloud for cyclical or unpredictable demand changes. That will keep peak demand decoupled from permanently provisioned capacity while saving on always-utilized capacity. Easier said than done, of course.
Re:Kubernetes Is Your Private Cloud by ecshafer on Wednesday November 12, 2025 @12:52PM (#45903296)
The route we ended up going, was hybrid cloud with a colocation for on-prem (Mainframes and a few servers), and then AWS for the cloud portion. Not sure on what the cost savings were. As I wrote the authentication service I knew what those numbers were, so I knew that on a normal day we would be sitting at 1% cpu usage basically all day except market open/close. And then fiscal quarter ends were a big bump. And then of course big news days. 99% of the time for 99% of the days were just nothing.
Re:Kubernetes Is Your Private Cloud by senorrib on Wednesday November 12, 2025 @12:52PM (#45903297)
K8s helps reduce that complexity a lot.
Re:Kubernetes Is Your Private Cloud by pyrale on Wednesday November 12, 2025 @01:36PM (#45904001)
My comparison was cloud vs renting metal, not buying your own machines.
Re:Kubernetes Is Your Private Cloud by gdubya on Wednesday November 12, 2025 @11:42AM (#45902302)
Re:Kubernetes Is Your Private Cloud by almosthere on Wednesday November 12, 2025 @11:45AM (#45902358)
It's crazy to me how many people deploy unmaintainable spaghetti mess in all other environments I've been in. "koober" environments are the most organized.
Re:Kubernetes Is Your Private Cloud by nostrebored on Wednesday November 12, 2025 @11:03PM (#45910479)
I find this to be an insane argument. I have never seen a k8s environment that wasn’t a garbage heap. But everyone is eager to tell you what a nice can they’ve found next to the rotting banana.

Onboarding new team members? A disaster. The design? All done by one dumpster diver with nobody to call them out on the mistakes because they have no idea what the hell is happening.

I’ve never seen a k8s shop where there weren’t a few principal engineers being roped from incident to incident because teams couldn’t manage their own.

With cloud providers, even ones we don’t use, people have a general idea of how the primitives work. With any reasonably skilled team you get back to talking about how your product works instead of talking about persistent volume replication for Postgres backups or the differing file system behavior or what cli everyone should be using to manage your entire company’s software or how to stop someone from deleting the entire of everything you own or or or

Re:Kubernetes Is Your Private Cloud by mikeocool on Wednesday November 12, 2025 @11:57AM (#45902528)
And particularly the upgrades every 3 months. Not just your nodes and masters, but every operator you use, and your manifests each time they deprecate a manifest beta version.
Re:Kubernetes Is Your Private Cloud by LeSaucy on Wednesday November 12, 2025 @12:04PM (#45902629)
Ive found nomad to be a much simpler replacement for smaller scale deployments.
Re:Kubernetes Is Your Private Cloud by dilyevsky on Wednesday November 12, 2025 @12:08PM (#45902671)
It’s a well known thing that if you run on ec2 they handle all those things for you (especially the security part)
Re:Kubernetes Is Your Private Cloud by icedchai on Wednesday November 12, 2025 @12:29PM (#45902956)
I wonder how many people will miss the sarcasm. ;) Cloud means you don't need any ops people, right?
Re:Kubernetes Is Your Private Cloud by nostrebored on Wednesday November 12, 2025 @11:07PM (#45910511)
They do manage almost all of them for you. If you’d like we can get some of the EC2 engineers on our team to tell you how.

If you don’t think IAM is a better set of semantics for securing your infrastructure than nightmarish k8s rbac i really don’t know what to tell you.

Can you think back to your batch and honestly tell me that the companies that used k8s are in a better place?

Re:Kubernetes Is Your Private Cloud by JoshTriplett on Wednesday November 12, 2025 @12:21PM (#45902842)
The way I've described this for years: Kubernetes makes managing 1000 servers as easy as managing 20 servers, and makes managing 3 servers as easy as managing 20 servers.
Re:Kubernetes Is Your Private Cloud by btown on Wednesday November 12, 2025 @12:42PM (#45903141)
My way of thinking about it is this: you have your own hyper-flexible Heroku, but (monkey's paw curls) you can only interact with it by typing large amounts of YAML.

Oh, and all the documentation for that YAML assumes you've memorized as much vocabulary as a Foreign Language 101 class.

(And there is a mad god that says: if you try to use click-ops to get around this without knowing the vocabulary, you're going to have a bad time.)

But on the other hand: to put it in terms of the "3 servers" - the moment you think you'll have 3 servers, and any level of uptime expectations, you'll inevitably have to rebuild them, services and logging and all, from scratch often and quickly enough that you might as well have 20 servers with how stressful that rebuild will be.

k8s can be a saving grace there, and I recommend it to anyone with the time and interest in how cluster best practices work! But it's not a free option or a weekend skill-up.

Re:Kubernetes Is Your Private Cloud by bob1029 on Wednesday November 12, 2025 @12:25PM (#45902908)
Most products barely need 1 server.
Re:Kubernetes Is Your Private Cloud by bakies on Wednesday November 12, 2025 @12:32PM (#45903006)
And if you ever outgrow that it's going to be a huge pain. Or a hardware failure. If you start on Kubernetes early you'll be able to add more servers for capacity very easily. Not to mention out of the box you get failover and HA. And you can deploy managed services and have database deployments, or object stores, etc.
Re:Kubernetes Is Your Private Cloud by eikenberry on Wednesday November 12, 2025 @02:11PM (#45904662)
But 99% of the time you don't outgrow it and don't have SLA's requiring them to have failover or HA. This is why so many sites can get away with using PostgreSQL with it's complete lack of good/native HA.
Re:Kubernetes Is Your Private Cloud by bakies on Wednesday November 12, 2025 @02:29PM (#45905021)
HA and failover, to me-in these small deploys, is more about hardware maintenance rather than maintaining SLAs. Being able to shutdown the computer hosting your containers to scale vertically.
Re:Kubernetes Is Your Private Cloud by spwa4 on Thursday November 13, 2025 @10:45AM (#45916248)
Single-site-2-absurdly-intense-days failover, yes. With weeks of cleanup afterwards.

What every company seems to want: multisite, multimaster immediate failover, no.

Also kubernetes buys you scaling. Compute. Disk. Database (with help). Etc.

Now I rail against companies for wanting that, and I think you're right. Your webshop does not need that. It has so many moving parts the redundancy will cause more outages than it solves. But you can do this, and so people will pay for it.

It is a technical accomplishment.

And with sufficiently good sysadmins, it can work well, and scale.

Re:Kubernetes Is Your Private Cloud by icedchai on Wednesday November 12, 2025 @02:28PM (#45905007)
You could always run a VM / VPS against a managed DB. Many small cloud / VPS providers, like Digital Ocean or Vultr, also offer managed DB services that are cheaper than AWS RDS.
Re:Kubernetes Is Your Private Cloud by jameson on Thursday November 13, 2025 @11:27AM (#45916850)
Ops cost to number of server to manage is logarithmic but cloud cost is linear, so there's an intersection it starts to make less sense for cloud. Also equipment depreciation is tax deductible whereas cloud bill isn't. A year of EC2 instance bill is comparable to buying the equivalent server

Also there are vendors renting out datacenters so you do less of hardware management.

Having worked at two companies spending 250M+ on cloud bills alone, they try hard to decouple from cloud but many things are vendor locked

Hybrid has been the answer to both. It shouldn't be a binary decision. stateless compute workload can fairly easily be offloaded to private cloud.

Re:Kubernetes Is Your Private Cloud by raducu on Tuesday November 18, 2025 @02:24AM (#45962309)
> Also equipment depreciation is tax deductible whereas cloud bill isn't.

Genuine question out curiosity (I have a master in finance, but never practiced it) -- aren't both the cloud bill and depreciation all tax deductible, eventually? the bill 100% in that year and the depreciation spread over multiple years?

> Hybrid has been the answer to both. It shouldn't be a binary decision. stateless compute workload can fairly easily be offloaded to private cloud.

Can you elaborate on that? I'm studying for saa-c03, and I was shocked by how expensive egress out of aws can be.