News for hackers. Stuff that matters.
Re:Escaping Surprise Bills and Over-Engineered Messes: Why I... by valenterry on Tuesday February 04, 2025 @12:15AM (#42928308)
It depends. If you want a service that doesn't have downtime because of a hardware issue, then you should have at least 2 machines. And then, you probably want to do deployments in a way where you spin up 2 new machines and then, only once confirmed that the deployment works, shut down the old 2.

That is super easy to do on AWS (and many other providers) but not so easy (or even impossible) anymore when you do things on-prem.

Re:Escaping Surprise Bills and Over-Engineered Messes: Why I... by esquire_900 on Tuesday February 04, 2025 @12:30AM (#42928421)
Super easy? Getting that setup for the first time from zero knowledge will probably take a few days. And that's before understanding all the intricacies like your AWS bill, the hidden costs of cloud, properly securing IAM and setting up your VPC etc etc
Re:Escaping Surprise Bills and Over-Engineered Messes: Why I... by wontondisregard on Tuesday February 04, 2025 @12:46AM (#42928512)
Not once in my entire career have I seen people successfully avail themselves of the cloud's purported benefits. I have, however, seen a lot of happy account managers in Las Vegas during Re:Invent, oh yes.
Re:Escaping Surprise Bills and Over-Engineered Messes: Why I... by valenterry on Tuesday February 04, 2025 @01:47AM (#42928923)
Comparably easy, yes. I'm not talking specifically about AWS btw, but even there it is easy.

If someone has zero knowledge, then everything will take them a long time, including hosting on-prem or so.

Re:Escaping Surprise Bills and Over-Engineered Messes: Why I... by pinoy420 on Tuesday February 04, 2025 @01:30AM (#42928807)
I got downvoted for saying something similar for standing up a web app from scratch and getting it deployed securely.

You are absolutely correct

Re:Escaping Surprise Bills and Over-Engineered Messes: Why I... by wontondisregard on Tuesday February 04, 2025 @12:45AM (#42928507)
I'm sorry, but the ten billionth CRUD web app really doesn't need nine nines.
Re:Escaping Surprise Bills and Over-Engineered Messes: Why I... by valenterry on Tuesday February 04, 2025 @01:43AM (#42928895)
I agree. It doesn't change my argument though.
Re:Escaping Surprise Bills and Over-Engineered Messes: Why I... by 2030ai on Tuesday February 04, 2025 @01:17AM (#42928723)
Just a blue green! But I agree somewhat. Single machine running Docker compose is pretty decent. Pets! Not cattle. (I'd rather care for a dog than a 100 sheep)
Re:Escaping Surprise Bills and Over-Engineered Messes: Why I... by freeone3000 on Tuesday February 04, 2025 @12:45AM (#42928508)
It is exactly as easy, or easier, to do this in proxmox than it is in AWS console, and would have a price breakeven point of less than a year (you only need two physical nodes).
Re:Escaping Surprise Bills and Over-Engineered Messes: Why I... by valenterry on Tuesday February 04, 2025 @01:44AM (#42928901)
Proxmox is just a way to virtualize things. You need hardware to run it. So what hardware are you talking about?
Re:Escaping Surprise Bills and Over-Engineered Messes: Why I... by vidarh on Tuesday February 04, 2025 @02:36AM (#42929304)
Missing the point that you don't need to be able to spin up/down servers. You can just rent 2 and spin up/down containers.
Re:Escaping Surprise Bills and Over-Engineered Messes: Why I... by valenterry on Tuesday February 04, 2025 @03:59AM (#42929948)
> You can just rent 2 and spin up/down containers.

Missing what point? You are apparently agreeing with me and disagreeing with the OP who said: "can run on a raspberry pi or mini PC".

Re:Escaping Surprise Bills and Over-Engineered Messes: Why I... by vidarh on Tuesday February 04, 2025 @04:49PM (#42939344)
No, you most definitely can just spin up/down containers on a raspberry pi or mini PC as well.

This is what I objected to:

> And then, you probably want to do deployments in a way where you spin up 2 new machines and then, only once confirmed that the deployment works, shut down the old 2.

You don't need to spin up more machines or shut down machines. As long as you ensure you have the headroom you spin up containers and shut down containers. By doing that, it's also relatively easy to set up hybrid setups where you can handle load-spikes by scaling into EC2 or similar if you should ever need to.

Re:Escaping Surprise Bills and Over-Engineered Messes: Why I... by valenterry on Tuesday February 04, 2025 @09:03PM (#42942603)
> As long as you ensure you have the headroom you spin up containers and shut down containers.

Are you very sure you are not introducing potential additional problems with that approach? Because if you do, then I wouldn't say it's "not needed" to spin more additional machines - it would depend on those additional problems.

Re:Escaping Surprise Bills and Over-Engineered Messes: Why I... by vidarh on Friday February 07, 2025 @12:36PM (#42975274)
Yes, I am, very much sure of that, based on 20 years of experience deploying containers that way, and nearly the same length of experience using cloud instances. They are equivalent in complexity. It is purely down to how you want to host things and whether cost matters to you - the managed approach is consistently significantly cheaper for all but the very, very smallest environments where factors like discounts and free tiers play in.
Re:Escaping Surprise Bills and Over-Engineered Messes: Why I... by freeone3000 on Tuesday February 04, 2025 @09:46AM (#42932991)
Literally any random x86 box you have lying around. NUCs work fine here, but so do used Optiplexes from Craigslist or your old gaming PC.
Re:Escaping Surprise Bills and Over-Engineered Messes: Why I... by panja on Tuesday February 04, 2025 @02:11AM (#42929096)
Does proxmox have 24/7 enterprise support yet?
Re:Escaping Surprise Bills and Over-Engineered Messes: Why I... by 2030ai on Tuesday February 04, 2025 @12:49AM (#42928533)
Digital Ocean I believe is pay as you go and supports this for anyone who can click "link my Github account" and auth themselves.
Re:Escaping Surprise Bills and Over-Engineered Messes: Why I... by valenterry on Tuesday February 04, 2025 @01:42AM (#42928888)
Digital Ocean is not equal to "run on a raspberry pi or mini PC and be just fine".
Re:Escaping Surprise Bills and Over-Engineered Messes: Why I... by snackbroken on Tuesday February 04, 2025 @12:50AM (#42928543)
Most web projects can tolerate an hour of scheduled downtime a week and the occasional (< 1/year) unscheduled hour of downtime to replace broken hardware. Running on a single machine with 2 disks in RAID1 right up until the point where the ~50 hours of downtime a year equal one engineer's salary in lost revenue makes sense if you pick a DC that will sell/rent you replacement hardware and do replacements for you. Just make sure you have offsite backups in case the DC burns down. If your office/home is near a well stocked computer store, most web projects can skip the DC altogether and just put the server in the closet under the stairs or whatever. The company will likely die before the server does anyways.
Re:Escaping Surprise Bills and Over-Engineered Messes: Why I... by valenterry on Tuesday February 04, 2025 @01:45AM (#42928914)
> Most web projects can tolerate an hour of scheduled downtime a week and the occasional (< 1/year) unscheduled hour of downtime to replace broken hardware.

An hour? What happens if it breaks while I'm on vacation and no one is there to take care?

Nah, that argument might work sometimes but not in general.

Re:Escaping Surprise Bills and Over-Engineered Messes: Why I... by snackbroken on Tuesday February 04, 2025 @02:25AM (#42929215)
Yes, an hour. In practice, most scheduled maintenance will take closer to a minute than the full scheduled hour. If it's good enough for the government and the bank, it's probably good enough for you.

What happens if you're on vacation and something breaks is that you call/email the DC and ask them to replace what's broken, same as if you were not on vacation. In my experience most DCs will have the replacement done in less than 30 min from receiving the ticket, good ones will do it in under 15. If the project is ran by more than one person, simply don't all take a vacation at the same time. Worst case scenario you point DNS to a free static page host that reads "Server broke. Working on a fix, check back tomorrow." (or the appropriate corporate-PR phrasing of the same if applicable) while you wait for the computer store to open so you can buy a new one.

Even if you are once-in-a-lifetime unlucky and the damn thing breaks while you're on vacation and for some reason your monitoring fails to notify you, big whoop. You can probably get away with just apologizing for the inconvenience and moving on. If a week long outage is enough to make or break your business, your business isn't gonna work out anyway.

This applies to practically all web projects from the home page of the pizza place on the corner of your street to 100-employee e-commerce sites. If a single machine can run it, it can probably run on a single machine. Use the dumbest tool that will get the job done, switch to a smarter tool when it won't.

Re:Escaping Surprise Bills and Over-Engineered Messes: Why I... by valenterry on Tuesday February 04, 2025 @04:02AM (#42929981)
Even just setting up and documenting all the things you mentioned is already taking so much time and effort, that I'd rather pay a few more bucks for a HA system...
Re:Escaping Surprise Bills and Over-Engineered Messes: Why I... by Orygin on Tuesday February 04, 2025 @05:07AM (#42930478)
> If a week long outage is enough to make or break your business, your business isn't gonna work out anyway.

If your business has an SLA with its customers, having a week long outage is going to break your business. Some contracts I saw had a 15 minute reaction time where you had to acknowledge the issue, and after 1 hour of downtime there are penalties. The same contract also required redundancy (of hardware, internet lines as well as location for the DCs). We could not have hosted that on a Raspi, or a single machine.

I don't think reducing the scale issue to "you ain't gonna need it" is right. Obviously to most projects it's not gonna be needed. But when it does and you need to grow, you'll be happy you have a redundant infrastructure that you can spin up/down easily instead of having to call your DC to ask for more hardware to be installed.

Re:Escaping Surprise Bills and Over-Engineered Messes: Why I... by valenterry on Tuesday February 04, 2025 @05:15AM (#42930554)
To be fair, that is probably out of scope for what OP meant.
Re:Escaping Surprise Bills and Over-Engineered Messes: Why I... by Orygin on Tuesday February 04, 2025 @05:22AM (#42930599)
Maybe for the author of the blog post, about his personal projects. But the user I replied to seemed to indicate even "100-employee e-commerce sites" don't need any redundancy for their website, which I disagree with.
Re:Escaping Surprise Bills and Over-Engineered Messes: Why I... by fxtentacle on Wednesday February 05, 2025 @02:51AM (#42945329)
"If your business has an SLA with its customers" then you also need a paid SLA with Amazon and now we're talking about the 100k annually cost range where you can also afford to hire freelancers to babysit your sever while you're gone.
Re:Escaping Surprise Bills and Over-Engineered Messes: Why I... by fxtentacle on Wednesday February 05, 2025 @02:48AM (#42945295)
What happens if AWS has a hiccup that restarts your VM?

The answer might be to buy management services. Or an SLA for AWS.

Re:Escaping Surprise Bills and Over-Engineered Messes: Why I... by theshrike79 on Tuesday February 04, 2025 @05:48AM (#42930771)
Most but not all.

If you're running an online shop/reservation system, a hour of downtime can be tens of thousands of lost revenue.

Re:Escaping Surprise Bills and Over-Engineered Messes: Why I... by fxtentacle on Wednesday February 05, 2025 @02:54AM (#42945359)
Revenue, not profit. Many small shops voluntarily turn off their website for some weeks so that the employees can take a Christmas break. So it seems like the business can handle a week offline.
Re:Escaping Surprise Bills and Over-Engineered Messes: Why I... by znpy on Tuesday February 04, 2025 @02:02AM (#42929036)
> That is super easy to do on AWS (and many other providers) but not so easy (or even impossible) anymore when you do things on-prem.

False. You can run kubernetes and/or some virtualisation engine as well and do the same on prem.

and overprovisioning physical hardware is much cheaper.

Re:Escaping Surprise Bills and Over-Engineered Messes: Why I... by valenterry on Tuesday February 04, 2025 @04:00AM (#42929964)
Well, yes. I would categorize "running kubernetes on prem" as "not so easy".
Re:Escaping Surprise Bills and Over-Engineered Messes: Why I... by theshrike79 on Tuesday February 04, 2025 @05:47AM (#42930763)
And you can easily have a separate clean environment for every pull request easily.

Can't do that with your own hardware unless it's also a Kubernetes cluster, which is a whole new set of crap you need to manage.

Re:Escaping Surprise Bills and Over-Engineered Messes: Why I... by sgarland on Tuesday February 04, 2025 @08:24PM (#42942193)
> not so easy (or even impossible) on-prem

It’s as if an entire generation of tech workers are utterly unaware that these have been solved problems for decades.

You put two machines behind HAProxy with keepalived. If you want to do a new deployment, you can spin up new machines in the same manner, and cut them over (or any of the many other ways to accomplish this).

AWS isn’t inventing any of the HA stuff they offer; they’re taking existing technologies and packaging them.

Re:Escaping Surprise Bills and Over-Engineered Messes: Why I... by valenterry on Tuesday February 04, 2025 @08:58PM (#42942547)
So how do you do all of that with "a raspberry pi or mini PC"? You would need at least 4 of them.
Re:Escaping Surprise Bills and Over-Engineered Messes: Why I... by sgarland on Wednesday February 05, 2025 @07:32AM (#42947579)
I was replying to your comment, not the other. But sure, get 4 RPis if you want. You could also be using containers on them, and not need to swap whole computers. Or run each app in its own chroot jail. Or run VMs. There are many, many ways to accomplish this goal.
Re:Escaping Surprise Bills and Over-Engineered Messes: Why I... by bschwindHN on Tuesday February 04, 2025 @10:59PM (#42943578)
Most web projects can afford to be down for a bit. Even the big guys have outages fairly often. It's not like AWS is always running smoothly either, and there are so many "_ is down" posts on HN alone.

I do agree that your projects should be designed to be well contained and easy to create a new instance of it, though.