Re:Optimizing Your Web App 100x Is Like Adding 99 Servers
by fefe23 on Saturday November 21, 2020 @06:18AM
(#25169063)
Unfortunately the probability of failure also scales up.
Let's say you bought good hardware and the probability it will fail is x.
Let's say you buy a hundred of those. The probability that one of them fails is now 100x. You just increased the probability of failure a hundredfold. You will need a load balancer who can detect failure and not forward to that machine anymore. The load balancer can fail, too. Now it's 101 fold. You also need a separate database if you share sate. 102 fold. Make it redundant. 103 fold. Add router / switches. 105 fold. Make those redundant. 107 fold.
I have never used more than one crappy PC for any project of mine, and some of those have had/have pretty substantial load on them. When I say one server, I mean one PC but with two power supplies and RAID storage. I'm not trying to tempt fate here.
But I have no devops team, no admins juggling failing machines, I don't have racks. I have one slot in a rack somewhere. The machines have been running for years. Every few years, as I approach MTBF of the hardware, I move to a new server. That entails a small downtime.
Result: People use my servers in their "is the internet up" scripts. Because my failure rate is a fraction of what distributed systems have.
In fact, practically all of my downtimes have been software bugs in my code, or me fat-fingering input somewhere. I can't even remember the last time the hardware failed. Not even sure I ever had an actual hardware failure that caused downtime. I once had a raid hdd throwing errors and it had to be replaced. I think that's it.
Let's say you bought good hardware and the probability it will fail is x.
Let's say you buy a hundred of those. The probability that one of them fails is now 100x. You just increased the probability of failure a hundredfold. You will need a load balancer who can detect failure and not forward to that machine anymore. The load balancer can fail, too. Now it's 101 fold. You also need a separate database if you share sate. 102 fold. Make it redundant. 103 fold. Add router / switches. 105 fold. Make those redundant. 107 fold.
I have never used more than one crappy PC for any project of mine, and some of those have had/have pretty substantial load on them. When I say one server, I mean one PC but with two power supplies and RAID storage. I'm not trying to tempt fate here.
But I have no devops team, no admins juggling failing machines, I don't have racks. I have one slot in a rack somewhere. The machines have been running for years. Every few years, as I approach MTBF of the hardware, I move to a new server. That entails a small downtime.
Result: People use my servers in their "is the internet up" scripts. Because my failure rate is a fraction of what distributed systems have.
In fact, practically all of my downtimes have been software bugs in my code, or me fat-fingering input somewhere. I can't even remember the last time the hardware failed. Not even sure I ever had an actual hardware failure that caused downtime. I once had a raid hdd throwing errors and it had to be replaced. I think that's it.