News for hackers. Stuff that matters.
Re:OVH CEO Octave Klaba speaking about the incident [video] by justicezyx on Thursday March 11, 2021 @03:23PM (#26428571)
This would be one of inherient difference between smaller vs. giga players in cloud hosting.

AWS/Google/Azure, if this happens, there should only be limited outage to a small fraction of customers. As a matter of fact, Google had such an incident before, and literally no customers (internal and external) noticed.

Re:OVH CEO Octave Klaba speaking about the incident [video] by ev1 on Thursday March 11, 2021 @03:25PM (#26428589)
This is an apples to oranges comparison. OVH largely sells bare metal; their public cloud wasn't really impacted.

If you are using AWS, Google, or Azure, ran a single (or multiple machines) inside a single AZ with no backups and opted out of snapshots, you would face the exact same situation.

I can definitely say I see people complaining about how everything they have is down on AWS when us-east-1 goes down periodically, while large players that deploy sanely like Netflix fail over to another region seamlessly.

This [only owning a single machine at all] is what most of their customers whinging the most were doing. People that have actual sane production workloads on AWS or GCP are not going to be running 100% of their workload on a single EC2 instance with no backups.

People that are running on OVH are running often things like gameservers etc that monopolise 100% of a physical machine and don't support horizontal scaling. You quite literally cannot force a srcds/hlds server to "load balance" dynamically and fail over on heartbeat.

Often they are kids or students too, and the $30/m for a machine with 32-64GB ram is all they can afford (though this doesn't absolve them of paying $1-2/m more for offsite backups elsewhere)

You can provision more physical machines with the OVH API and have them be up in a different city in a minute or two. You get linespeed bandwidth between OVH DCs. It's up to you to use it.

Re:OVH CEO Octave Klaba speaking about the incident [video] by jonas21 on Thursday March 11, 2021 @03:37PM (#26428719)
On the other hand, just about every month, there's a story on HN saying why are you wasting your money on AWS when OVH is so much cheaper (for example https://news.ycombinator.com/item?id=24966028).

And well, I guess this is one of the reasons.

Re:OVH CEO Octave Klaba speaking about the incident [video] by ev1 on Thursday March 11, 2021 @03:43PM (#26428785)
If you choose to run 100% of your workload on a single EC2 VM in us-east-chaos-monkey and put nothing in S3, only local mounted block storage that also disappears when you reboot your on-demand EC2, that is on you.
Re:OVH CEO Octave Klaba speaking about the incident [video] by kuschku on Thursday March 11, 2021 @03:54PM (#26428894)
2 OVH dedicated servers in different countries are still cheaper than one AWS instance.

e.g. I've got servers at OVH SBG-2, Hetzner's Falckenstein, and Online.net's AMS datacenters — the total of which is still almost a magnitude less than the same cost on AWS or GCP (granted, that's including traffic)

Re:OVH CEO Octave Klaba speaking about the incident [video] by ev1 on Thursday March 11, 2021 @03:55PM (#26428898)
Pretty much. Use 5% of the money you saved moving your workload from AWS to OVH to support failover, DR, and backups. You can probably buy like five or six machines of equivalent spec or more.
Re:OVH CEO Octave Klaba speaking about the incident [video] by tmikaeld on Friday March 12, 2021 @12:06AM (#26432388)
Not only that, but the OVH load balancer (API gateway) is enormously much cheaper at 2TB of traffic than AWS and they don't charge for the amount of requests.
Re:OVH CEO Octave Klaba speaking about the incident [video] by dvfjsdhgfv on Thursday March 11, 2021 @05:40PM (#26429929)
No, it doesn't work like this. I have several bare-metal (with Heztner, I use OVH for DNS), it's been over 10 years already. I know that if I only rent one machine in one location, I'm asking for trouble. Based on my experience, I would say that every 2-6 years something dies in a server. A disk, a controller, a fan, you name it. It's rare to have servers running for longer than 7 years without any issues, and they're outdated by that time anyway so they need to be migrated to a new machine.

So, as a bare minimum, you rent at least two different machines at two different locations for each project and make offsite backups. It's still way less expensive than AWS.

If I don't need a powerful server and just need to spin some instances for testing or small projects, I use Hetzner Cloud, it's ridiculously cheap.

Re:OVH CEO Octave Klaba speaking about the incident [video] by kazen44 on Thursday March 11, 2021 @03:38PM (#26428732)
how so? using OVH and their bare metal servers doesn't absolve you from doing your own due dilligence.

As said earlier, their cloud service is unaffected.

Re:OVH CEO Octave Klaba speaking about the incident [video] by justicezyx on Thursday March 11, 2021 @03:31PM (#26428660)
Oh good to know. I don't use OvH, and my limited understanding were from their products page which lists VM style offerings. I had assumed VMs were the major use cases on OvH.
Re:OVH CEO Octave Klaba speaking about the incident [video] by ev1 on Thursday March 11, 2021 @03:34PM (#26428689)
The pricing on their dedicated servers are cheaper than most 1-2GB VMs on cloud: https://www.ovhcloud.com/en/bare-metal/ - this is their flagship and most expensive brand, the cheaper ones are even less

The funniest tweets demanding their data and saying they'll lose everything are the people running:

https://twitter.com/Sensity_RP/status/1369496048998223873 - GTA5 multiplayer gameserver that begs for donations. Running on one of the cheaper sub-brands of OVH probably (soyoustart, for GAME ddos-protection). $30-40/machine.

https://twitter.com/pdfshift/status/1369550522479480833 - I can't tell if this is a troll

https://twitter.com/KatsanosAlex/status/1369501497348812801 - I can't tell if this is a troll

Re:OVH CEO Octave Klaba speaking about the incident [video] by gowld on Friday March 12, 2021 @01:55PM (#26439429)
I can imagine a provider that offers "Disaster Recovery Plan" in the sense that you push a button and they spin up [whatever machines you subscribe to] in a different data center and restore your latest backup to it.

Or the button flips your DNS from your usual primary servers to a hot backup (or a cold backup that you brought online yourself).

Re:OVH CEO Octave Klaba speaking about the incident [video] by stefan_ on Thursday March 11, 2021 @04:07PM (#26429012)
This is a difference in what you are buying. When you are buying a dedicated server, there isn't exactly a good way to hide that the thing has just gone up in smokes.

When you buy a storage API, sure, failure rates go up, latency increases 100x, but after a few hours its probably back to normal.

Of course, with the increased abstraction, you get more problems. "Availability zones" are useless when most cloud outages are because of configuration or systemic issues that tend to bring the whole thing down, no matter which AZ you are. But apparently it's now considered "good enough" to just go "oh we are down because AWS is down".

Re:OVH CEO Octave Klaba speaking about the incident [video] by tormeh on Friday March 12, 2021 @11:55AM (#26437839)
If you're in with the big cloud providers you have no choice. Hybrid cloud is economically impossible due to the bandwidth costs.

Yet somehow, at smaller providers and dedicated hosters bandwidth is usually included as a too-cheap-to-meter feature. Gotta love cloud innovation.

Re:OVH CEO Octave Klaba speaking about the incident [video] by gowld on Friday March 12, 2021 @01:56PM (#26439446)
Bandwidth is cheap until you run out and then it's very expensive.
Re:OVH CEO Octave Klaba speaking about the incident [video] by bauruine on Friday March 12, 2021 @06:58AM (#26434828)
AWS is the new IBM. Nobody ever got fired for using AWS.
Re:OVH CEO Octave Klaba speaking about the incident [video] by greggyb on Friday March 12, 2021 @09:48AM (#26436352)
And funny enough, it's now considered crazy to do a hardware startup, even if staffed up with industry vets. The reaction to Oxide is funny to watch, especially.
Re:OVH CEO Octave Klaba speaking about the incident [video] by Saris on Thursday March 11, 2021 @03:32PM (#26428666)
It also depends if you're renting a dedicated server, vs cloud/VPS. AWS/Google/Azure deal with virtualized systems that can be moved around to another server easily.

OVH has a lot of dedicated servers as well though, so if you're using one of those then it can't be moved very easily to avoid downtime.

Re:OVH CEO Octave Klaba speaking about the incident [video] by dilyevsky on Thursday March 11, 2021 @11:54PM (#26432342)
Just to note - only google does live migration (with ~100ms blackout) with others you will take some downtime anyway (assuming the remaining zones even have enough capacity to fit everyone)
Re:OVH CEO Octave Klaba speaking about the incident [video] by londons_explore on Friday March 12, 2021 @03:43AM (#26433495)
Google never migrates to a different physical location, so it wouldn't actually protect against a whole class of issues (major flooding, war, employee strike, etc)
Re:OVH CEO Octave Klaba speaking about the incident [video] by dilyevsky on Friday March 12, 2021 @01:50PM (#26439354)
Zones are often in separate locations but only within a few miles so yeah. Also i think they won’t even migrate inter-zone so there’s that
Re:OVH CEO Octave Klaba speaking about the incident [video] by jeffbee on Thursday March 11, 2021 @03:26PM (#26428601)
I can't even find any press articles about the Google incident.
Re:OVH CEO Octave Klaba speaking about the incident [video] by Nomikos on Thursday March 11, 2021 @03:31PM (#26428658)
Are you googling? ;-)
Re:OVH CEO Octave Klaba speaking about the incident [video] by londons_explore on Friday March 12, 2021 @03:47AM (#26433515)
At the time that Google data center was not known to be owned by Google. So it will just be "fire in industrial estate"