News for hackers. Stuff that matters.
Re:Fly.io Postgres cluster down for 3 days, no word from the... by YetAnotherNick on Friday July 21, 2023 @02:50AM (#36810760)
Does anyone know how Hetzner pricing is half of DO yet is profitable, while DO is loss making with 6% operating margin?
Re:Fly.io Postgres cluster down for 3 days, no word from the... by stevefan1999 on Friday July 21, 2023 @02:56AM (#36810793)
Simple, Hetzner mainly operates on Germany, the people are mostly Germans, and they automate the stuff to a point a small team could manage it well even if not remotely, so they have less cost on human resources.
Re:Fly.io Postgres cluster down for 3 days, no word from the... by rahkiin on Friday July 21, 2023 @03:37AM (#36811063)
They also build their own servers in their own datacenters
Re:Fly.io Postgres cluster down for 3 days, no word from the... by raybb on Friday July 21, 2023 @04:00AM (#36811239)
Does digital ocean not do this?
Re:Fly.io Postgres cluster down for 3 days, no word from the... by re-thc on Friday July 21, 2023 @04:48AM (#36811553)
They don’t.
Re:Fly.io Postgres cluster down for 3 days, no word from the... by cutemonster on Friday July 21, 2023 @10:41AM (#36814363)
Where do DO get their servers and data centers from? ... Apparently they run on AWS, I'm surprised
Re:Fly.io Postgres cluster down for 3 days, no word from the... by re-thc on Sunday July 23, 2023 @07:29AM (#36834390)
> Apparently they run on AWS, I'm surprised

They don't run on AWS. Not sure what sort of rumors are running :(

> data centers from?

The major players e.g. Equinix, Coresite, etc. Varies per location. Even AWS don't build most of their data centers.

Re:Fly.io Postgres cluster down for 3 days, no word from the... by stevefan1999 on Friday July 21, 2023 @06:46AM (#36812259)
The competitor of DO, Vultr does this IIRC, yet it is not really cheaper
Re:Fly.io Postgres cluster down for 3 days, no word from the... by KronisLV on Friday July 21, 2023 @07:31AM (#36812572)
> Simple, Hetzner mainly operates on Germany, the people are mostly Germans, and they automate the stuff to a point a small team could manage it well even if not remotely, so they have less cost on human resources.

I feel like there might be more to it, especially considering the situation with electricity prices in some places in EU recently.

I used (and still use) a Lithuanian platform called Time4VPS which was cheaper than Hetzner previously, yet had to increase their prices somewhat for that reason. Now only some of their plans are competitive with Hetzner, while Hetzner also provides some managed services as well.

Hetzner docs also went into some of the details regarding the pricing: https://docs.hetzner.com/robot/general/pricing/hetzner-prici...

And yet, I can't help but to wonder why they don't give in to the desire to maximize profit margins, like happened to say Scaleway (good platform, but as expensive as DigitalOcean).

Re:Fly.io Postgres cluster down for 3 days, no word from the... by api on Friday July 21, 2023 @09:21AM (#36813431)
I've wondered how they can host this cheap in Germany given their very high electricity prices.

Maybe that's not actually the dominant cost, or they've optimized everything else so well they can just eat the electric bill.

Re:Fly.io Postgres cluster down for 3 days, no word from the... by ushakov on Friday July 21, 2023 @04:43AM (#36811511)
Me and my partner have paid a visit to their datacenter in Nüremberg. The answer is efficiency. They get more processing power than the other providers for the energy they have to put in
Re:Fly.io Postgres cluster down for 3 days, no word from the... by arrowsmith on Friday July 21, 2023 @04:59AM (#36811622)
What do they do that makes them more efficient?
Re:Fly.io Postgres cluster down for 3 days, no word from the... by abwizz on Friday July 21, 2023 @05:29AM (#36811798)
i'll guess they pick optimized components for it.

like the longtime workhorse was a high performance skylake desktop cpu w/o ecc ram

Re:Fly.io Postgres cluster down for 3 days, no word from the... by ushakov on Friday July 21, 2023 @09:24AM (#36813464)
The secret is in the cooling system. They have individual cooling systems for each server. Less heat = longer sustained loads
Re:Fly.io Postgres cluster down for 3 days, no word from the... by abwizz on Saturday July 22, 2023 @04:55AM (#36824593)
pardon my ingorance but i cannot quite see how cooling individual machines vs. the hole rack or row makes a difference in total heat production per machine
Re:Fly.io Postgres cluster down for 3 days, no word from the... by ushakov on Friday July 21, 2023 @04:51AM (#36811571)
Efficiency. They get much more processing power per kw/h of energy than everybode else
Re:Fly.io Postgres cluster down for 3 days, no word from the... by devjab on Friday July 21, 2023 @05:04AM (#36811650)
They run their own data centres and have for a while. There is a pretty big industry for that sort of thing as an alternative to “the cloud” here in Europe.

We used to use nianet to house our hardware in Denmark. Basically these companies does hardware renting and they also do hardware renting with more steps which is where you rent rack space but own the hardware. They provide the place for the hardware and they also have multiple locations so that you have both backup and redundancy, and while it doesn’t scale globally in 20 years I’ve literally never worked on anything that needed to beyond having some buffer caches for clients logging in on their vacations or something like that.

What Hetzner seems to be doing with the DO styled hosting, and this is just a guess, is that they are one or the many EU companies preparing for the big EU exodus from the non-EU cloud. Which is frankly a solid bet these days where both AWS and Azure are increasing prices and are becoming more and more unusable because of EU legislation. Part of this is privacy which Microsoft and Amazon are great with in terms of compliance, but part of it is also national security. I work in an investment bank that builds solar plants, since finance and energy are both critical sectors we risk being told that half of the finance/energy companies in the world can’t use Microsoft because the EU seems it as a single point of failure if our entire energy sector relies on Azure. Which is sort of reasonable right? But what this means for us is that we can’t vendor lock-in, not really, because we need to have up-to-date exit strategies for how we plan on being fully operation a month after leaving Azure. Which is easy when you just containerise everything and run it in VMs or similar, and really annoying if you go full in on things like AKS. Which doesn’t help our Azure costs.

Anyway, right now we are planning on leaving Azure because of cost. Not today, not next week but sometime in the next 5-10 years and a lot of these EU cloud alternatives that actually operate the hardware instead of renting it are likely going to be a very realistic alternative. And that is the private sector, I spend time in the EU public sector which is a massive amount of money and I’m guessing it’ll leave both AWS and Azure by 2050. Some of these EU cloud initiatives is going to explode when that happens, and right now, hetzner is one of the best bets.

To get back to your question, DO rents server space. I have no idea where they’d rent it in Germany but they could potentially be renting it from Hetzner.

Re:Fly.io Postgres cluster down for 3 days, no word from the... by dpeckett on Friday July 21, 2023 @05:36AM (#36811838)
Couldn't agree more, I think Hetzner is probably Europe's best bet on a hyperscaler. One of the more telling indicators IMO is their growing market share outside of the EU/DACH.

To add on to the comments about Hetzner building their own custom hardware, they also custom built their own software stack. They rejected the hype that was OpenStack and worked diligently on their own hypervisor platform (that they are incredibly secretive about) and that appears to be paying off in spades for them. Most sovereign cloud plays end up being suffocated by the complexity, and incoherence, of the OpenStack ecosystem. It just becomes impossible to ship.

I know Hetzner is working like crazy on a managed Kubernetes offering atm, I think when that lands it could be very interesting.

For a fascinatingly different take on how to build a datacenter: https://www.youtube.com/watch?v=5eo8nz_niiM

Re:Fly.io Postgres cluster down for 3 days, no word from the... by bigjoes on Friday July 21, 2023 @06:25AM (#36812139)
Could you please elaborate how and what you know about managed Kubernetes on Hetzner?

I am asking for this since a while and was told there is no way Hetzner would offer such a service. Certain Posts on Social Media have also never been answered with any kind of indication that they are actually working on it.

Please provide some Details on this.

Re:Fly.io Postgres cluster down for 3 days, no word from the... by dpeckett on Friday July 21, 2023 @07:02AM (#36812348)
They were in person recruiting at KubeCon EU this year and were advertising a good number of Kubernetes engineering roles. Definitely gave me the impression they were taking Kubernetes seriously but looking back a managed offering was just speculation on my part.

So huge grain of salt, you are totally right. It could be internal platform work only.

Re:Fly.io Postgres cluster down for 3 days, no word from the... by dpeckett on Friday July 21, 2023 @07:14AM (#36812449)
For anyone interested in Kubernetes on Hetzner, there's a really interesting CAPI provider being actively developed:

https://github.com/syself/cluster-api-provider-hetzner

Re:Fly.io Postgres cluster down for 3 days, no word from the... by abwizz on Friday July 21, 2023 @05:25AM (#36811770)
commendable to plan a few years ahead, but betting on the state of cloud business 26years from now seems a bit over the top
Re:Fly.io Postgres cluster down for 3 days, no word from the... by detourdog on Friday July 21, 2023 @06:26AM (#36812142)
I think multi-national energy sector should be working toward the goals without the regulations. The more prep done before the change the smoother the transition.
Re:Fly.io Postgres cluster down for 3 days, no word from the... by devjab on Friday July 21, 2023 @12:26PM (#36815807)
I think you might misunderstand me. The 2050 is a guesstimate and it's just my opinion on the matter. As far as planning ahead goes, you plan for 5-10 years when you try to figure out where to "iron" your enterprise IT. This is because that's how long your hardware will last if you go the route of renting rack space with your own hardware. I think we tend to plan for 8 years, with some space for "unintended" early failures on things like controllers after 4 years. So while you can contract big-cloud vendors for shorter, I think ours is on 3 year contracts right now, you still sort of do the business case for much longer. Maybe not every 3 years, but at least every 6 years.

You do the same on the other side of the table. Companies like Hetzner knows that EU cloud sollutions are likely to see growth, so it's only natural that they invest in the tech to put themselves in a prime position to jump on the opportunity. Selling a good product while you do so is the way I would do it personally, but you also have EU cloud initiatives backed by VC money going straight for the endgame.

Re:Fly.io Postgres cluster down for 3 days, no word from the... by thejosh on Friday July 21, 2023 @08:29AM (#36813024)
Hetzner also do some crazy-cool stuff, especially around the 7950X3D, cooling, AM5 etc. (https://www.youtube.com/watch?v=V2P8mjWRqpk). They also do some amazing stuff with ARM (their cloud offering is really solid for this).
Re:Fly.io Postgres cluster down for 3 days, no word from the... by fxtentacle on Friday July 21, 2023 @06:21AM (#36812116)
I've been with them for a long time and my guesses would be:

1. Strict rules and strict customer verification. Crypto mining that wastes SSDs is not allowed. Portscans, mass emails, etc. are not allowed. They also don't offer GPUs to the general public because it has been abused in the past. You usually need to send in ID documents just to open an account. My guess is this allows them to avoid most bad actors and, thereby, waste less money on fraud.

2. Extremely long-term investments. They typically build their own hardware and then use it over 10 years. They have their own flea market where you can rent older server models for a steep discount. That means they will have a long time where the hardware is fully paid off and still generating revenue.

3. Great service. With a mid-sized company, I can call their technicians in the middle of the night. The fact that we could call them in case of a crisis has generated A LOT of good will. But I would be truly surprised if they didn't make a profit off those phone calls, as they charge roughly 4x the salary cost.

4. High-margin managed services. In addition to just the cheap servers, they also offer a managed service where they will do OS and security upgrades for you. It's roughly 2x the price of the server and it appears to be almost fully automated. I know some freelance web designers who will insist on using Hetzner Managed for deployment for their clients, because it is just so convenient. You effectively pass off all recurring maintenance for €300 a month and your client is happy to have an emergency phone number (see #3) in case the box goes down.

Re:Fly.io Postgres cluster down for 3 days, no word from the... by EspressoGPT on Friday July 21, 2023 @08:15AM (#36812917)
Overstaffed, overinflated and inefficient Silicon Valley startup vs. organically-grown, well-adjusted, efficient German company.
Re:Fly.io Postgres cluster down for 3 days, no word from the... by wongarsu on Friday July 21, 2023 @08:55AM (#36813209)
Not to mention a German company that has price sensitivity in their DNA. Their first servers were just regular consumer tower PCs to drastically cut hardware costs. Now many years later it's a highly optimized mix of consumer, server and inhouse parts (e.g. they use their own racking system instead of 19", and the datacenters are built to make use of convection for a lot of the cooling). They also offer regular Dell servers for those that want them, but at 2x-4x the price of their homegrown boxes.