News for hackers. Stuff that matters.
Re:Ask HN: Do you find working on large distributed systems ... by fxtentacle on Saturday February 19, 2022 @10:01AM (#30397589)
You might be surprised. The performance equivalent of $100k monthly in EC2 spend fits into a 16m2 cage with 52HU racks.
Re:Ask HN: Do you find working on large distributed systems ... by benjamir on Saturday February 19, 2022 @10:12AM (#30397680)
Which costs you more than $100k monthly to operate with the same level of manageability and reliability.

We don't use AWS, because our use cases don't require that level of reliability and we simply cannot afford it, but if I needed a company to depend on IT that generates enough revenue... I probably wouldn't argue about the AWS bill. So long, prepaid at hetzner + in-house works good enough, but I know what I cannot offer with the click of a button to my user!

Re:Ask HN: Do you find working on large distributed systems ... by Spooky23 on Saturday February 19, 2022 @10:29AM (#30397796)
This is a religious debate among many. The IT/engineering nerd stuff doesn’t matter at all. Cloud migration decisions are always made by accounting and tax factors.

I run two critical apps, one on-prem and one cloud. There is no difference in people cost, and the cloud service costs about 20% more on the infrastructure side. We went cloud because customer uptake was unknown and making capital investments didn’t make sense.

I’ve had a few scenarios where we’ve moved workloads from cloud to on-prem and reverse. These things are tools and it doesn’t pay to be dogmatic.

Re:Ask HN: Do you find working on large distributed systems ... by sdoering on Saturday February 19, 2022 @11:57AM (#30398500)
> These things are tools and it doesn’t pay to be dogmatic.

I wish I would hear this line more often.

So many things today are (pseudo-) religious now. The right frsmework/language, cloud or on prem, x vs not x.

Especially bad imho when somebody tries to tell you how you could do better with 'not x' instead of x you are currently using without even trying to understand the context this decision resides in.

[Edit] typo

Re:Ask HN: Do you find working on large distributed systems ... by qorrect on Saturday February 19, 2022 @05:35PM (#30401183)
> So many things today are (pseudo-) religious now. The right frsmework/language, cloud or on prem, x vs not x.

Might have always been that way? We just have so many more tools to argue over now.

Re:Ask HN: Do you find working on large distributed systems ... by dekhn on Saturday February 19, 2022 @10:31AM (#30397822)
that cage is a liability, not an asset. How is the networking in that rack? What's its connection to large-scale storage (IE, petabytes, since that's what I work with). What happens if a meteor hits the cage? Etc.
Re:Ask HN: Do you find working on large distributed systems ... by fxtentacle on Saturday February 19, 2022 @11:06AM (#30398095)
That depends on what contracts you have. You could have multiple of these cages in different locations. Also, 1 PB is only 56 large enterprise HDDs. So you just put storage into the cage, too.

But my point wasn't about how precisely the hardware is managed. My point was that with a large cloud, a mid-sized company has effectively NO SUPPORT. So anything that gives you more control is an improvement.

Re:Ask HN: Do you find working on large distributed systems ... by dekhn on Saturday February 19, 2022 @12:10PM (#30398601)
"1 pb is only 56 large enterprise hdds".

umm, what happens when one fails?

With large cloud my startup had excellent support. We negotiated a contract. That's how it works.

Re:Ask HN: Do you find working on large distributed systems ... by fxtentacle on Saturday February 19, 2022 @01:25PM (#30399235)
Typically people use RAID or ZFS to prevent data loss when a few hdds fail.
Re:Ask HN: Do you find working on large distributed systems ... by dekhn on Saturday February 19, 2022 @01:27PM (#30399255)
OK, so basically you're in a completely different class of expectations about how systems perform under disk loss and heavy load then me. A drive array is very different from large-scale cloud storage.
Re:Ask HN: Do you find working on large distributed systems ... by fxtentacle on Saturday February 19, 2022 @01:55PM (#30399491)
Hard to say. My impression is:

- A large ZFS pool of SSDs is much faster than any cloud storage.

- Cloud storage failed much more often than the SSDs in our pool.

- "Noisy neighbor" is an issue on the cloud

Re:Ask HN: Do you find working on large distributed systems ... by qorrect on Saturday February 19, 2022 @05:36PM (#30401190)
This cracked me up. Thanks fxtentacle :D.
Re:Ask HN: Do you find working on large distributed systems ... by dekhn on Monday February 21, 2022 @10:00AM (#30416573)
of course, the reason that's wrong is that if one drive fails you don't have a 56pb storage system, you have something smaller because of redundancy.

That redundancy, and the performance that scales due to it, place cloud services in an entirely different class from on prem servers.