News for hackers. Stuff that matters.
Re:The many lies about reducing complexity part 2: Cloud by harikb on Sunday January 10, 2021 @12:16PM (#25716836)
+1

I bet cloud providers are incentivized not to provide detailed billing/usage stats. I remember having to use a 3rd party service to analyze our S3 usage.

Infinite scalability is also a curse - we had a case where pruning history from an S3 bucket was failing for months and we didn’t know until the storage bill became significant enough to notice. I guess in some ways it is better than being woken up in the middle of night but we wasted millions storing useless data

Azure also has similar issues - deleting a VM sometimes doesn’t cleanup dependent resources and it is a mess to find and delete later - only because the dependent resources are deliberately not named with a matching tag.

Re:The many lies about reducing complexity part 2: Cloud by zmmmmm on Sunday January 10, 2021 @04:27PM (#25720090)
> Infinite scalability is also a curse

People don't like to admit it, but in many circumstances, having a service that is escalating to 10x or 100x its normal demand go off line is probably the desirable thing.

Re:The many lies about reducing complexity part 2: Cloud by Silhouette on Sunday January 10, 2021 @06:07PM (#25721302)
There seem to be plenty of successful online businesses that share some information about their back end infrastructure, and when you look it turns out they just have a handful of web servers running their app behind a load balancer, enough DB servers for the throughput and redundancy they need and some sort of failover arrangements in case something dies, and some sort of cache and/or CDN arrangements just for efficiency. Even if they're running in the cloud, it's probably just a load of manually allocated EC2 instances, maybe RDS to save the hassle of managing the database manually, and maybe S3.

I wonder how many businesses exist in the entire world that truly need to scale their server counts up or down by a factor of several so quickly that it has to be done automatically and not as a result of, say, daily human reviews of what's happening. I feel like even large businesses that need to run thousands of servers at all are probably relatively rare, as indeed large businesses themselves are. The number that might realistically need to scale from say 1,000 web servers to 1,500 within a matter of hours must be even rarer. But I have nothing even resembling a useful data source to tell whether this intuition is correct.

Re:The many lies about reducing complexity part 2: Cloud by threentaway on Sunday January 10, 2021 @01:00PM (#25717379)
This seems like you didn't have proper monitoring and alerting set up for your job, not sure how that is a downside of AWS.
Re:The many lies about reducing complexity part 2: Cloud by milesvp on Sunday January 10, 2021 @04:41PM (#25720253)
> Infinite scalability is also a curse

This was the key sentence, I think. This type of problem actually shows up in other domains as well, queueing theory comes immediately to mind. Even the halting problem is only a problem with infinite tape, and becomes easier with (known?) limited resources.

When you have some parameter that is unbounded you need to add extra checks to bound them yourself to some sane value. You are right, in that the parent failed to monitor some infrastructure, but if they were in their own datacenter, once they filled their NAS, I’m positive someone would have noticed, if only because other checks, like diskspace are less likely to be forgotten.

Also, getting a huge surprise bill is a downside of any option, and the risk needs to be factored into the cost. I’m constantly paranoid when working in a cloud environment, even doing something as trivial as a directory listing from the command line on S3 costs money. I had a back and forths with AWS support just to be clear what the order of magnitude of the bill would be for a simple cleanup action since there were 2 documented ways to do what I needed, and one appeared to be easier, yet significantly more expensive.

Re:The many lies about reducing complexity part 2: Cloud by jjoonathan on Sunday January 10, 2021 @04:31PM (#25720144)
AWS monitoring (and billing) is garbage because they make an extraordinary amount of money on unintentional spend.

"But look at how many monitoring solutions they have in the dashboard! Why, just last re:invent they announced 20 new monitoring features!"

They make a big fuss and show about improving monitoring but it's always crippled in some way that makes it easy to get wrong and time-consuming or expensive to get right.

Re:The many lies about reducing complexity part 2: Cloud by catlifeonmars on Monday January 11, 2021 @02:09AM (#25725690)
I’m genuinely curious. Which parts of monitoring is crippled or difficult to use?

Disclaimer: I work at AWS.