News for hackers. Stuff that matters.
Re:The Cloud by nostrademons on Wednesday August 07, 2019 @06:32PM (#20639907)
I wouldn't go so far as the author when recommending tech stacks for startups or hobby projects. The main benefit of cloud providers is that you don't have to learn and pay attention to all the stuff he lists, and you can focus on building the software you want to build.

But I do think he's performing a useful service in highlighting just how much you're paying for not having to hire a devops or sysadmin person. 10x performance differentials have been replicated not just by him but in other benchmarks. I remember seeing a chart when I was still working at Google and GCP was being justified that showed a graph of hard disk prices vs. S3 rates - since inception (2007-2012 at this time), S3 rates had gone down by a factor of about 2-3x, but price/GB of physical hard disk space had gone down by ~100x. Amazon is hiding all of this improvement in the physical hardware behind vCPUs and opaque billing, and it becomes pure profit to them.

I'm surprised more mid-size companies - those with AWS bills in the 6-figure-per-month range - don't leave the cloud, get physical hardware, and pay some sysadmins. At that level you could easily afford them, and you probably can get old Linux greybeards cheap now that everyone thinks the job description is obsolete.

Re:The Cloud by Slartie on Wednesday August 07, 2019 @06:45PM (#20640001)
> I'm surprised more mid-size companies - those with AWS bills in the 6-figure-per-month range - don't leave the cloud, get physical hardware, and pay some sysadmins.

I'm not surprised at all: that's thanks to the beauty of lock-in effects. Especially all those cloud-specific managed services make it really hard to switch, and even if you are treating the cloud as a pure VM platform, it is considerable work and risk to switch to a self-managed bare metal setup.

Basically, if you don't start bare-metal (incurring having to assemble the necessary know-how to do so) you will have a hard time to get clean from the cloud drug later on, when you think you're ready to afford having the know-how. I think this aspect has been a main driver behind the linked article.

Re:The Cloud by rzzzt on Wednesday August 07, 2019 @07:22PM (#20640267)
This is why I'd really like to see projects like Eucalyptus take off, along with (possibly) some standardization of "infrastructure API": https://www.eucalyptus.cloud/index.html
Re:The Cloud by derefr on Wednesday August 07, 2019 @06:50PM (#20640034)
S3 isn’t a hard disk; that’s EBS. S3 is ~17 copies on object-level-mirrored hard disks (allowing a choice of different mirrors per object, preventing overloading and rebuild degradation) attached to a lifecycle event queue, a load-balanced webserver cluster, and an async tape archive.

S3 costs pretty much what that infrastructure costs to run yourself.

But certainly, go ahead and run a Riak CS cluster (plus those other things) on top of some Herzberg servers, and try to outcompete S3 on price. I—and many others—will be waiting. :)

Re:The Cloud by paulddraper on Wednesday August 07, 2019 @07:09PM (#20640164)
The cost of S3 is pretty dang cheap for advertised 99.99% available, 99.999999999% durable storage: $0.02/GB/month.

And that's ignoring the Glacier storage class (which probably didn't show up on that historical graph), which is a fifth of that.

Re:The Cloud by votepaunchy on Wednesday August 07, 2019 @08:02PM (#20640520)
> And that's ignoring the Glacier storage class, which is a fifth of that.

And Glacier Deep Archive is a further fourth of that.

Re:The Cloud by cobbzilla on Wednesday August 07, 2019 @07:25PM (#20640294)
Bandwidth prices vary widely and some are bad, so the “all in” price can be hard to figure. It’s definitely cheap to let your data just sit there. Move it anywhere — it might get expensive.
Re:The Cloud by paulddraper on Wednesday August 07, 2019 @07:35PM (#20640355)
AWS charges $5 per million write requests, $0.40 per million read requests. Inbound bandwidth is free; outbound bandwidth ranges from free (within same AWS region) to $0.05/GB for public internet at scale.

The latter number can be pricey. But, irrelevant with respect to the failing costs of hard drives.

Re:The Cloud by dekhn on Thursday August 08, 2019 @10:48AM (#20645075)
BTW, I use a wonderful tool called goofys that lets you mount S3 buckets as local volumes. There are a number of constraints (can open for read, or write a file once) but it works well for a wide range of legacy applications.
Re:The Cloud by rsync on Thursday August 08, 2019 @11:25AM (#20645405)
"BTW, I use a wonderful tool called goofys that lets you mount S3 buckets as local volumes."

You may be interested in the 'rclone' tool which does all of these things, and more, with all of the cloud providers:

https://rclone.org/

Re:The Cloud by acdha on Thursday August 08, 2019 @11:28AM (#20645424)
You might want to highlight that rclone has FUSE support – you have to scroll down the page a bit to learn that your comment wasn't comparing unlike things:

https://rclone.org/commands/rclone_mount/

Re:The Cloud by dekhn on Thursday August 08, 2019 @11:46AM (#20645573)
Does it work better than goofys?
Re:The Cloud by jasode on Wednesday August 07, 2019 @06:59PM (#20640095)
>The main benefit of cloud providers is that you don't have to learn and pay attention to all the stuff he lists, [...] I'm surprised more mid-size companies - those with AWS bills in the 6-figure-per-month range - don't leave the cloud, get physical hardware, and pay some sysadmins.

$12 million/year for cloud seems like a high enough threshold to go internal but I'm not even sure that's enough money to sway executives away from cloud.

The main issue is that internal IT departments at corporations have slow service from the IT staff compared to AWS/GCP/Azure and its engineers. Corporate IT departments typically don't treat their coworkers in other departments as internal customers. Instead, they treat them as adversaries and a nuisance.

That's why at non-tech corporations, the first group that experimented with the new AWS cloud service were the development teams. They got fed up with IT service backlogs waiting for new dev/sandbox/text servers. They got tired of waiting for IT to requisition a server and then install the os, db, etc. If the programming manager then asked IT to install an Oracle update, the IT department might say "well, Susan our db admin is on vacation till next Monday so we'll get to it then." Those kinds of inter-department interactions were frustrating to the internal customers of the IT department. With AWS, the programmers got their sandbox servers in spun up minutes instead of weeks or months. Once the dev teams were sold, the more risk-averse and mission-critical production workloads eventually moved to AWS too.

The author of this article doesn't seem to understand the dysfunctional relationship between internal IT and their coworkers they serve. It's more about responsiveness and iteration speed of the AWS/GCP/Azure employees' vs the IT employees than expensive AWS EC2 compute power vs cheaper internal racks of cpu.

E.g. The Guardian pays AWS more for cpu+diskspace+network than what they can build on their own. That's the raw hardware costs. But it doesn't matter because Guardian's internal IT staff with an internal cloud stack just can't match AWS: https://www.computerworld.com/article/3427004/the-guardian-g...

To me, it totally makes sense for Dropbox to migrate off of AWS to save money. They're a tech company and have the engineering culture to do it. A lot of non-tech companies (like The Guardian) don't.

Re:The Cloud by zrm on Wednesday August 07, 2019 @07:34PM (#20640348)
> With AWS, the programmers got their sandbox servers in spun up minutes instead of weeks or months.

That's how it starts but not how it ends.

Once you move everything to the cloud, the same IT staff get put in charge of it. Because they have a requirement from somewhere up high to make sure everything has antivirus on it, and there is a Windows domain for the desktops so the servers need to be in it too, and they want to have a test environment with specific versions of specific software matching the ones in production which means you have to use the corporate VM images instead of the AWS ones and have IT go through their VM provisioning checklist for each new instance, etc.

And some of those reasons are bad and some of them aren't, but either way they exist, so if you want to provision something from AWS now it has to go through IT who takes just as long as they ever did to do what amounts to the same thing as before only now it's more expensive.

Re:The Cloud by johngalt on Wednesday August 07, 2019 @08:43PM (#20640768)
Counterpoint from someone who did developer support back in the pre-cloud days.

Even pre-virtualization, I could instantiate servers very rapidly with network based imaging. With pre-staged templates of various os/program combinations. Applied them to both lab and production deployment. So that Dev, QA, and Prod could all work from the same base images. With versioning of the images based on patch level.

Then I would subsequently get blamed for every program error by the dev team. With no other justification than "it worked in the lab. Fix your server!" Running a packet analyzer, I would often find things like queries trying to go to a lab db server rather than prod. Or a dependency on an out of date/unpatched/vulnerable version of some supporting software that the development team would then attempt to pressure into production rather than update their code.

For every obstinate sysadmin you've encountered, there is an ops pro who has received a ticket with nothing but a list of compiler errors combined with 'pls fix asap'.

Re:The Cloud by clhodapp on Wednesday August 07, 2019 @07:12PM (#20640187)
>Corporate IT groups typically don't treat their non-IT coworkers as customers. Instead, they treat them as adversaries and a nuisance.

One deep problem with the "internal customer" model is that you don't get a larger budget when you serve your "internal customers" better.

Re:The Cloud by GFischer on Thursday August 08, 2019 @06:04PM (#20649319)
I have replicated the performance differentials (for Azure), and they definitely suck.

But people are REALLY REALLY expensive (not only the ongoing costs, but hiring, firing and overhead), and Cloud has another huge advantage: accounting tricks (which help a lot to said midsize firms).

Literally one of the first things they teach you at the Azure training is the difference between CapEx and OpEx: https://docs.microsoft.com/en-us/learn/paths/azure-fundament...

Re:The Cloud by ianmobbs on Thursday August 08, 2019 @08:03PM (#20650179)
Re:The Cloud by spydum on Wednesday August 07, 2019 @06:43PM (#20639986)
because saying your business is built for and in the cloud gets you investors and street cred. further, if you are having a bad quarter, you can squeeze your infra to lower opex. If you built all onprem, you had to invest capex heavily upfront, and you can't slow down your opex very much in the bad times.
Re:The Cloud by wyclif on Thursday August 08, 2019 @12:43AM (#20641910)
I know the CW is that sysadmin/devops is being automated away, and a lot of people think programming is about automating all the things (or at least as much as you can), but doesn't what you're saying prove that they aren't obsolete for large-scale providers?