I see everybody around me moving to cloud, without really good explanation why. Only reasonable thing I can see as an pattern is that cloud experience on top of data things gets paid 30% more. It made me consider cloud a lot.

I was considering switching to cloud, just so I can put in my CV "experience with migration to cloud".

For next person commenting that it makes sense: it doesn't with 200gb database and super predictable workload, growth and usage.

Depends on the company. I've been working for marketing agencies for the last 15 years and they're generally staffed by, at most, 1 IT person who is in charge of a third-party vendor relationship that offers managed IT services. Those IT resources (internal or vendor) don't specialize in data and often don't know how to deal with it well, predictable workloads or not, and often offer up solutions which are not appropriate (cost or otherwise) whereas there are managed solutions from cloud vendors that do. BigQuery, for example, handles compute and storage for you and rolls it into one reasonable query price. No need to worry about any management of anything there and just sit Data Studio (included) or any other BI tool (Tableau, etc) on top and you're good to go.

I get your skepticism and welcome it, but you're being a little rough. We're cloud native at my current company (I did a partial cloud migration at my last which was completed after I left) and it makes my life leading a Data Engineering and Data Science team MUCH easier without upfront hardware/software costs or long-term contracts, which were STAGGERING and left us with much more difficult to maintain, upgrade, etc hardware that took up most of my job as opposed to almost none of it today.

YMMV.

For anything above what we have I think it weirdly depends on country/ salaries.

In US, 100k per year is no brainer for cloud as even 1 FTE would cost much more.

In non-western Europe, 100k is deal killer as you can hire 2 senior DBAs and you still have enough money for quite a reasonable server.

Edit: sorry I see that you are the same person I was replying to. Then to understand it better, who is consumer of so vast amount of data?

That is something completely different from migrating your inhouse applications to the cloud by introducing kubernetes. I think your example show nicely that there is not one solution for everything. As an industry we have to put way more emphasis on deciding tradeoffs by understanding requirements and risks.

People just buy into the "cloud" marketing. They don't have the ability to think and reason, and so don't understand that "cloud" just means "renting someone else's computer."

I built a complex in-house medical system. Quick, reliable, and liked by the users.

I was in the middle of adding a major new feature when all of the management in the IT department quit. The new people immediately decided that the whole thing had to be "in the cloud." I was removed from the project, and they hired three people full-time to rebuild it.

That was three years ago. The new system is still not online. The users are still using my old system, and the feature I was working on never got added because my presence and input was not welcome because I "don't understand the cloud." So I got moved to other projects.

People talk about "the cloud" with the same fervor and language as members of a cult. And you will be an outcast if you dare challenge their way of thinking.

Moving me out brings no advantages to either part of the org. In fact, it's probably a disadvantage to IT because it has to burn three people from its allocated headcount. Where previously, it had a bonus person that didn't come out of the IT budget.

The difference is the abstraction layer. I would define “the cloud” as a layer that abstracts away the physical infrastructure. (Which works until it doesn’t but that’s a longer comment.)

You can even apply “the cloud” to your very own fleet of computers with the right software.

Like I said in the original comment, it’s a specific solution to a specific problem. Whether it’s right for your system I have no idea.

If it's on-prem, it's not a cloud, it's a fog.

At least, that's what I've been calling some of my deployments.

It sounds you're trying to dismiss and downplay solutions to problems you're oblivious to.

Even if you want to simplisticly frame "the cloud" as "renting someone else's computer", keep in mind that:

* It's not one computer but as many as you'd like, and get them at the click of a button,

* These are computers which are managed 24/7 by a team of highly trained specialists,

* These computers can be located anywhere in the world and simultaneously in multiple regions,

* These computers are designed to be fault-tolerant and handle (and survive) way more stuff that can conceivably be thrown at them.

> People talk about "the cloud" with the same fervor and language as members of a cult.

I'm sorry to say but you sound like you're militantly opposed to a solution to problems you either don't understand or refuse to understand.

"The cloud", even when following the simplistic and clueless belief that it's just "renting someone else's computer", solves whole classes of technical and business problema that your box under your desk cannot solve.

By the way, moving 'to the cloud' never means a specific thing because people have made words (intentionally?) vague to the point where you have to explicitly specify all the factors you take into account with your work in order to figure out which 'cloud' they had in mind.

Running a static workload doesn't require elasticity, but a 'cloud' isn't just elasticity. If you want "a program on a server with some storage that is always there" without having to deal with hardware, software, networking, storage, backup, maintenance, upgrades, service contracts etc. then an arbitrary virtual machine where someone else takes care of the rest makes total sense.

I'm not a bean counter, all I know is those guys at my last job would rattle off about it like zombies. IIRC its a tax thing

Is that really the only benefit?

Depending on taxation rules you might want to buy or lease.

In that case you certainly aren't looking very hard for that explanation.

The rationale of moving to "the cloud" is crisp and very well-defined, to the point where even AWS makes it their point to explain in very clear terms in their Architecting With AWS courses.

Basically it's all about being able to scale without bothering with procurement, not having to manage everything down to power bills and rentals, turning big capex into small opex, and being able to deploy globally and reliably with a click of a button.

That, however, comes at a steep cost. The more serverless you get, the higher the price tag.

Also, beyond a certain scale you're better off going back to managing your own infrastructure.

> Only reasonable thing I can see as an pattern is that cloud experience on top of data things gets paid 30% more. It made me consider cloud a lot.

People with cloud experience are paid more because they can deliver more value. A random guy with, say, experience in EC2 and CloudFormation and CloudWatch, can single-handedly put together a robust fault-tolerant webservice that's deployed globally and automatically scales to meet any demand fluctuation.

Let me fix that for you! :)

I work for a Fortune 200 company who has their own data centers and has for decades. We just completed the construction of two new data centers about four years ago at a cost of $110 million dollars. Those new data centers are now at 70% of their capacity for power requirements. You should take a look at the specs on Intel's latest server-class processors. It's insane! We're talking half horse power and higher for each processor - and you want blades full of these things and then a rack of those blades and a row of those racks! Data centers now consume more power than industrial manufacturing! Most maddening of all - most of that power is going to go to heat, and guess what? You need giant chiller units to cool the place down. The power costs alone are insane.

You also lose agility. Need a new server? Go to your favorite cloud console and provision it. Better yet, use a serverless architecture and don't even worry about servers! Your own data center? Right now there's a 4-6 month wait time for new servers and storage equipment, and an 8-12 month wait time for networking equipment. Not exactly agile, is it?

You already know you need to staff to manage and patch your servers but you're also going to need staff to procure and manage your software licenses - and those licenses aren't cheap! Don't forget the ongoing tail - you pay for the license and then you get to pay 20% per annum forever after for that license. Those licenses also restrict your agility - you're pressured to use the software that's licensed in order to get better value. Try working with a procurement-based architecture!

You think you're going to avoid the software licensing hell by using open source? Good luck with that! That means you are responsible for the packaging, distribution, and support of whatever it is you're utilizing. That's more staff you need. They will inevitably miss patching some critical vulnerability that's going to land you in the news!

I can go on, and on, and on. But here's one thing I can say about applications that we've moved to the cloud: they cost 30% as much to host in the cloud as they do on-premise, and that's not even accounting for the entirety of all the costs I enumerated above! We have ultimate flexibility and agility, and we can quickly utilize managed open source platforms to solve business problems. On-prem? You lose all of that.

Faster, cheaper, better - that's why people are moving to the cloud.

Sure, that all works, as long as you have infinite money. My last job was at a company that was heavily invested into a serverless architecture, using the full suite of AWS tooling. They did an audit of one of their codebases and they found that generating a particular piece of data cost $20, almost entirely in fees to AWS. The company's roadmap involved scaling up the generation of this data, so they embarked on a huge process of optimizing and refactoring, to try to get the cost down. This effort tied up probably 40-45 engineers for at least six months. Probably cost them roughly 3 million dollars in salary alone, let alone opportunity cost in terms of other projects not delivered. No one amongst the management ever seriously considered moving to an on-prem solution, even though it was blatantly clear that the only value the company's massive usage of AWS Lambda, SQS, RDS and other managed services was delivering was additional profit margin for Amazon's balance sheet, and resume line items for the engineers who now got to check off the "used Terraform" box when applying to their next job.

Every app I've migrated to the cloud so far has resulted in a 70% cost reduction, but I'm also not trying to take my entire application portfolio to the cloud. To wit, we're not planning on abandoning our data centers anytime soon and that's not just because we can't move our applications to the cloud fast enough. At this point in time we recognize there are some applications best kept on-premise - but they're a minority, maybe 20% tops of my application portfolio. That means 80%+ of my application portfolio can be moved to the cloud, makes sense to move to the cloud, and I can save 70% on costs by doing so. That's a fantastic deal!

You do raise a fair point - what about the applications that are more economical to host on-prem? In my own portfolio I estimate 20% or so of my applications make more sense to host on-prem. What to do about those?

The most practical option is to mandate every application you build in the cloud have a plan for how to run on-prem. Note those that would be very difficult to run on-prem - those are your key cloud dependencies. For example, in my own portfolio I have a couple of applications depending on Lex and therefore would be very difficult to host on-prem.

Pay special attention to your so-called "crown jewels" - your apps that distinguish you from your competitors. If those were dependent on a key technology such as Lex and would be difficult to migrate on-prem then you need to identify other cloud providers you could migrate to. In this particular example I know Microsoft's and Google's clouds both offer similar services.

If you design and build your applications with no thought of the future then you're already in a bad position.

On-prem or cloud, why would I be waiting until 1 month before launch to provision anything?

You're pretending like on-prem is full of all these physical limitations, but Amazon, Microsoft, Google, Oracle, et. al. can just magically make hardware appear out of nowhere. It's the same hardware, subject to the same supply limitations, and I'm going to be paying either way. Either I pay more per hour for my cloud instances, or I pay a bunch up front to provision my on-prem servers.

Cloud makes sense if the load on your service is intermittent or unpredictable enough that you're okay paying AWS's premiums in order to have the ability to scale on demand. But if the load on the services is knowable, then hosting that service on the cloud is paying a 25% markup for no reason whatsoever. AWS is by far the most profitable portion of Amazon's business units. Those profits are coming from somewhere. They're coming from your business.

Just as an example, consider Hadoop and its ecosystem. Another example is Elasticsearch. There's work involved in making these platforms production-ready and keeping them and their dependencies up-to-date and ensuring all your applications in your portfolio are using the same set of software. Most organizations are not prepared to take that on. So they turn to a vendor who will do it for them for much cheaper than they could do it themselves.

That's why the saying is "free as in freedom, not free as in beer" because open source software is not free as in beer. At least not when factoring in the total costs.

The packaging argument was made in the context of complying with software licenses. Responsible companies which perform due diligence on the software they run have to track the provenance of all software that ships as part of their dependency closure. If you want to ensure you're not vulnerable to lawsuits then you can't simply apt-get stuff from a PPA. You need to build it yourself, and track exactly what goes into that build.

Meanwhile, if you opt to run managed service from a cloud provider, you don't have to bother with that because that's not your problem (or liability) anymore.

Yes, and it does indeed apply to all readily available container images.

In fact, it applies to any and all software packages put together by third-parties.

I mean, who in their right mind downloads random stuff from the internet and expects to just drop it in production software which you build your business upon?

Do people successfully lawsuit their cloud providers for downtime?

My company offers two deployment scenarios: host it yourself and cloud hosted (SaaS). Many of our customers choose the latter because their internal IT systems require more process, and they just want to get something up and running.

With Postgres[1], you would need to be using an absolute beast of a server to be paying that much, even if you're paying the 2x premium to go month-to-month, and a low-powered machine could be around $50/month for a 200GB DB.

[1] https://azure.microsoft.com/en-us/pricing/details/postgresql...

Based on link you provided, I calculated that 40gb memory would be $290 and 80gb $570 per month (in UK).

So price seems still quite steep, considering that same specs on-site would be ~$2000 one off. (8 months return on investment).

Then for this server, I honestly don't believe its easier to do maintenance in cloud, as you need to do lots of initial configuration there (be it IAM, permissions, firewall etc. ) compared to have it just in local network

A single box on Hetzner is by far not comparable to, say, using a managed service as AWS RDS. It's not even apples to oranges, and more of apples to freshly-squeezed orange juice served by a buttler. Think about it: do you get any form of fault tolerance with your single box?

Ah yes I agree, sorry for not being clearer. I wanted to add on to what you said.

Even though I'm a big fan of Hetzner and a happy customer for years, it's important to know what we're buying and be mindful of what we're not having as a tradeoff.

There is nothing to say against a cloud version of your office suit but otherwise it is quite underwhelming. Short lived applications with a 90% API because the developer want to keep the door open to imprison the users once the application has captured a critical mass of users.

I host on AWS because I don't pay the fairly high invoices. Less domain and TLS maintenance for me. Personally I rent a server and host my personal stuff there including software repositories.

edit: Guess that technically would also qualify as cloud, but I think a differentiation is necessary. Cloud mostly means product x or y instead of just another hosted server.

You say that like a pay rise of 30% is not a good enough reason all by itself for many people.

> For next person commenting that it makes sense: it doesn't with 200gb database and super predictable workload, growth and usage.

Perhaps not for technical or business financial reasons, but those are not the only possible reasons someone might do something. As mentioned before it can make a lot of sense to migrate to cloud if it means you can get a 30% pay rise.

As far as I can tell it only makes sense if you have a ton of data, and only ever use a fraction of it at a time, for one-off jobs, infrequently.

Instead of applying knowledge, he would bring up all these buzzwords to meetings and not really understand what he was talking about. I get angry just thinking about it.

Nothing in your comment points to resume-driven development. You even failed to mention any project or design decision. You just decided to get angry at a coworker because of something.

I believe these two phenomenons are producing a positive feedback loop in the industry. Selected technologies address one challenge, but introduce complexity. The complexity becomes difficult to manage. So other technologies are incorporated to manage the complexity. In the midst of all this, core technologies are replaced, swapped out, or transitioned to under the desires of the dev(s). For example, switching from one js rendering library to another, while preserving the old legacy code. The complexity footprint keeps growing and it doesn't stop because the engineers themselves aren't entirely committed to the project. They can incorporate the new technology to the project, pad their resume, and bail to a new employer if things grow out of wack.

I spend more time trawling GitHub issues to find workarounds than I do actually using the tools.

Or just no real incentives of the people involved to do so. As a dev, I don't get any real credit for biz outcomes.

That's a great way to describe the phenomenon.