Plus, as mentioned, you get a vast menu of services to choose from, all of which are managed. Just in terms of databases alone, you could go with RDS (raw MySql/PG), Aurora, Dynamo, Elasticache/Redis, DocumentDB (Mongo) and more. Plus managed backups, failover, multi-AZ, security, and connectivity built-in.
If your team is filled with devops engineers, then, sure, go with the non-AWS route. But it's a lifesaver for agile companies who can't afford a lot of infrastructure work and need easy levers to scale, and can afford a modest markup.
This seems backwards to me - running a simple VPS on something like OVH/DigitalOcean/Linode is a matter of creating the instance, setting the size, and setting up your server software. Super simple, and likely about the same complexity as setting up a dev environment.
Setting the same thing up on AWS requires slogging through the documentation of their many services, deciding what makes the most sense for you (EC2? ECS? EKS? Lambda? Fargate?), and then configuring more options with less-friendly UI than any other provider I've seen. If you can fully wrap your head around all those services, create a stack that makes sense, and maintain it long-term, you're probably well on your way to being qualified for a DevOps engineer job. I can spin up a quick DigitalOcean droplet for a new project in less than half the time it'd take me to set up an EC2 instance, configure the exact amount of storage I need, and pull up 2 additional browser windows to determine the specs and pricing of different instance sizes in different regions.
Until that server goes down.
I wonder what percentage of all sites hosted at AWS are actually ready for a zone to fail.
From OVH VPS. Funny thing it's 666 today. YMMV
# uptime 01:47:03 up 640 days, 6:53, 2 users, load average: 2.39, 2.62, 2.73
From hetzner dedicated server. YMMV
# uptime 01:48:11 up 482 days, 20 min, 2 users, load average: 1.53, 2.10, 2.78
From leaseweb dedicated server. YMMV
03:48:56 up 238 days, 15:04, 1 user, load average: 0.00, 0.01, 0.05
Having run a simple web server + database on both EC2 and Digital Ocean I disagree. The two services are nearly identical at that scale.
In fact I don’t see any significant differences between any of the proper cloud providers at this level. The cost & functionality are very nearly the same.
SO's read to write ratio is enormous. While they talk about a handful of different bare metal servers a vast majority of their hits are handled by their caching layer which is not one of their bare metal machines.
Their write loads are not explicitly time sensitive either, if there's a delay between a successful post and cache invalidation it's not a big deal for a majority of their traffic.
Not every model is quite so forgiving. But even then there's a lot of stupid wasteful "infrastructure" made by people who think they're going to be Google, or at least tell their investors they'll be Google.
StackOverflow is useful because it reminds people that machines are fast, and you probably don't need that many of them to scale to large sizes. StackOverflow is used by the entire global population of developers more or less and it runs off of one large MS SQL Server + some web servers. No auto scaling (not enough hw expense to be worth it), no need for fancy cloud LBs etc.
Sure maybe your service is gonna scale to more than the world's population of developers. Great. But ... a lot of services won't even go that far. For them it's hard to conclude it's really needed.
To pull off a StackOverflow you do need skilled sysadmins though. As the article points out, a surprisingly large number of people who call themselves devops don't really know UNIX sysadmin anymore.
Changing capacity automatically always has the potential to backfire. And as AWS & Co need to keep those servers running during times of less demand there's no way it's cheaper unless you have really unusual traffic patterns (even then it's probably not).
StackExchange is largely a CRUD app. High volume of tiny requests that hit the app layer, then the database, and back. Other organizations have lower volumes of compute-intensive requests, async tasks, etc.
With respect to the size of an organization, the cost of coordinating deployments and maintenance over a handful of servers grows with the size of the organization. It frequently behooves larger organizations to allow dev teams to operate their own services.
None of this is to say that there isn't waste or poor decisions throughout our industry; only that it's not the sole factor and SO's isn't the ideal architecture for all applications.
> my guy
I'm not your guy.
Comparatively speaking even compare them to the same CRUD app and DB, most are using 5 to 10x more servers with 1/2 to 1/5 of the traffic.
edit: To be clear, not saying SO isn't an achievement, but its one type of use case that yields a really simple tech stack.
https://stackexchange.com/performance
Their DB handles peak of 11,000 qps and peaks at only 15% CPU usage. That's after caching. There are also some ElasticSearch servers. Sure, their traffic is heavily read only, but it's also a site that exists purely for user-generated content. They could probably handle far higher write loads than they do, and they handle a lot of traffic as-is.
What specific complexities would be introduced by an even higher write load that AWS specifically would help them address?
No doubt, but how large are those other CRUD app organizations? Do they have a staff that is 20x the size all trying to coordinate deployments on the same handful of servers? What are their opportunity costs? Is trimming down server expenses really their most valuable opportunity? No doubt that SO has a great Ops capability, but it's not the only variable at play.
Just look at the modern SPA. They are slower than ones 10 years ago, and the JavaScript VM is much faster plus all the hardware gains. Why does it make Twitter and Gmail a few seconds to load?
Not all startups have the server and networking know-how to pull that kind of stuff off, or even set it up in the first place.
https://blog.serverfault.com/2015/03/05/how-we-upgrade-a-liv...
I also think colocation or own datacenters are a poor fit for most. But dedicated servers are underrated. They can be offered much cheaper as it's simply renting standard hardware and you don't need any of the expertise or size you'd need for colocation.
A startup doesn't need AWS right off the bat. Planning for scale right from the beginning is a way to quickly bleed $. Of course, if you have VC money, why not spend that cash right?
Where I work we've started non-AWS and have continued non-AWS. We don't have a team of devops engineers, but rather a team where there are a engineers who _can_ do devops. I dread the day we need to move to AWS, but it's much easier moving to AWS than off it.
You think you want Elasticache? It's a fifteen minute operation. In that time your devops guy isn't going to have even downloaded the binary and figured out the docs.
Sure, you'll always have idle capacity, but this way you could use it. With AWS, Amazon runs that idle capacity and charges you for it.
If it can be undone in minutes, it isn't much of a decision. A service that can be enabled or canceled on a whim is unnecessary.
Realistically, analyzing the guarantees offered by a cloud platform, your corresponding requirements, and how everything is supposed to work is going to take days, and actually developing and testing disaster recovery procedures is going to take even longer.
The scale can happen once you've validated your startup, and when that happens it's a lot easier to just turn on 30 more EC2 instances than to get on the phone with Dell.
Look at it like this; GitHub mostly ran in their own datacenter using AWS only for spillover capacity. They scaled just fine. Virtually no apps have the problem of runaway growth that they can't handle without instantly provisioned resources.
To actually do more advanced stuff (the thing that AWS is good for) and utilize tools such as Terraform, you'd essentially need to hire engineers that are experts in AWS in addition to engineers who can do devops, as there's only so much "magic" AWS can provide.
Especially when the alternative is to hear a lot of, "What? You don't expect to scale? I thought you took this seriously."
I'm with you until that statement. AWS is nothing approaching "modest" in their markup. 20% minimum and typically much higher if you know how to negotiate when purchasing your on-prem gear. And if you happen to be a shop that sweats assets for 7+ years that number starts being measured in the hundreds of percentage points more expensive.
As a example, 20TB of egress bandwidth will cost you around $1,750 on Azure, almost $90/TB! Your typical server/VPS provider gives you that kind of bandwidth allowance for free, because it costs them close to nothing.
I have a feeling almost all complaints about cloud costs would disappear forever if they'd only stop gouging customers on bandwidth.
The likes of Hetzner and OVH are starting to offer more cloudy features, like load balancers and block storage. My hope is that eventually they become viable alternatives to the big 3, and they are finally forced to reduce their ludicrous bandwidth prices.
"Come on in; there's no commitment. Pay as you go, and did I mention our ingress rates are free! Bring us your data and give the cloud a try. It's the future, you know..."
"Oh you're leaving and want your data back now?"
A 20% premium on a $10k/mo server bill is an incredibly small premium to pay versus a single FTE.
Basically, if you model senior eng time as $5k/week -- not even counting the opportunity cost -- you'll understand why AWS.
While I definitely think AWS is very expensive for the servers, they are not overall expensive. Again, setting up CloudWatch in an afternoon or dumping tons of heterogenous logs into S3 and querying them with Athena or spinning up full environment stacks (front end pool, back end pool, load balancers, databases, vpcs, vpns, etc) is an enormous cost savings vs finding, installing, configuring, and maintaining batches of software to do the same.
edit: pay an eng $180k. Fully loaded, you're looking at $220k plus. Add in either recruiting time or a recruiting fee that probably runs us $2k/mo amortized over the first year on the low end. Add in the fact that the first 6 weeks weren't super productive, and also absorbed tons of time from other SE to get this person up to speed. Add in PM time bringing this person up to speed on our product.
Yup, $5k/week.
Clients on AWS were always my best ones, because they were used to paying so much over the odds I could set my rates much higher.
They didn't want to increase the head count. The AWS spend was a different silo/team/contract.
Corporate accounting is insane.
It almost smelled like "well, we negotiated this great AWS discount, so it doesn't bother us that we are spending an extra million. We're getting it at a DISCOUNT!"
Is there some MBA trend where every headcount is some ticking time bomb of liability that is 10x worse than their yearly salary?
You spend 150k on servers, they don't improve over a year.
You spend 150k on a decent employee, not even rockstar ninja 100xer, and they will deliver useful, although sometimes hidden, productivity and optimization.
I've yet to see an AWS deployment be competitive on cost with onprem or managed hosting. In fact, I used to make good money helping people cut costs by moving off AWS.
That includes paying more for devops, because AWS is incredibly complicated, and most people struggle to get it set up well.
There are valid reasons to pick AWS. Cost is not one of them.
> The overhead of bare metal on top of cloud VMs is basically knowing how to handle RAID and set up load balancing.
This implies human beings to handle raid and set up load balancing, which suggests that you need to compare the cost of cloud providers with the cost of hardware and the cost of those engineering resources.
In addition to RAID and load balancing, most organizations/applications also need networking (good luck balancing load without a network), databases (including backup management), TLS, DNS, access management, etc, etc. All of this takes humans to build and operate. AWS services do a lot of this for you. In the on-prem world, you have to build (or buy/integrate) this yourself, but that's not free so you have to account for that cost in your comparison.
You can still make the argument that on-prem is a better value proposition when accounting for the total cost of ownership, but that's a different argument than those which ignore engineering costs altogether.
However for the many, many people who do already have those skills, and for the many pieces of software that are basically web servers + databases, the overhead of bare metal vs cloud VMs or services is basically a bit of Linux sysadmin work. People are acting like you have to hire a full time wizard to even consider using anything other than high cost AWS services but that is wrong: AWS requires learning too, people who understand UNIX are plentiful even if apparently not as plentiful as they once were, and a well run setup will not require constant sysadmin work. A well rounded developer will often be able to do the ops work as well.
Also "bare metal" doesn't mean running your own datacenter. You can buy hardware, send it to a colo and they'll rack it and run the network for you. That's why I say it's mostly a matter of understanding RAID: when the hardware fails (almost always disk or less commonly, RAM units), you file a ticket, blink the HDD LED and some remote hands go and swap out the part for you. Those "hands as a service" come cheap compared to AWS.
They've not seen servers with IPMI tied into PXE boot and a TFTP setup. They don't realise once that server is racked up, we can log in to an IPMI console, power it on remotely, have it boot straight into a bootstrap script, remotely install an OS like CoreOS/Flatcar without manual intervention, have orchestration scripts install the necessary basic services on it without manual intervention, and boom, it's part of your own "private cloud" ready to deploy containers to that may well not require maintenance "below" the container layer for years.
And they don't realise you can contract someone to do this for you for low enough day rates that it'll take you a truly massive setup before you're paying the equivalent of a full time engineer.
Meanwhile we've had 20 years of media and culture telling people that those who get rich are the software guys, never the infrastructure guys, and that the easiest way to get into the industry is by learning web design.
Also, I suspect the industry is exhausting the pool of talent that learned computing on UNIX workstations at university in the 80s and early-mid 90s. The people who lived and breathed servers and UNIX are retiring out of the workforce, and not really being replaced. AWS is stepping in the fill the breach by selling sysadmins-as-a-service, basically, with a thin veneer of GUI programming slapped on top, and some proper documentation writers. Selling pickaxes and shovels to the gold miners - classic.
Finally, I think a lot of firms have developed a "no contractors" policy. I've never worked at a company that allowed contractors and I'm about mid-way through my own career, a bit less. At some point it has become received CEO wisdom that they're building a committed team of true believers, so "mercenaries" aren't welcome. It leads to the kind of thinking you see in this thread where "I don't have the skills, therefore I must hire someone full time, and they must earn as much as me, therefore it's too expensive" ends up leading to "and therefore I must outsource to a firm that 'contracts' for an hourly rate billed via credit card".
What’s the connectivity into your 7+ year old asset cost? What about the space, power and cooling to provide it a home?
What happens when your environment gets attacked by a volumetric DDOS, is that 100Mbps circuit going to get overwhelmed?
I’m not arguing AWS is cheaper, simply that there are lots of things around the physical server which also have costs and complexities.
10Gbe ethernet was standard in 2013/2014. Space power and cooling is company dependent. If they're tiny, a closet in the building or a colocation facility, if they're larger, a room. If they're huge, a datacenter.
>What happens when your environment gets attacked by a volumetric DDOS, is that 100Mbps circuit going to get overwhelmed?
I can't recall the last time a customer got hit by a "volumetric DDoS". End-user facing systems sit behind cloudflare. Internal apps are... internal. Who cares if the WAN connection is getting hit by DDoS when your users are on campus? In post-covid world, they've got multiple tunnels with SD-WAN.
That's ignoring that fact that Centurylink or Comcast or Telia are pretty good at mitigating DDoS in 2020. Just because they don't post about the attacks they endure on a blog doesn't mean they're inept.
https://business.comcast.com/help-and-support/ethernet/comca...
https://assets.centurylink.com/is/content/centurylink/centur...
>I’m not arguing AWS is cheaper, simply that there are lots of things around the physical server which also have costs and complexities.
Sure, and they're all complexities which we solved 30+ years ago.
The argument isn't that they are solved or not, but the cost of implementing and maintaining them vs AWS.
To me AWS killer features are RDS and ability to provision 100 servers in a few minutes. Most of the time scaling is not needed and RDS alone cannot justify the cost of AWS.
If you include the growth in electric efficiency over time, it can be bad to run old servers after a certain point.
I spend half my time debugging Hyperscale Bullshit™ that the client never needed and that failed the moment the devops guy left.
If you want this stuff, you need a long term tech team, and a devops member on hand. Every time you hand your project around this stuff costs you hours. You also need to tell your devs to use the devops guy like tech support. They built this rickety tower of obtuse config files, make the devops guy fix it and don't let your devs spin their wheels for 4 hours before their assets compile on the Super Cool Distributed Docker VM microservice stack of teetering jenga blocks.
But I do recommend companies have a means of deploying stuff to AWS and doing basic security there for clients that really require AWS. Having a VPC ready to go costs nothing operational and is good for at least having some semblance of collective skill with a major public cloud provider.
Invariably though when I see undisciplined developers take on colo style hosting systems become brittle and unable to accept changes and with any success it becomes a Herculean task to deploy new features without putting something important at risk. This results in more time spent on systems archaeology for new engineers eventually than creating new features or improving maintainability.
In 2020 I’d recommend developing as much as possible on PaaS type platforms like Heroku, a managed K8S, or even App Engine to avoid more bike shedding that really doesn’t buy a company anything materially advantageous early on like deciding which kind of EC2 instance to standardize on. Until engineers know what to optimize and concentrate upon in terms of process and there’s perhaps tens of thousands in monthly infrastructure costs most skilled (read: expensive) ops engineers won’t really be of much value except offloading work that developers shouldn’t have been thinking about in the first place.
Their site was so ridiculously low traffic that I did some napkin math and figured out they were paying around $0.10 per web request.
It's only $7/month to deploy a web application. More if you want some of the paid features and a database instance. It all works out of the box with instant deployment out of the box, it's fantastic.
Then it suddenly goes $50 more per gigabyte of RAM, which makes it massively expensive for any serious workload. It's crazy hot much they can try to charge, makes AWS looks like a bunch of clowns in comparison.
Which is what I think the OP misses discussing in as much detail as they could with AWS -- are there ways AWS is saving some customers developer/ops staff time over the cheaper alternatives? Cause that is often more expensive than the resources themselves. Could just be "because that's what we're familiar with" (and that's not illegitimate as cost savings), but could be actually better dashboard UIs, APIs, integrations, whatever.
[I am currently investigating heroku for our low-traffic tiny-team app, after we had our sysadmin/devops position eliminated. The performance characteristics are surprising me negatively (I can't figure out how anyone gets by with a `standard` instead of `performance` dyno, even for a small low-traffic app; currently writing up my findings for public consumption), but the value of the management they are doing so we don't have to is meeting and exceeding my expectations. (We currently manage our own AWS resources directly, so I know what it is we can't do sustainably anymore with the eliminated position, and the value we're getting from not having to do it).]
As such a lot of devops consultants certainly have all the wrong incentives to recommend it, and these days most of them also lack experience of how to price out alternatives.
E.g. a typical beginners mistake is to think you'd price out a one-to-one match of servers between AWS and an alternative, but one of the benefits of picking other options is that you're able to look at your app and design a setup that fits your needs better. Network latency matters. Ability to add enough RAM to fit your database working set in RAM if at all possible matters. And so on. With AWS this is often possible but often at the cost of scaling out other things too that you don't need.
And most developers won't do it if you don't give them budget responsibility and make them justify the cost and then cut their budget.
Development teams used to AWS tends to spin up instance after instance instead of actually measuring and figuring out why they're running into limits to keep costs down. Giving dev teams control over infra without having someone with extensive operations experience is an absolute disaster if you want to manage costs.
I work for a VC now. When I evaluate the tech teams of people who apply to us, it's perfectly fine if they use AWS, but it's a massive red flag to me if they don't understand the costs and the costs of alternatives. Usually the ones who do know they're paying for speed and convenience, and have thoughts on how to cut costs by moving parts or all of their services off AWS as they scale, or they have a rationale for why their hosting is never going to be a big part of their overall cost base.
The only case where AWS is cost effective is if you get big enough to negotiate really hefty discounts. It's possible - I've heard examples.
But if you're paying AWS list prices, chances are sooner or later you'll come across a competitor that isn't.
I'm having trouble following your theory of why having clients stay on AWS ends up leading to more consultant billable hours, I think because I don't understand what alternatives you are comparing it to. I am pretty sure it is not heroku, as in the earlier part of the thread?
Or are you talking about compared to simpler "vps" hosts like, say, linode? Doesn't that require a lot more ops skillset and time to set up and run compared to aws, or you don't think it does?
When moving them off AWS it'd usually be to managed hosting on monthly contracts.
Heroku turns expensive real fast. You're paying for AWS + their margins on top of AWS.
Managed hosting ranges from API-based provisioning not much different than AWS to ordering server by server.
In practice the amount of devops time spent dealing with the server itself for me at least is generally at most matter of downloading a bootstrap script that will provision CoreOS/Flatcar and tie it into a VPN and record the details. The rest of the job can be done by simple orchestration elsewhere. I have servers I haven't needed to touch in 5 years other than recently to switch from CoreOS to Flatcar (other than that the OS auto-updates, and everything runs in containers). Once you've done that, it's irrelevant what the server is or where it is.
For modern server hardware, if you run your own colo setup, that's a matter of having PXE and tftp set up once in a colo, and you can then use an IPMI connection to do the OS installation and config remotely, so even with colocated servers, I'd typically visit the data center once or twice a year to manage several racks of servers. The occasional dead disk would be swapped by data centre staff. Everything else would typically be handled via IPMI.
E.g. one of my setups involved 1k containers across New Zealand, Germany and several colo facilities in the UK. Hetzner (Germany) was the first managed hosting provider we found that could compete on total cost ownership with leasing servers and putting them in racks in the UK. Had we been located in Germany (cheaper colo facilities than near London), they'd not been able to compete, but putting stuff in a colo facility somewhere we didn't have people nearby would be too much of a hassle and the cost difference was relatively minor.
Small parts of the bootstrap scripts we had were the only thing different between deploying into KVM VMs (New Zealand), managed servers not in the same racks (Hetzner), and colocated bare metal booting via PXE on their own physical networks (UK). Once they were tied into the VPN and the container runtime and firewall was in place, our orchestration scripts (couple of weeks of work, long before Kubernetes etc. was a thing - we were originally deploying openvz containers and so the same tool could deploy to openvz, KVM and docker over the years) would deploy VMs/containers to them, run backups and failover setups, and dynamically tie them into our frontend load balancers.
We did toy with the idea of tieing in AWS instances to that setup too, but over many years of regularly reviewing the cost we could never get AWS cheap enough to justify it. We kept trying because there was a constant stream of people in the business who believed - with no data - that it'd be cheaper, but the closest we got to with experiments with AWS was ca twice the cost.
For the record, in my current job we do use AWS entirely. I could cut the cost of what we're using it for by ~80%-90% by moving it to Hetzner. But the cost is low enough that it's not worth investing the time in doing the move at this point, and it's not likely to grow much (it's used mostly for internal services for a small team). That's the kind of scenario where AWS is great - offloading developer time on setups that are cheap to run even at AWS markups.
I tend to recommend to people that it's fine to start with AWS to deploy fast and let their dev team cobble something together. But they need to keep an eye on the bill, and have some sort of plan for how to manage the costs as their system gets more complex. That means also thinking long and hard before adding complicated dependencies on AWS. E.g. try to hide AWS dependencies behind APIs they can replace.
Although whether it is worth it depends on your view.
( I keep thinking SalesForces is just not a good fit for Heroku, they should have sold it to MS or Google )
When I moved from an ecommerce company with 1000 logins/sec to a SaaS company I could not believe the low traffic :-)
I'm forced to think about stuff like Kubernetes because the devs are popping "micro-services" for almost anything and the answer to trying to keep things manageable is, hopefully, K8s.
Of course they then send funny memes and argue that K8s is overkill. Yet they have no idea how much a devops guy has to do to actually ramp up one of their multi-gigabyte container.
The whole micro-services mentality is proving to be backwards in my environment. It's seen as the answer to everything. "Oh we have a behemoth, let's just not use this part of the application and reimplement it somewhere else." In essence that's great. However, the grunt of the work is making tests to capture old behaviour/semantics; and who are the (indirect) clients of this piece of code. Ignoring a piece of code to death and reimplementing it somewhere else behind a socket is only part of the solution.
The hyperscalars are benefiting greatly of this mindset imo.
I tried my best not to rag on devops as I am always incredibly impressed by these systems and what can be achieved and automated. It absolutely has it's place.
It's just that all the technology that big companies use become trendy and then end up used at small companies too. Small companies don't realise that the tools aren't solving the problems they have, and they don't understand the vendor lock ins and dependencies that they've introduced into their platform for no real gain.
This happens in all kinds of ways, not just devops. Your example of developers using the microservice pattern is exactly that. When I see a developer recommend microservices my mind flashes forward to all of the extra complexity that entails and how we've just turned a small monolith on one server into 6 pieces of software across distributed servers. Great for big companies, too much complexity for small companies.
The points that always seem to come up:
* AWS is a known quantity and it's easier to evaluate our business with it.
* AWS provides "outage damage control" because AWS outages make the news and customers are more understanding. When our ISP has issues it just looks bad on us.
* Our company doesn't look as innovative because we're not cloud. Bleh.
Our app is compute, storage, and data transfer heavy but switching to AWS being a, literally, 10x cost for us is apparently not enough a good enough answer.
However the startup was all built on oracle and sun boxes because "this is what investors want to see" and we'll get .80 on the dollar if we have to liquidate. We had some nimrod spend 8 months trying to get oracle to run on bare drives for our 100 tps (max) website.
They refused to let us use mysql or linux even tho the owner was very familliar with them from the ISP.
I think we spent 3x headcount on the hardware and software, eg we could have run for another 2 years had we been more scrappy. Not that the business idea was all that good.
We also got .3 on the dollar iirc.
Anecdotal, but AWS had more global problems than the triple connected data centers I've used in twenty years.
I suffered through many rough times, data center connection or power problems were not (very seldom) one of those.
Most of the problems came from apps that we didn't build well to scale or that had bugs (most frequent cause of problems and sitedowns).
"Confident"? "Most" failures? Are you merely hopeful that the probability of a bad failure is low, or are you able to test the AWS resiliency techniques you mention and to ensure that they stay working? At what cost?
Multi-DC failover is handled by announcing our public IP block at both locations with different weights. It’s technically active/active because traffic can come in at the secondary DC but we have a internal site-to-site VPN that is used to direct traffic to the primary. If the primary DC goes down the secondary starts handling the traffic instead of passing it along. All the database masters flip to the secondary and things keep humming along.
If we lose the site-to-site then the secondary stops advertising altogether and all traffic is forced to the primary.
So we can lose the site-to-site (which is dedicated) or one of the DCs at any time.
Not sure if you do not know anything about typical ESXi and vSphere setups, but if a server fails, all virtual machines are automatically migrated to a healthy server. And of course, your HPE G10s are compute only, all storage is on the fiber channel connected SAN.
You’re still paying a cloud provider, but in this case it’s VMware
Sure, if it's an established company then using their own hardware (and people to set up and manage it) might make sense; iirc Dropbox is a fairly recent big player that made that move. But otherwise it's a big upfront investment to make, and you can't know if it'll pay itself back.
So sure, AWS can be 2x or more as expensive as renting servers at a company like OVH or building your own datacenter, but it's paid by the minute if need be, not all in one go. If your startup or its VC money runs out in six months at least you can quickly scale down and pull out.
Comparing managed services like databases is maybe more meaningful than just EC2, but also so much more difficult.
Last time I checked, GitHub was also mostly on bare metal and cloud-free.
Or something like the Olympics where 95% of your demand comes in a predictable spike across a few days.
Why would you build exactly the same setup in AWS as for on-prem, unless your objective is to (dishonestly) show that on-prem is cheaper?
Lift-and-shift-to-the-cloud is known to be more expensive, because you aren't taking advantage of the features available to you which would allow you to reduce your costs.
It was far better to invest a little up front, and maintain at $600 my operations than the same for $96K a month, that's why.
I never "lifted and shifted", I built and deployed, with physical servers, a 3-way duplicated environment that flew like a hot rod. At a fraction of cloud's expense.
I personally think there's room for both, and I think hybrids between on-prem and cloud are the ideal for long running apps: you size your on-prem infrastructure to handle 99% of the load, and scale to the cloud for that one-off peak.
That's still pretty complicated due to different types of vendor lock in (or lock out in some cases). Google has invested in k8s to get people some value for moving away from AWS.
This is where your ALBs and ASGs come in. If your app doesn't use local writing and you can shift your caching to a shared cache, the cost savings are good.
Counterpoint: I've seen a great many false economies with people trying to go on-prem and do alternative hosting because they don't think the AWS premium for e.g. GPU instances is worth it. I don't think that has generally worked out well.
It's not an impossible scenario... But imagine the sort of company that trains their own model instead of using a huggingface refinement or an off-the-shelf redistillation. (These can be done reasonably on an average gaming PC, no need for a cluster.) Such a company has expensive human resources. They bothered to get a data scientist and at least a research engineer, if not a full researcher. Were they hired on six-month contract as well? This is a huge expense, so it must be an important differentiator to have built a custom model -- and it's a one-and-done? I don't see it. I think it's going to be an ongoing project, or it shouldn't have been approved in the first place.
I could spend a million EUR a year on AWS without the need for most of the services of AWS?
But it YMMV and 1M EUR/y is not meaningful, perhaps we differ there.
1. IaaS providers are incentivized to create services that lure you with managed solutions that seem like a great deal on paper, while they are actually more expensive to operate than their self-rolled alternative.
2. "DevOps" and "Infra" people charged with making these decisions often follow the industry zeitgeist, which for the moment is whatever AWS, GCP, and Azure are offering. This is in spite of the best interests of the company they work for. The right decision in so many cases is what choice one can defend when something goes wrong, or what you can parlay into the next gig. If costs, performance, and features aren't closely managed, scrutinized, compared, going with a solution like Aurora/Redshift/Dynamo won't be challenged.
3. Nothing is easily comparable because these services aren't just a CPU as you mention, instead they're hardware and software rolled into one. This is intentional, and has little to do with making things easier for you. At best, IaaS providers defend this as "differentiation" but it's closer to obfuscation of true cost. Go ahead and ask your AWS rep if use case x, y, and z will perform well on lambdas and you'll likely get a near canned response that they all fit perfectly, even if y is a job with high load spread uniformly through time. The only way you can make this comparison is by creating the two services, standing them up in production for a week and checking the bill. In other cases such as FPGAs and databases, you'll have a much harder time as not only is the software/hardware offering unique, but the user code will be completely different and require entire rewrites to get what _might_ be a fair comparison.
My last company went the dedicated route, and they were perpetually in need of more servers because nobody was willing to pre-plan capacity. To the point that the database team just couldn’t provision any new databases for a year.
AWS makes some things easy sure, but at my house I have what would cost $15k a year and I spent ~1500 all-in including 10G switch(used) power use is a joke, even with expensive electricity where i am AWS still a ripoff. Same processors as AWS essentially as he was quoting, 10G to my home nodes that need it and bobs your uncle. And I do tensorflow things at home too, i bet that would cost 30k/year, i do it for $1500 BOUGHT and for another $500 for a 3080 or a few in my servers and id be saving tons. Yeah it can go down, but just buy two or a rack, its going to be cheaper than AWS hosted colo. I could scale this myself to a few thousand servers, ive seen it done wrong so many times.....
The truth is more like he mentioned in hist first paragraph, if you have a fan boy at the helm it doesn't matter if he's CTO he knows it all, Ive worked with a lot of CTO's lately they don't have a clue and I feel sorry for them, wasting so many company assets because underlings couldn't possibly know more, i love the x google/facebook guys saying basically scrap everything re-architect its funny and so wasteful but at least they are making more money than me!
One guy said he hates Jenkins, some teams have hundreds of jobs in jenkins that work just fine, he said it should all be redeveloped, some of the stuff has been working fine for 10+ years, not sure he has company interest in mind, he want' everything to be "serverless" its the same damn thing effectively genius.
Hire some sysadmins to run your show, the good ones cost a ton, and computers at their core haven't really changed much in 20+ years nor has the fundamental way the internet works. Good experienced sysadmins can save you a ton and its just as reliable. You just have to trust experience over advertising and real math over funny math.
I’m fully in agreement that owning your own hardware is cheaper. But if you want extreme redundancy as a company now and be mostly certain it works, you move to AWS instead of trying to build a team that will set that all up for you.
Also, when doing tensorflow things on AWS I can just temporarily rent that monster server with 12? GPU’s to train my model in a few hours, instead of waiting for my home server to do the same thing in days (even though that’s cheaper).
This is a common trap for startups. AWS has been around long enough that everyone has a 1st-hand or 2nd-hand horror story about someone with a $5000/month AWS bill for a basic website. This creates a false narrative that AWS is bad and or dangerously expensive. Or worse, that clever devops engineers can simply roll their own solutions with open source tools for half the cost.
The reality is that if a startup truly needs $5K/month of AWS services, they’re getting a bargain by buying it from Amazon instead of building it out themselves. $5K/month won’t even begin to buy you another qualified devops engineer to build and maintain custom infrastructure. The first rule of startup engineering is to use every tool available at your disposal to get to market as fast as possible. Cost reduction can come later, but you can never get back wasted time spent rolling your own solutions when an off the shelf vendor could have solved your problem in days rather than months.
However, the other trap is when inexperienced or overeager engineers see the long list of AWS product offerings and think the goal is to use as many of them as possible. There are some misaligned incentives for engineers who want to gain as much AWS experience as possible on their employer’s dime, regardless of whether or not it’s the right business decision. Worst case, mid-level engineers use as many AWS services as possible so they can pad their resumes, then use the experience to pivot into a higher paying job elsewhere and leave the startup with a mess of half-implemented, unnecessary AWS infrastructure and no one to maintain it. Unfortunately, it happens frequently in the startup world where companies are more likely to hire junior engineers who are eager to tinker with AWS and build their resumes.
The brute force way to avoid this problem is to simply constrain the engineering team to a less powerful platform like OVH or DO. If you don’t need any features of the bigger clouds, this works. However, as soon as you do need the big-cloud features you’re going to waste huge amounts of money building them out yourself. It won’t show up on the monthly provider bills, but rather be hidden in the much higher engineering costs and longer project timelines.
The real solution is to hire experienced engineering leadership who will keep complexity in check, then use AWS or other big cloud providers responsibly.
Basically, AWS is squeezing as many dollars as possible from your pockets (good for them).
I also think the "pay someone to set this up" factor is underestimated. Person time cost eclipsed computer time cost a long time ago. If you've got the market demand to need that kind of infrastructure, you're almost certainly operating at a scale where you can afford to dish out the extra dough because it will reduce downtime and overhead enough to make the ROI worth it.
I'm not an expert in this (I'm just a code monkey who works with scientists, and side hustles as a small time sysadmin), but reading books like "Release It!" and reading blogs (especially DevOps, as I'm looking at getting into that) gives me the impression that if someone could reduce their costs while still getting the value AWS gives, they'd be doing it. I mean, maybe it's a well-kept secret (especially if it gives a market advantage), but if it's so easy, I'd think everyone would be doing it, and AWS would be out of business.
Edit: I concede that startups probably don't need AWS; what I'm thinking of above is big, old, established companies that believe "no one was ever fired for buying AWS." But I do believe that you have to have smart, motivated people in order to grow while staying lean, and just because we hear the survivor stories of startups that do, doesn't mean there aren't dozens that die because inexperienced devs though they could do better than AWS, and were proven wrong.
I can do that a little bit with DO. What are my options with OVH? I see them bragging about having the biggest data center, as if I am supposed to think that's a good idea instead of a terrible one.
OVH have datacentres in multiple countries just like DO.
I should clarify that I am playing devil's advocate here. I've never been pro-Amazon (although I slipped for a bit there, I'm feeling much better again), they just have answers for some questions that I don't.
Or at least, not until multi-cloud is on everyone's radar. The combined coverage map of any two AWS competitors generally looks more competitive, but we collectively have to 'work' on the people who think that would be too hard.
It's about economics not accounting.
Once you get into cost comparisons you've already lost. AWS is purpose built for an accounting narrative.
To illustrate what a dead end accounting is, accounting can't explain things like, "I spend money today to possibly make money more than 1 year in the future." That's basically every startup and accounting doesn't have a story for it.
Of course a free tier looks good accounting-wise. How does one "account" for lock-in though? If you can't figure that out you will not convince bean counters to consider alternatives.
And good luck teaching economics (as opposed to accounting) to bean counters.
Here are a few compelling explanations: the cheapest bid (i.e. free tier) is always the worst one. Some people are so addicted to accounting storytelling - as a way to organize their world, a whole philosophy - they are actually always convinced the cheapest bid is always the best one.
Another: AWS is so overpriced, the $50k you spend on developing against AWS services to use $500k of "free credits" will deliver less value than $50k you spend developing for a single vertically scaled beige box computer with simple ergonomics and running it.
I've worked most of my career in places where technology was king and AWS is certainly technologically ahead of its competitors.
Fast forward a couple decades and I ended up in financial institutions where economics is king and they don't use AWS, not as much as they should, if you listen to the average internet guru.
Not because AWS doesn't do what they say it does, but because economically it's not advantageous, at least until it is.
Which is not at the beginning.
To put it in simple words: they don't ask "are we ready to scale" but instead "are we making money out of this".
That and the 9's. I don't know how good OVH is with outage and availability. Sure the raw compute is cheaper.
There's also storage. He's renting physical drives on OVH meaning he has to take care of replication and back-up himself. The disks in his AWS VMs should be persistent. If his OVH server goes down due to hardware failure, what happens to his disks? I genuinely don't know.
Lastly, OVH doesn't have regions everywhere. Then it's yet another different provider with slightly different product for every geographical region he wants to support.
> I don't know how good OVH is with outage and availability.
I've personally observed failure rates similar to AWS EC2 and EBS. You need to plan for failure, like any cloud provider.
> the raw compute is cheaper
Dramatically so. 20% of the price.
> There's also storage
You can get local NVMe, cheaply, with literally 100x the perf you can get with the most expensive EBS volumes. AWS does not have a storage offering that comes close, not even their instance store NVMe.
> what happens to his disks
VM images can be backed up automatically without downtime (on a schedule, not live.) They offer an EBS equivalent.
> OVH doesn't have regions everywhere
They have enough. us-west, us-east, eu-west, APAC, others. I'll grant that the US regions are operated by a different corporate entity and a different web interface, and that is annoying.
But if you need to persist transactions you still need to replicate and persist state yourself. Or can I mount an OVH virtual drive that I know will survive across VM migrations?
> I'll grant that the US regions are operated by a different corporate entity and a different web interface, and that is annoying.
Why? Do they at least support the same API? Can OVH actually scale on demand?
Yes, Block Storage: https://us.ovhcloud.com/public-cloud/block-storage/
> Do they at least support the same API?
Yes
> Can OVH actually scale on demand?
Not as big as AWS, but yes.
Remember that you're getting 5x the compute resources for the same money. You don't need to autoscale as much. You're not fighting crappy EBS performance. You're not architecting around remote-network RDS databases with milliseconds of RTT. I firmly believe that for most applications you're going to be better off just massively overprovisioning your hardware and keeping the software stack simple.
If you need a predefined small number of VMs and no other functionality, it would be silly to go with AWS. But on the other hand, if you want a set of servers of a given class spawning on demand, with traffic coming in via load balancers, with integrated certificate and DNS management, with programmable lifecycle hooks, with integrated authn/authz, with full audit logs and account management, with configurable private networking to other services, etc. etc. ... You'll pay more than the price difference for someone to implement all of that from scratch.
Compared to this many other points sound like conspiracy theories. Meanwhile people either don't know they can do better, or use AWS because they want more features.