Respectfully, I'm going to disagree. I consult full time on AWS cost optimization / reduction / understanding.
If you blindly run things on AWS without an understanding of the costing model, that'll work for a time. As you scale, you start to realize "oh my god it runs on money."
There are myriad ways around that, but a blanket "never use AWS" isn't going to address it constructively.
Curious, have you ever consulted a client not to use AWS? As in "your cost optimization is going to be not using AWS and using $x instead"?
That said, clients who come to me already have a significant investment with AWS. Getting rid of that is a high bar in terms of engineering effort.
That said, I've no financial incentive to recommend one provider over another. My clients pay me, the vendors do not.
We've pushed clients to competitors' products a few times, I was young and thought that was silly. However those clients spread the word about us and we didn't even have much of a marketing department, it was all word of mouth. Gaining their trust was gold and a really good long term investment.
"If you have content where you can't put a cache in front of it, and it drives significant bandwidth use, then AWS is not a great option"
Data driven applications on AWS are already expensive because of the high traffic costs, but storing/serving media on S3 single handedly lets your bill explode. For everyone who feels like the statements above are exaggerations, just check out their actual prices (https://aws.amazon.com/ec2/pricing/on-demand/) in comparison with a regular CDN (https://www.stackpath.com/pricing/).
But that's not what the commenter was saying, of course.
More like (paraphrasing) "before you go full-hog on AWS, do some simple math first." Which an amazingly high percentage of people neglect to do, these days.
I had a coworker who was turned off of AWS by clicking the "Dedicated IP" checkbox when deploying a CloudFlare distribution with SSL.
One month and $600[0] later he swears up and down that AWS is the devil.
You can spend an amazing amount of money on AWS by accident and/or without need. Hourly pricing is a huge footgun. Best to run everything through the "Simple" Monthly Calculator: https://calculator.s3.amazonaws.com/index.html
† About the monthly you-know-what bill now well into 5 figures, and climbing.
Your time is valuable too. It could be spent developing something novel instead of messing around with server configuration.
At the single instance very small scale, it simply doesn't matter all that much. "Our business failed because our infrastructure bill was $400 instead of $150 every month" isn't that common of a story.
Base load can be effectively migrated to AWS, depending upon how it's built / structured. It's hard to lose money on infrastructure by converting worker servers to spot instances or lambda functions, as a for-instance-- but it's VERY easy to lose months of engineering effort in doing so.
Remember as well: done right you're not just buying services from the cloud, but flexibility as well. Resizing on the fly, bursting, duplicating your infrastructure for a test-- these things add value.
A lot of teams rent space from places like OVH or Hetzner because the raw power is so much cheaper, and they already have the talent and tools to scale and manage a computing fleet (people like to think it's harder than it actually is).
Those marginal gains could potentially pay for a dozen engineers...
It depends. I have a toy service that exists entirely as an autoscaling group. That's pretty dynamic!
Counterpoint: Mainframes.
Step 1 is always "figure out what you've got, and why it is this way."
As you grow, start adding project tags to things. Automatically tag resources with the IAM user who creates them. Have an "untagged resources report" that goes out on a schedule.
Build with an eye towards "when someone freaks out in a year about the bill, what questions will they have, and how can I best position Future Me to answer them?"
Generally most of the time the best approach is to create solutions that can scale out independent from the HW / infrastructure they run on. With frameworks like K8s / Docker / e.a. this is now relatively easy.
With that for example you can start of with a few small cloud machines / docker containers. When you get continuous load - migrate some of them to 1-2 dedicated machines. Same with peak load - just spin up more images.
All that of course depends on what kind of application you are providing. If you are running a data intensive apps you might start directly with a dedicated machine.
Easy in, expensive out. One cost with cloud services many forget is the cost to get out again. Have a look at the AWS / Azure price-lists.
I think the real issue is that people assume all workloads are public cloud workloads. The bigger and less dynamic your workload the less that is true.
Seeing the opposite trend.
Because your own machines are so expensive and annoying to manage, you only get 500GB memory 40 cores servers. You don't optimize and you run stuff randomly on whatever seem available.
Whereas in the cloud, you put VMs per role, with appropriate sizing. And when someone asks for 5 machines with 16 cores, you can be like, WTF are you running with all that power???
Came here to share my story, which is very close to what you describe.
tl;dr: Moved from AWS to colo in desperate attempt to make balance sheet more attractive. It worked.
Before I took over the engineering bits at this plateau/failing startup, they were all in on AWS. Monthly costs were running at around £3k. Which is not much, but since the revenue was tiny, the costs of hosting was "considerable". After a monumental effort on my part to try to make things more efficient and streamlined, I eventually run out of motivation (but that's another story) so I expressed my wish to leave to the founder and he finally agreed to my years old suggestion of going lean-and-mean and in one last attempt to make the company more attractive to one last investment round (E by then). Make it or break it, as the say goes.
On that same week I came across a HPE "Buy One Get One Free" offer (similar to this one https://www.serversdirect.co.uk/pdf/BOGOF-Gen9-Servers.pdf). Then found guy who was re-selling colo space at a under-used datacentre in London. Since I (sysadmin/DevOps/InfoSec/Backend engineer Jack-of-all-trades master of none) was the person who would end up managing the new metal, I picked a location within an hours drive from home.
And so it happened.
2x 1U HPE proliant servers with decent CPU and 32G of RAM. And 6x Samsung 850 SSDs off Amazon. Total bill for the hardware (BOGOF refund included): £2600 ish.
Post migration, monthly hosting costs: £50 for the colo; £50-ish AWS S3+CloudFront (10M assets would increase the SSD storage costs too much); £40 in taxi fares when I visit the DC once a month for kernel updates.
And just like that, we broke even the very next month after the last "big" AWS bill. Our product wasn't all that exciting, but 2 months later we found a buyer who was happy to snatch this "non loss making" operation.
3 months after the acquisition, they had already migrated the whole thing back to AWS and the HPE proliant servers were gathering dust in their office.
If RDS saves you 30 minutes a week I'd be surprised.
The 50%+ profit margins have to be coming from somewhere. AWS is not made of magic, it's made from largely the same PC parts you buy on newegg
So, if you provision an x1.32xlarge with 128 vCPU and 1.92TB of RAM, you get a single, dedicated host with that much CPU and memory. Nobody else gets it - it's dedicated to you 100%.
The profit AWS is making is purely due to datacenter efficiency and being able to automate their operations at scale.
That's what they want you to think, but a lot of it comes in the form of ripping you off for bandwidth to actually get data out of EC2.
What is a vCPU?
As soon as 32GB DIMMs drop enough in price, I would expect them to do exactly that. You can configure a Dell R930 with e7 Haswell CPUs online, if you want to double-check it. Not sure if the R930 can go to 8 CPUs (which the e7 Haswells CPUs support).
Charge more than what it costs you. That's how to make money.
if anything its a diversion from fixing what is broke or even admitting to it within some organizations
You put bald tires that are over-inflated on a Porsche and you're gonna have a shitty experience. Sure, bare-metal is gonna have performance benefits over AWS and at times might be cheaper -- but that isn't the only thing an eng organization is looking for.
tl;dr – you can build systems on AWS that perform well and are reasonably priced.
This is off topic, but bald tires have better traction than grooved ones in dry conditions. (But the slightest bit of rain and they will not hold.)
Also tire inflation does not have a strong correlation with traction (there is some, but it's not large), so it's not so obvious that over-inflated means a bad experience.
But yeah, AWS generally is expensive, which is part of why it's making so much money.
It definitely makes business sense to try to avoid locking yourself in to AWS but at the same time it's a good place to ramp up or test your business ideas without needing to invest a lot of money in servers, real estate etc.
[You can probably substitute any other cloud provider for AWS here]
We made do for quite a while on a single colocated database with 32 GB of RAM and a few fast SSDs (for the time, which was about 6 years ago), and a few multi-TB archival drives, but disk space is a concern since our compressed innodb tables are well over 100GB now. I set up a Google Cloud SQL instance (the largest possible at the time I believe, 104GB RAM, and enough disks to hit peak IOPS) a few months back in preparation of migrating the data store to it, and found it wholly inadequate for our needs prior to even getting it going.
First, the second generation instances don't support being a replication slave to a non Cloud SQL instance, making migration a pain. Second, just running through the replication logs on a client and piping it to a connection to the cloud SQL instance showed that the cloud SQL instance was just barely able to keep up with the insert and update traffic from the replication log, even while handling no other queries. Catching up the the master would take over a week, even though the backup was from less than 48 hours prior.
We signed up for an actual hardware server from Rackspace at about 2/3 the cost, but with 128 GB of RAM (which I can request expanded), and nice hardware RAID 10 of fast SSD drives, and tons of logical CPUs. It's appropriately faster compared to the original database server, to the degree you would expect from from stats quoted. It fits our needs, because really I just don't want to manage the hardware anymore (this the decision not to deploy more hardware in the local datacenter we are colocated at), and that's covered.
On demand spin up/down of resources and per-minute billing is nice, and I've used it to good effect for non-database resources before, so I understand the versatility of it and how it can really enable some new types of usage and am a fan of that aspect, but I just wasn't prepared for how substandard the DB offerings are if you actually have any sort of load. At this point, the only good thing I have to say about the higher end database offerings is that they were quick to destroy so I didn't get billed anymore. :/
Disclosure: I work on Google Cloud and Cloud SQL
Use the right tool for the job.
Once the company grows big enough, yes, then it might make sense to go off AWS (or not - see: Netflix).
That's something that should be applied almost universally. :)
I'm thinking of services, say the automated robo-built sadwiches, or the Easy 401 service, or the luxury shoes startup (all real YC-accepted companies), where the core business proposition isn't necessarily dependent on having a huge infrastructure.
Lab 4 is a beast with more lives than <insert metaphor>. The moment you think you've finally written your distributed system correctly, the unit tests will prove your service fails during XYZ network partition topologies. It's very worthwhile to be forced to think about issues like that and to design distributed systems for correctness.
But to address your question more directly, it's generally just a matter of scaling your service as much as possible on a single server. The server has finite resources (CPU, Memory, network, disk) so you either know how your system consumes those resources (because e.g. you wrote the service, and you know it uses O(n) memory w.r.t. the workload) or you graph your usage over time and try to predict when you'll exhaust your available resources. At that point you can usually think of some straightforward optimizations, which keeps everything on a single server. But eventually you might run out of resources with no obvious path to optimize it, so what then?
It depends entirely on your service, but typically you can just use off-the-shelf software to scale to multiple servers. For example you could set up three servers, each running Redis, and have Redis keep a list of "work to be done." Then your central process just farms out the workload to each of the three servers round-robin style, for example.
But at that point your service becomes a lot more brittle, e.g. you'll need to set up a failover solution so that your cluster can survive partitions and outages. (Redis uses Sentinel for that.) So it's worth keeping everything on a single server for as long as possible, if you can work out the optimizations to do so, since it's so much simpler with only one server. (HN is still running on a single core and serving 350k daily uniques, I believe, which shows just how effective it can be to keep your architecture as simple as possible.)
My question is learning about design ideas like having Redis keeping a list of "work to be done" and such. Using modern tools to combat 'modern' problems. Is it just something you figure out after knowing the fundamentals (i.e. when you can identify a problem you know what the solution needs to be)?
If it sounds mysterious, think of it this way: Imagine you were thrown into a room with a computer, internet, and endless food, and the only way out of the room was to solve this problem. I bet you'd figure it out within a week or two, or a couple months max. (If only to go have a shower.)
One thing to watch out for: Solving modern problems can be pretty unsatisfying. Before you experience what life is like at a modern company, you tend to think of them as functional, orderly, and planned. Real companies are almost universally dysfunctional, disorderly, and haphazard. It's very rare that a plan is conceived, followed, and deployed without growing some warts and throwing in some hacks to get it done.
So I think you should enjoy this time, where you're free to think of thought-problems like "What's the most correct and extensible way to solve this problem?" instead of being forced to solve them on a time crunch.
Well, actually it doesn't.
RAM and Xeons are expensive.
- 1U single socket Supermicro Xeon E5v4-2630 10core+1GTX1070 GPU = 1500EUR
- operation costs (power+cooling) over 3 years = 1500EUR
- cost of 2 hardware admins 160k/year
- renting local rack space 15EUR/U
This equals about 145 servers for 3 years for us for 1 Million. The setup is highly optimized for _one_ set of data crunching tasks, not DB. We do not need much ram, or network but require a specific CPU/GPU ratio.
This is by far the most expensive item in your estimate, and I'm sure that you don't suddenly need to hire another pair of admins when you buy a single Xeon blade.
Once you factor this fact in your estimates, you'll realise that the unit cost of each server is, unlike your estimate, negligible.
That's about the upfront costs of servers ONLY.
Ignoring cooling, all personal costs, electricity, networking gear, network storage and future maintenance.
Figure a 10% quantity discount and you are looking at at least 200 of those for about $1 million per year.
For 500€ a month, it's possible to buy your own 40 core blade with 256gb of RAM. In fact, it's possible to buy a pair of those servers.
The way you wrote your comment about using a dedicated server implies that there are no other viable cloud providers besides AWS. That is the problem, people do not know or do not acknowledge the existence of a number of other cloud/VPS providers that, in most cases, would work perfectly fine for their startup, and be a massively better value.
Such as Digital Ocean, Linode, or many small relatively less well-known providers that are perfectly fine like UltraVPS etc.
Sure, some few people do actually need DynamoDB or some other ultra-scaling AWS service, but people who just need VPSs can get a much better deal elsewhere.
While it was obviously a good trade off for this company, it seems silly to then extrapolate that all companies would benefit by getting rid of redundancy and scalability.
If they can indeed fit their footprint in one single server, then it should be fisible to just host the server at someone's home with enough upload speed, no?
He contacted the developer who said that they were shutting it down because the server costs were higher than the money they were making.
They were spending 5k a month on AWS crap and claimed it was impossible to get any lower.
He helped them consolidate everything onto a single rented dedicated server costing 400 a month. Now the service is profitable, and will stay up.
It runs way faster on the single server. It also has required less maintenance after the move too.
This kind of shit is everywhere. At this point simply not using AWS is a competitive advantage.