Your comment makes it sound like unless you know your growth pattern, can predict your spikes, don’t plan to pivot, and you know these things perfectly, you will lose. That’s not the case. The reason for that is that you have a quite significant cost difference between AWS and DIY. DIY is cheaper by a significant enough margin that you might be able to buy, say, double the capacity you need and still spend less. So if you made a mistake of up to 2x your growth and your spikes, you are still fine.
Even if you are a small operation, you still have the option to lease hardware. Then your response time to add new servers is usually hours, not days like if you were to go the full own-your-hardware route.
As an exercise, you can try to rent and benchmark a $300/month server from e.g. SoftLayer and then compare that against a $300/month EC2 instance. Chances are, you will be blown away by the performance difference.
But the calculation is harder than that. People are terrible at estimating the ops personnel cost for DIY. Turns out it's hard running with minimal downtime, or engineering hardware + software to be fault-tolerant. It's hard to put a price on dev tools/APIs/doco/the whole eco-system.
Especially for that last reason, I have never been "blown away" by Softlayer, even when their instances were way beefier than anything AWS/GCP offered. YMMV.
It's hard to do that with cloud too. Your instances can go down at any time.
What you're trading off is a small number of people on-site at the data center (and it can in fact be small!) plus some slightly-old-school sysadminning versus new-school cloud-native development. Maybe the latter is easier to hire for (although I doubt that) or your current people find it more fun. But it's not like physical hardware dies all the time, or that you're immune from seeing the effects of physical hardware dying just because you're on the cloud.
You might save on hiring one person to replace disks when they fail and one person to write some PXE boot automation, but you'll need more than two people to run a Kubernetes at the same scale of compute capacity than those two people could have run for you.
https://thehftguy.com/2018/01/15/the-inevitable-demise-of-ib...
If you are willing to pay a lot of money upfront, which you are since you're talking about building your own datacenter, AWS and Google will both give you huge discount and be significantly cheaper than SoftLayer.
Memory is easy to compare. But how does it stack up CPU-wise.
Another question is whether lower tier servers compare favorably or not. How does a $200-500/month server compare?
Lastly, is it possible that SL is just not competitive anymore, but another provider is? I have gotten out of that gig a few years ago so I honestly don’t know, but is it possible that Hetzner or someone similar is now the leader in leased hardware?
SoftLayer is a disaster on lower tier servers. You only go to them for higher tiers.
The market is very competitive and moving fast. Both Google Cloud and AWS have been rolling out high memory hardware recently at a reasonable rate.
Hetzner is amateur tier. It's cheap second hands servers with no ECC. It's not comparable to enterprise infrastructure.
(But then they were bought by a company who's "cloud strategy" is to put the word "cloud" in as many DB2 and mainframe contracts as possible.)
It's noteworthy that SoftLayer is now worse on all aspects.
I wasn't sure what your use "heavy" means here -- is it "a lot of" or "disproportionate"? Years ago there was much less flexibility with IaaS machine shapes, but I was super impressed when Google launched "custom machine types". There's a little slider to scale up/down your RAM/CPU independently, and of course storage is already independent. In fairness, there is some correlation between CPU allocation and storage IOPS, but that's inherent to scheduling a portion of the host machine reasonably.
https://cloud.google.com/compute/docs/instances/creating-ins...
Second, it depends on your expertise. And you don’t need scale. I ran a successfully bit of infrastructure on SL with seven servers that was quite a bit cheaper than an equivalent AWS setup. I was pretty much solely responsible for the infrastructure and still my responsibilities included more than 80% coding and later management. Given my salary at the time, it was quite cost effective.
if you have experience (and/or your team does) with dedicated, that should probably be for a non-trivial in decision making. likewise if you have more aws/cloud experience (or have those skills available on staff), there's benefits to that.
cloud skills don't magically happen - someone with minimal AWS skills can provide you with a setup that is more expensive to run, and possibly more insecure than a locked down dedicated box.
Handling variable load is not free in the cloud, so if you are going to pay it anyway, you can keep your predictable cost down and use the cloud when you have that temporary extra load.
Probably not.
Outgrowing AWS is a great problem to have.
> For a Linux user, you can already build such a system yourself quite trivially by getting an FTP account, mounting it locally with curlftpfs, and then using SVN or CVS on the mounted filesystem. From Windows or Mac, this FTP account could be accessed through built-in software.
Dropbox is not even an option for me because there is no official support or reliable way to run it on OpenBSD.
Dropbox is a consumer product with limited features and use cases. Smugly dismissing the needs and technical capabilities of power users is not going to endear you to anyone here.
I agree that hashkb's comment was flippant and unhelpful, but your reply was not any better and was the one that went off-topic.
1. a nice web interface;
2. synchronisation;
3. an easy-to-understand pricing model;
4. tightly integrated desktop support (at least for OS X).
S3 and EC2 are by design commodity, low level artifacts. Like any other commodity, you should always be shopping around.
The big issue with Amazon has always been their “roach motel” philosophy where data goes in but doesn’t come out. (ie they hose you with network costs)
If you have enough scale and enough dollars, it is often cheaper to build bespoke infrastructure vs renting somebody else’s at margin.
This is why many companies with a lot of money still rent their offices – even if they intend to stay in the (often customized) buildings so they don't need flexibility. It just would not make sense to buy and to bind the capital in a building, it can be invested in their core business with higher returns.
An exception might be made in cases where a company has suddenly so much money that investing it all in its core business might diminish the capital/return rate, eg. through the US tax break, where a company shifts the money stored oversee back to the US. This is why for Google it might make sense to buy property, even though its not their business: They just have to much money.
[1] S3 at 0.023/GB/month * 500 = $11.50
A recent place I worked at didn't understand this. They were going to the cloud as a strategic move because they didn't want to run a data center any more. Their computing needs were stable and predictable - no "Black Friday" sales that needed triple the servers for a short part of the year. They were going to end up paying far more for the total computing cost than if they had just kept buying blade servers and VMWare licenses.
I'd argue that companies where the hosting costs are the primary, or even a significant cost, are a small minority.
Granted, I think this is more of a case of the prisoners dilemma [0] between lower management and upper management (lower management doesn't want to work on it because it doesn't produce features that make him look good but he also does not want to propose the additional developer for it because then upper management will just tell lower management to do it without the additional developer).
Fire team, comeback a few years with a new cheaper team and in-house pricing.
Job done
For most startups I would actually advise to start with Heroku, which is even more expensive than AWS (it is built on top of AWS). But you save on devops and building the CI/CD pipeline. For a small team it can make a big difference in terms of productivity.
For worker workloads like CI, renting dedicated hardware (like Hetzner) is usually cheaper and produces more reliable results. spot instances also work but have less reliability dues to machines cycling. The main factor for keeping everything under AWS would be egress bandwidth pricing or if the workload spikes are bigger than 2x.
For number 2 especially, there have been some cool projects for efficiency gains when different parts of the organization experience different traffic spikes. Like Netflix, where transcoding spikes precede viewing spikes, and they pulled off a load balancing coup to reduce peak instances.
I think the right thing for a tech company to do is to run their own data center in at least one town where they operate, and use cloud services for geographic distribution and load shedding.
The temptation to reduce your truck numbers below long term sustainable rates is too great, and so is lock-in. The best hunt I think you can do for the long term health of your companies these days is to hold cloud infrastructure at arm’s length. Participate, but strongly encourage choosing technologies that could be run in your own server rooms of another cloud provider. Like Kafka, Kube, and readily available database solutions. Make them avoid the siren song of the proprietary solutions. We have already forgotten the vendor lock-in lessons of the 90’s.
Much easier to go hybrid on Azure, at least until AWS and vSphere integration is ready for prime time
A startup I talked to not too long ago has a revenue of around 1M a month - pretty great. Their Amazon bill is around 1M a month - not so good. Their entire architecture is one AWS service strung into another - no optionality at all.
You're talking about vendor lock-in. Which is a totally valid point and something to be aware of, but basically orthogonal.
> (2) you have traffic spikes, so you to over-provision. There is then a tradeoff doing it yourself: do you pay for infrastructure you barely ever use, or do you have reliability problems at peak traffic?
> (3) your business plans shift or pivot
Anyone at any real scale has a multi-datacenter setup and AWS is effectively just a very elastic datacenter you can tap into. You could just as easily tap into Google Cloud or Azure. You do not need to operate 90% of your business in AWS to use AWS.
> The corallary of this is, for a well-established business with predictable needs, going in-house will probably be cheaper. But for a growing or changing or inherently unpredictable business, the flexibility AWS sells makes more sense!
Its still cheaper to go in-house with a multi-DC setup in anything but a single DC setup in AWS with very few nodes.
Architecture for a decent sized business should be setup with 3 DCs in mind /anyway/. You have two primaries, you scale <insert cloud provider> as needed and only leave some DB servers there most of the time.
Another commenter on this story mentioned $100k redshift workloads that were optimizable to $2k.
Yes. AWS shines when you have a small team with no competent DevOps people and a desire to build a single datacenter setup. However, at that scale...we are talking a small business / startup with less than 50 employees. If you are still doing that when you've grown past that point, you've got architectural problems.
Once you scale past that point, you still need a competent sysadmin to build a reproducible process...at which point even leasing hardware from any provider works. You cannot build a multi-DC setup that integrates redundant providers without a competent sysadmin.
Even if you stick with how AWS has you do things it will eat you alive if you do not optimize your workloads. I'm not sure how you would do that effectively without an experienced sysadmin acting as an analyst.
Also, in my opinion the know-how and full control of the complete stack is paramount to maintain a quality service.
In addition, you can do both. Tying yourself to one provider is an unnecessary risk and tying yourself to the cloud is not ideal.
I assume that answer is a no.
For me the logic is more like: get cheapers machines (be it in-house or with cheaper alternatives), that run kubernetes for example, and monitor them with Prometheus.
If you run out of capacity, defined by whatever metric you fancy from Prometheus, start EC2 machines for the burst.
Every month, re-evaluate your base needs.
Now DO, vultr, and other options exist to fill the gaps more.
Aws was never the solution to content distribution or infrastructure replacement. Just a company smart enough to notice big gaps and not afraid to fill them to get the ball rolling, maintain and progress, then move on.
(1) your business scales at a different rate than you planned -- either faster or slower are problems!
(2) you have traffic spikes, so you to over-provision. There is then a tradeoff doing it yourself: do you pay for infrastructure you barely ever use, or do you have reliability problems at peak traffic?
(3) your business plans shift or pivot
A big chunk of the Amazon price should be considered as providing flexibility in the future. It isn't fair to compare prices backwards-looking: where you know what were your actual needs and can compare what it would have cost to meet those by AWS vs in house.
The valid comparison is forward looking: what will it cost to meet needs over an uncertain variety of scenarios by AWS compared to in-house.
The corallary of this is, for a well-established business with predictable needs, going in-house will probably be cheaper. But for a growing or changing or inherently unpredictable business, the flexibility AWS sells makes more sense!