I hope so

99.9% of startups never have a need for more scale/infrastructure than a single VPS. Maybe a dedicated server.

Cloud infrastructure is great when organizations actually need it. But I think most startups would save money and time by keeping it simple until they really need more.

Depends on the cloud.

Looking at Supabase, I think it would be a good choice for most startups[0] and it will scale as the business does, but it removes most of the overhead of getting started. It has Auth, A database, Lambda Support, CDN and File Storage. Baked in are REST APIs and GraphQL APIs out of the box.

Its reasonably well priced and there's little friction on setting up and far, far less to worry about hosting wise. No need to setup a box and maintain updates etc, and it can scale if the business really does scale.

I don't think all cloud propositions are the same. AWS, GCP (maybe sans Firebase?) and Azure definitely have their place but I don't think they're great for the average startup.

[0]: Even medium sized and enterprise businesses could leverage it. Its all powered by Postgres at the end of the day

Yeah but we are talking about startups. Brand new ones. They have no customers. Let alone enough customers to necessitate cloud scaling.

Gotta build the prototype on something eventually. It’s a solid OOTB tech stack. Lets you focus on building the business with peace of mind. Not fiddling with servers etc.

For the past several years, “startup” meant at least 5 years in operation with over $1 billion valuation.

I've used Supabase for some production level projects, and it's seriously a very good starting point which you can scale up to mid-tier levels. Brand new ones with no customers can easily leverage their free tier.

This. Cloud is great when a startup hits an unexpected growth spurt and can scale infra without getting distracted from the core mission. It is not cheap, but it may be worth it.

But this is a fairly narrow use case and until hitting limitations of a couple of VPSes, moving to the cloud does not make sense for most startups. My 2c.

> But I think most startups would save money and time by keeping it simple until they really need more.

Having done this several times I can definitively say given my experience it's best to start in the cloud with a cloud native architecture. I can run a fully containerized application in ECS or EKS for a few hundred bucks a month. Why would I incur all the costs and limitations of a VPS or even worse a server I have to look after myself? How much can I really save? $50 a month maybe less?

A long time ago when I was working a major telecomm provider I had the privilege of working with a great software mentor. He instilled in me the lesson of knowing approximately where you are going to land and not do anything now that would jeopardize that landing.

If I was a CTO at a greenfield startup. I would insist we deploy on a PaaS that supports serverless, object storage, and container orchestration. We would recognize and enforce well accepted patterns that will not impair our ability to scale later. There is plenty I can do to keep the costs low(most importantly turning things off) then when I need to scale up its as easy as turning a knob.

It seems like such a waste of time and complexity for a knob that only a tiny, tiny fraction of startups will ever have to turn.

I think of cloud platforms as less complex than managing your own systems. It's trivial to go from zero to functional product on GCP or AWS, and it's honestly not very expensive to get started either. GCP and AWS very clearly subsidize costs for early-stage startups in an effort to lock them in. A tiny startup might have a cloud hosting cost of 3 figures a year until they really take off.

And when your product does take off, scaling is often trivial.

Yes, scaling is trivial on AWS if you rewrite from scratch 3 times and rearchitect 4 times, but at least you don’t have to enter your credit card number twice.

His assumption is that you're familiar with cloud design patterns. So if you know what you're doing, costs are manageable and scaling is a breeze, that's what the cloud was literally designed for.

I just don't see this. Building what I am describing can be done in days sometimes hours. Where exactly is the time and complexity? I would be willing to bet given a true greenfield(e.g. I can't use an ansible script that stole from my last job) I can build an autoscaling flask app with CI/CD and local environment on ECS twice as fast as accomplishing a lesser facsimile with a bare metal server or a VPS. If I built on a VPS I would still have concerns like backups, High Availability, DNS, Networking, etc. etc. to deal with.

I consider the latter to be far more complex.

> He instilled in me the lesson of knowing approximately where you are going to land and not do anything now that would jeopardize that landing.

Thanks for sharing that.

> object storage

Is this like S3 or more like DynamoDB, or something else?

S3

I'm often perplexed when I hear people complain about cloud costs. I can't tell if people just have really different workloads than I've experienced or if they just haven't considered cost when they pick their compute, databases, etc. If you use object storage, some serverless database, and some serverless compute a small company (by which I'm thinking "normally very low volume with occasional, very important, often large bursts") can keep their cloud spend to tens of dollars per month and save considerably by not having to manage all of the stuff that goes along with running VMs, database instances, or physical hosts.

Like, I assume I'm the one missing something because people swear up and down that it's the other way around, but I haven't found the happy path for managing hosts (virtual or physical) that delivers on the promise of simpler-than-cloud. There's a disconnect somewhere because I'm sure there are a lot of people who can't find the happy cloud path that delivers on the simpler-than-diy approach.

The stuff you’re saving time on, managing hosts, just isn’t that much compared to dealing with aws/terraform/etc. I’m at a few orders of magnitude more spend than that, and in absolute dollars is very much noticeable how much more expensive aws is. But as a percentage of costs it’s so little next to wages, as is your example, that cost is really the wrong thing to get hung up on. Most engineers I’ve worked with don’t get that, they see a value they can objectively minimize, without realizing that even dropping it to zero his little business effect.

> The stuff you’re saving time on, managing hosts, just isn’t that much compared to dealing with aws/terraform/etc.

This is where I feel like I'm doing stuff wrong. If we're comparing Terraform, it means we probably care about reproducibility (rather than an "AWS console vs SSH into pet hosts" scenario), so the like-for-like comparison involves bringing in something like Ansible. On top of that, you need to pick, install, and configure logging exfiltrators, monitoring agents, process managers, etc and you need to operate systems that let you explore those logs and metrics. You also need to configure SSH access and manage keys. You may also need a custom base image, so maybe you're doing packer stuff as well? On top of that, you need to run some database which means managing backups and running replicas with failover (or maybe we/re a small business and we don't care that much about reliability?). And again, we care about reproducibility, so we need to encode all of this stuff in Ansible playbooks or similar. You probably also need something like security groups to restrict which things are allowed to talk to which other things, and encoding this in Ansible or similar is maybe impossible if you don't have software-defined-networks.

It seems like a lot to get to parity with what someone could throw together with API Gateway, Lambda, S3/DynamoDB in a couple reasonably-sized Terraform files in a few hours for a pretty marginal cloud spend (most small businesses would probably stay pretty close to the free tier--these services are extremely inexpensive).

Yea, I mostly agree. I feel like a lot of the negativity comes from hobbyists, where an extra hundred dollars does matter.

I think one place that I disagree is while you could do that in a few hours, your average developer couldn’t. You’re now talking about everyone to learn lambda and terraform and whatnot, whereas with a “standard” web server, that people are familiar with, a lot of that is more easily centralized. just throw some annotations and routes are done, vs the arcane api gateway config. The tools and frameworks for lambda just didn’t seem to be there yet.

Fwiw I’m all in on aws, cost was one of the easiest arguments to deflect. Ultimately we needed to show developer velocity increases as that’s the cost that mattered. And security isn’t compromised, which the bigger the company the more roadblocks I’ve seen to just give devs terraform.

Agreed that there is a learning curve, I guess I was assuming some competence with both stacks—not starting from scratch (although I think if you were starting from scratch it would still be easier to learn AWS than self-managed hosts).

Yeah, big companies don’t like giving devs raw Terraform. My company has sandbox environments where devs can do iteration with permissive Terraform access. That works pretty well for stuff like this.

Also, I use AWS for hobbyist stuff and you can easily use this serverless stack for <$5/month.

I would like to see some real numbers on this. I spend between $5 and $6 million a year on infrastructure. I have a physical datacenter in a Colo and two clouds AWS and Azure. I have detailed KPIs on my spend. One of the KPIs is total cost of ownership of an instance or server. My physical instances cost 3 times as much as the exact same compute in the cloud.

This is because I have to factor all the costs. This includes electricity, maintenance, incident response, networking, renting the cage, vendored software for backups, threat detection, fire suppression, equipment upgrades, licensing, alerting, and it goes on and on and on...

I'd challenge you to break down the full cost of owning a server as you see it. I bet you will miss 75% of the actual costs involved. I promise short of seizing a colo like its Nakatomi Plaza and running it at gunpoint you will never in a million years come close to the total ownership cost of cloud instance. You can't compete with the economies of scale and the caliber of the engineering.

37signals disagrees. They’re moving off the cloud and have published several blog entries about the process and the approx ~1mm/yr (fully loaded) they expect to save.

I don’t have a horse in this race, but it’s interesting that your experiences are so different. Would love to hear your take on their numbers.

The most recent article, with lots of hard numbers, is here:

https://dev.37signals.com/our-cloud-spend-in-2022/

There was a recent article that made the HN in front page that broke down the savings they expected, but I’m on mobile and can’t find it now. Something about “two datacenter racks.”

Where are the numbers and how do they calculate them? This article lists their cloud spend in detail then "waves hands" at the Datacenter costs by say "it will be far far less". Ok why not break it down?

I’ve done the modeling down to pulling actual quotes in my area and with a decent footprint (like couple hundred xxlarge instances) you break even after a year (including colo, remote hands and networking). Cost is decisively not the reason most orgs choose public cloud

Here's their article. "We stand to save $7m over five years from cloud exit."

https://world.hey.com/dhh/we-stand-to-save-7m-over-five-year...

>But this isn't just about cost. It's also about what kind of internet we want to operate in the future. It strikes me as downright tragic that this decentralized wonder of the world is now largely operating on computers owned by a handful of mega corporations.

Yep read that guys blog history and the agenda just pops right out. It's not just about cost for Basecamp its ideological. I can't help but imagine this bias leaks into the financial and operational calculations.

We had a similar situation. We had a team did not want to move to the cloud, the business forced them, so they built the system in a way that fought the cloud. Then the self fulfilling prophecy kicked into high gear. "See we told you it was a bad idea, look at all the problems we have!". The problems were created through half baked attempts to be "agnostic" to the cloud. Once we removed those elements we were able to reduce the cost of the system by over 90%. It was far cheaper than when it was running in the Colo. These folks had no interest in optimizing for cloud native execution they were already planning their move back into the Colo.

These complaints mostly seem to come from folks who are spinning up lots of VMs and architecting solutions the same way they would build in a data center. If you're running VMs 24/7 of course it's going to be expensive.

This feels right to me. If you build with lambda, api gateway, and s3/dynamo a small business can stay pretty close to the free tier. Obviously this doesn’t work for every single application, but I’d bet there are a lot of VM applications that it would suit just fine.

You can get a VPS for like $5/month. So you can save near enough hundreds of dollars a month.

I don’t think cloud cost is anywhere near as much as the cost to build a semi-decent HA/DR system and the devops process around it. Every minute the site is down it’s real monetary loss for most startups. It also takes time and energy away from building features. I have done both bare metal and cloud in two different startups. (This assumes you can’t afford to lose data or be offline for extended periods of time).

Here’s what you’ll need to setup and run a single database :-

- Installation script and infra provisioning. You need a repeatable way to do this with as little manual intervention as possible and I can swear it never just works.

- Backup and recovery scripts and associated monitoring

- Replication setup, monitoring and debugging. It will break often.

- Monitoring for infra - CPU, memory, network. There’ll be unforeseen spikes and random slow downs and a whole bunch of slow queries you’ll debug on a regular basis.

- Security - users, permissions, auditing. Both machine and database. Regular password rotation and firewall rules. Ability to off-board users.

- If you survive all this, being able to scale up. And I mean just vertical scale up will take you far but you cross that limit, you now suddenly have to deal with sharding or partitioning.

So when I joined a new startup and we were building it all from scratch it was a very easy decision to go with the cloud. We spent $3500 a month. Not a lot but nothing compared to the salaries and marketing spend.

The hundreds of dollars a month was quoting the comment I was replying to.

The underlying point I made that is being ignored is that 99.9% of start ups never need any of the stuff you are talking about. And hundreds of dollars a month is a large amount of money for them.

So, say you're saving $2500 a year? That seems pretty cheap for an option to scale up without fundamentally changing your operational architecture.

The bigger question mark than the money, I think, is whether this scale up approach actually works, or whether you end up fundamentally changing everything anyway. In my experience it's a mixed bag. There are some advantages, but it also isn't a panacea, and certainly isn't as easy as turning that dynos knob on heroku was. But at the end of the day, I land on it being worth it for a business (though not for a hobby). But it isn't a no-brainer.

A couple hours of engineer time per month. Maybe 5 if we're being generous.

And even more importantly, dozens of hours per month.

I have to disagree. There is an initial phase where a startup can simply use docker to provide their service. Having prepared that initial layer they can easily "scale up" and move parts to cloud services as they require.

I disagree, mostly because if your product takes off, that single day or few weeks where the adoption rate sky rockets may be your defining moment for your startup, and scaling up on the cloud is trivial in comparison. Remember, when you're that small, the expenses of the cloud vs dedicated hardware is tiny compared to your employee expenses anyways.

Twitter used to go down regularly in the early days. Reddit did too, even years after it became popular. Facebook I can't be certain now, but I think it would break in the early days.

And even right now today, ChatGPT regularly displays "too busy" messages.

i.e. this is absolute tosh.

And if you know what you're doing you can take something down overnight and put it on a dedicated server.

"ChatGPT regularly displays "too busy" messages."

ChatGPT scaled from 0 users to 100 million in 2 months. Probably millions online at any given second.

Its too busy, because it literally exhausted all the GPUs Azure has as its disposal.

Without the Cloud, scaling ChatGPT would have been impossible. It would have been nowhere near its current success. Remember that GPT3 was released to the public in Mid 2022, had no public reaction, because it was put behind a paywall. ChatGPT had to be free, and able to cope with the load, to be successful.

False. It is not coping with the load. It is successful.

Therefore, it did not have to be able to cope with the load to be successful.

All you did is illustrate my point.

That's because ChatGPT was able to sustain service past the point of mass adoption. You're not going to scale to millions of users in a short timespan with dedicated hardware.

cloud infrastructure was never the limiting factor.

I think it was in a way.

Having an old school style IT structure was a pain. Maintaining your severs, networking gear if needed, OS updates ... on and on.

For very small teams (maybe not the well funded start ups) that could eat a lot of time and trouble.

You used to be able to scale up to millions of users in less time than you could memorize all the AWS acronyms, and with lower hardware and bandwidth costs to boot.

It’s true that it would take a couple weeks to order all the hardware and a couple more to install and configure it, but that’s about the same amount of time you spend writing all the terraform and Kubernetes scripts and configuring cloudformation and VPCs and IAMs.

Yeah, but then you wouldn't have high availability to the same level, for example. I mean, you could, but then tack on another 100 non-AWS acronyms to your on-premise solution.

Looking back 20-30 years, it absolutely was a primary limiting factor. Massive capital outlays were required to serve the kind of traffic you can get with the free tier of modern cloud infra providers.

That depends on what you are doing. Serving static websites needs a great pipe, but doesn't need much CPU.

Yup, just gotta lay good pipe.

Absolutely not.

not really, i built websites w hundreds of thousands/millions of users on a VPS that cost me like $30/mo.

Aren't startup made to be bought by the tech giants? Until there is a shift in their goals, I can't see thé gafam losing here.

Some are, but a lot of them are also made to be successful long term companies. Small business is still a large employer overall. Some businesses need to be very big, and if you want to be "filthy rich" you have to be in one of those. However you can be nicely wealthy at a small company.

I still have a hard time understanding what's going on with startups : the power of money makes it too attractive to sell and "call it a life" I guess. That's why even if they don't plan on doing this, startup might actually accept being bought. I guess.

a thé gafam : it use to taste really good but now it taste awfully bad, yet we cannot stop drinking it!

> Both Google and Amazon are barreling down the path of democratizing scale and infrastructure with their cloud offerings. It's never been easier to reach millions of users in a short amount of time with your product.

The second statement does not follow from the first.

What is true (partly because of the first statement) is that building is now easier than ever. Precisely because of that, however, rising above all the noise and reaching millions of users is now arguably harder than it was previously.

Presumably "it's never been easier" means "the technology is no longer the bottleneck".

The key word in the phrase is 'reach'. If by 'reach' you mean the equivalent of getting an email into someone's account, then the statement is true. You can easily spam millions of email addresses for pennies. You can get ad space on lots of browsers. You can cold call millions of people with automated systems.

On the other hand if by 'reach' you mean actually getting someone's physical and mental attention, then it can be harder than ever. Your message gets filtered out or swallowed up in the sea of other messages and ignored either by technology or by the users themselves.

It is like those pictures of NY Times Square (or any corner in Asia) where you have 10,000 billboards of all shapes and sizes all flashing some marketing message. It is easier than ever to buy a few seconds or minutes of ad time on one of them. It is harder than ever to get the passing crowd to take notice of your message when it is drowned out by all the others.

True. So laid off marketers are probably more successful founders than laid off engineers these days.

I don't understand why Google and Amazon would invest all this money in these employees building up their tech and tribal knowledge just to lay them off. Do these companies not expect to grow in the future? Seems like just freezing hiring for a year or two would save you more money in the long run rather than having to rehire and rebuild all this expertise.

> Seems like just freezing hiring for a year or two would save you more money in the long run rather than having to rehire and rebuild all this expertise.

It took Google 6 years from 2012 to double headcount.

In that time, revenue went from $50B to $136B (almost triple). Profit from $10.7B to $30.7B (triple).

In 4 years from 2018 to 2022, Google more than doubled headcount.

In that time, revenue went from $136B to $279B (barely double). Profit from $30.7B to $50.9B (only up 65%).

Almost all these companies have similar stories.

They're still good companies. They just hired way too many people thinking that the pandemic trend would continue for a long period of time - rather than revert to the mean.

If Google continue to grow at it's more long-term average rate - without the layoffs, you're looking at being overstaffed for years.

Why?

Because the recruiting process can cost over a year's salary, and that's ignoring the time it takes to get up to the same productivity as other employees once they start.

> Do these companies not expect to grow in the future?

Yes and wages were outpacing that growth, so as soon as someone took the first step, all the big tech companies piled on in short order. And, low and beyond, offers appear to be down right now. Mission accomplished?

On hiring outpaced need for employees or ROI

COVID era was a hiring boom. Most of the companies I saw doing layoffs still have significantly larger employment numbers than pre-COVID.

Unfortunately the market does not work on a year or two timelines.

There is a concept of internal layoff to invent, where a part of the workforce as to apply to a different teams, being fired from the previous one and hire by the next one.

For the anecdote, i know someone that was let go by IBM and hire by RedHat the same month two years ago.

These companies still have more employees than just before the pandemic.

“1 or 2 years?? Are you planning on living for ever??? That’s like 4 to 8 quarters!”

A lot of people even at Google just do run-of-the-mill stuff. Not everyone is making deep learning artificial big table or some such.

A rule of thumb is at any organization roughly square root of employees are critical. So at Google that number would be < 500 people. So even when I consider that layoffs are totally random (unlikely), there would be 40 or so people laid off which could be considered important loss. This is bad but not going to hamper Google significantly where they are going.

I haven't heard the "square root of employees are critical" rule of thumb. I have previously heard of Price's Law - paraphrasing: "half of the output is created by a square root of the organization". But as I understand it, there's not a notion of criticality. What if 80% of your output is critical? What if the n-n^0.5 group is on critical output and your square root group is slinging schlock at high volume?

Sometimes, you'll hear pundits say that these kinds of layoffs are aimed at "the bottom 5%" or whatever, but who's been around for these cycles knows that lots of good people get cut alongside the actual underperformers. There's also a lot of "last to be hired, first to be fired" in big layoffs, so presumably with those people you aren't losing too much tribal knowledge.

In some cases, the layoffs were targeted at particular areas of the company, like Amazon targeting their devices group. If they don't anticipate growth in that area, it may not be worth it (to management, anyways) to reallocate those employees to other parts of the company.

All that being said, these things tend to be driven by short-term financial metrics rather than long-term ones. A hiring freeze wouldn't have as dramatic an impact on next quarter's earnings.

> by short-term financial metrics rather than long-term ones. A hiring freeze wouldn't have as dramatic an impact on next quarter's earnings.

In the most recent quarter, Google’s profit was down 34% and it was the fourth consecutive quarter with decrease in profits. How many quarters do you think they should wait out before trying to stem the tide of costs outpacing revenues?

https://www.nytimes.com/2023/02/02/technology/alphabet-earni...

We are all docker containers running on a kubernetes cluster. Can be upscaled or downscaled as needed.

That's a horrible analogy since it assumes that a brand new hire is instantly as productive as any other employee.

That's not an analogy. It's a polemic.

> OpenAI challenging search

Not even close.