Back in 2012 or 2013 I built an auto-scaling WordPress hosting off of Azure. It worked OK, better than shared hosting, but the $15 base VM at Azure (it was a cheaper time back then) couldn't compete with the base $10 droplet at Digital Ocean (it was a more expensive time back then).

On top of that Digital Ocean's API was so much better than Azure. So I spent a week and switched to Digital Ocean. Needed to scale the instances less, cost less, so more profit!

Unfortunately after 2 months of development, and half a year of marketing it, we only managed to attract a handful of customers, and had to shut it down.

Why did try to scale when you had no customers?

I guess I wasn't perfectly clear. The technology was to scale the customer's wordpress installation when certain metrics were met CPU or RAM > 75%, clone droplet and load balance.

The technology was basically an a beefy server for NFS, a beefy MySql server, and a beefy load balancer. When you started hosting with us, you paid $50+/mo and got what felt like a standard WordPress install on a dedicated digital ocean droplet, when metrics showed that you were hitting the scale threshold, we would create a new droplet run an initialization script to mount NFS, set up the nginx server, and then attach it to the loadbalancer.

When the combined CPU or RAM droped below the initial scale threshold, it would drop one VM from the load balancer at a time keeping it provisioned for an hour in case it needed to scale again.

Worked amazingly well. Sure the load balancer was a single point of failure, but in the 10 or so months that it was running, we never had a second of outage based on pinging the websites.

Now that is very cool! I've always wanted to try something like that.

What was it written in?

The main application I left hosted at Azure. It was an ASP.Net MVC app.

I used the Digital Ocean API to provision infrastructure, and SSH to remotely login and run the initialization bash script. The init script configured nginx, NFS (maybe it was SMB, i dont rmember now), a cron job to minutely heartbeat statistics back to the MVC app.

There are so many better ways to do it now, though. Off the shelf load balancers with letsencrypt and an api, hosted dns, managed & clustered databases, monitoring api. It would be a lot simpler today than it was just 2 years ago, let alone 7.

These days you can do it (relatively) easily by using wordpress docker image on kubernetes (or any other container platform). e.g. using Traefik for load balancer, a mysql container or an external mysql db instance (aws, google and do now offer managed mysql), and storage provider that can be mounted from multiple nodes (nfs, glusterfs, ceph, etc) and you're done! You can throw in an sftp container that mounts /var/www so you can manage the wordpress installation using an sftp client.

If you want to offer a completely automated process, auto-scaling is important. Otherwise it'll fail as soon as the first customers come in.

Maybe because Windows updates and xbox love run through the network?

Pretty sure all of Netflix runs on AWS and I've got to think that's several orders of magnitude more traffic than Windows updates or xbl.

Netflix hosts video content at ISPs' sites: https://openconnect.netflix.com/en/

Edge content yes, but everything else is in AWS

The edge traffic makes up 99%+ of the traffic in question.

Do they publish how many ISPs participate? I am sure the big ones do but who knows if local Fiber is connected to someone who is behind an OpenConnect

Dominican here studying Telecommunications. I was giving a tour to the Datacenter of the 3rd largest telecom company in my country. They were using the Netflix box. I was giving a tour to the 4th largest telecom company, they too had the Netflix box. I was told the 1st and 2nd largest telecoms in the country also had the box.

My understanding is that Netflix will pretty much subsidize the cost of the CDN box to nearly any ISP above a certain minimum traffic level ingress from Netflix.

Great model to attack the incumbent [transport / transit] carrier costs - end run around the middle-men and drop your CDN boxes directly at the edge (last-mile) ISPs.

ISP here, the netflix cache appliance boxes (a single 1RU server is the minimum) are free, but you have to provide rack space, power and cooling for them. For an ISP in a place like a caribbean island it's a great idea because their upstream/IP transit rates are significantly higher than in the US48 states.

Non-participating ISPs get it from participating ISPs. Basically no one hits AWS.

FWIW, the video streaming part of Netflix does not run in AWS (but, AIUI, pretty much everything else does).

Oh, TIL I guess. Thanks for the correction.

It's massively, massively overcommitted. They managed to actually run out of hardware in a whole US region recently, even though of course most customers had lots of unused quota. A typical Azure VM must be extraordinarily busy given some of the latencies and speeds I've seen on them.