It really depends on the reliability of your underlying servers; it's pretty easy to get nginx or haproxy to be reliable: get your config set and don't touch it. Then you're just down to hardware issues, which are generally fairly rare. If you need to mitigate hardware issues, you'll need to do something fancy with IP takeover/ECMP etc, which I agree is a significant amount of work.
The one thing that people really need to think about though is when you let other people do TLS termination on your behalf, you're putting the private information of your customers into the hands of someone else -- they could disclose it due to bugs in their system, rogue employees, system intruders, etc. Layer 7 load balancing features can be nice, but they come at a price. This is something to consider even if your load balancer is provided by the same people providing your servers.
The usual way to have failover on load balancers is VRRP [1] and that doesn't work well in the cloud.
[1] Also known as HSRP, VRRP, keepalived, CARP
Edit: my point is, if a "cloud" load balancer can't provide the features you want (e.g. TLS SNI) and the "cloud" VM's can't provide the features you need to run your own LB (e.g. IP failover) maybe this is an opportunity to learn that the "cloud" isn't some golden goose shitting out golden eggs.
Scalability means that you don't have to worry whether it's 10 or 1M reqs/sec. And downtime is not acceptable for us, which is why using a more reliable service makes a difference.
> And downtime is not acceptable for us
You do know the Google SLA allows load-balancer downtime in excess of 20 minutes a month, right?
And yes, Google's SLA is fine with us, considering they will ultimately be far more reliable than we could be with our own team.
> Worth it for what exactly? I'm not sure what tradeoff you're talking about.
Now it's:
> It's a tradeoff which makes sense for probably 99% of the users
Anyway. So lets look at your other points.
> Reliability, security, performance, ops overhead and cost make a big difference.
Right, so your suggested solution is to put a HTTPS terminating CDN in front of Google Load Balancers (which presumably means you then need a different LB config+Cert for each domain you're handling in the SNI CDN?)..
> Reliability
Adding a layer to your stack like this doesn't improve your reliability - it adds a moving part, and moving parts can break.
> security
I'll just leave this here. https://news.ycombinator.com/item?id=13718752
> performance
Right, because a 'all things to all men, using shared resources' is always going to perform better than something you control, and thus configure to your specific needs.
> ops overhead
Yes, you need people to work on infrastructure you control.
Guess what. You still need people to work on infrastructure you don't fucking control. You find a "cloud" provider who will claim that Ops staff are not required, and I'll show you a fucking liar.
> cost
Right, because who cares if it doesn't do what you actually need and isn't really secure at what it does do, it's cheap.
You can put Cloudfront with SNI in front of GCP and all have it point to the same LB.
That's not how reliability works. Every non-trivial app has hundreds of "moving parts". So what if cloudflare had an issue? There are thousands of security issues every day, it's a fact of life. Better to have a well-funded and talented team who takes care of it.
And no, you don't need ops people to manage a cloud load balancer. What is there to do? And yes, cost is (one of many factors) that matter to business and it's still cheaper and better to put both services together (for the few who need SNI). Why do you keep saying it doesn't work?
It seems you have some irrational hatred for cloud and managed services but I'd much rather invest in and trust them than have you run my infrastructure and waste time building some fragile load balancing system instead.
You're arguing that a "cloud" solution must be more secure, and that this is very important, but when I point out that the exact multi-tenant zero-customer-control setup you are championing, led to a massive leak of data that people everywhere assumed was private, your response is:
> So what if cloudflare had an issue? There are thousands of security issues every day, it's a fact of life.
> It seems you have some irrational hatred for cloud and managed services
No. I have a strong dislike of people cargo-culting the fucking shit out of whatever cool kid buzzword they last heard, without a clue what the alternatives are, if they even need it, or what the consequences of their choices are.
> I'd much rather invest in and trust them than have you run my infrastructure and waste time building some fragile load balancing system
I won't lose any sleep over not getting your business. As you have all the answers already, what you're looking for, is called a "yes-man".
I look for people who understand business trade-offs, how to effectively spend time and money, have a proper understanding of risk, and know when a specialized vendor is a better for non-mainline business functions.
What exactly are you going to do when you find a hole in the load balancing software you use? Or do you write that software yourself too? What about the OS? What about the hardware? What about the datacenter? What about the transit fiber? Do you just build your own internet then? No matter what you do, you're trusting other vendors at some point.
The only possibility is to look at the entire situation and assess the risks and costs - which from this conversation seems to be your biggest problem.
You too will realize that when your load balancer will go down in the evening or in the week-end while you are away. Your site will be 100% dead for a very long amount of time.
Google gives you away the solution to your problem for free. Take it.
> Google gives you away the solution to your problem for free. Take it
I'm pretty suspicious of anything any company "gives away for free", particularly an overblown advertising company.
Google provides an incredible global load balancer tied into their networking fabric which you can't compete with. Why not use it? Businesses have limited resources and spending them on creating the same functionality at less scale and reliability just to have "control" (whatever that means here) is not worth it.