What practices do you use to ensure higher availability than Slack? Hope is not a strategy.

(Answering my own question, I'd guess that the strategy is "never change anything". To some extent, I don't need 80% of the features that Slack offers, and code that you don't have is code that can't break. Not restarting stuff to release new code is also good for uptime. Would be interesting to see a SaaS that has a super-stable-never-change-anything release track, hosted on infra with similar goals, and see how that uptime compares to people's homegrown version of the same strategy.)

My public facing servers (well https with client certificates that access them) are constantly upgraded, there's two of them, occasionally one goes down for a reboot overnight to get a new kernel, but clients will point to the other one.

Radius does vrrp, not instant, but acceptable

For true high availability (when downtime of milliseconds being unacceptable), we have A and B systems (sometimes more). E.g. syslog - router sends the messages out of two interfaces to two different servers which don't go down at the same time.

For local staff wanting access to internal webpages, we can cope with 20 second downtimes at an appropiate time.

I don't run a local irc server, but I'd be very surprised if it was any less stable than any other standard linux box -- do an apt or yum upgrade occasionally, any service outages you plan during a team meeting (thus we don't care if the chat system is down as we're talking via another method).

Stability is far easier when

1) Your server isn't open to the whole world

2) Your server isn't scaling to millions of users

Hell I've got a tcpdump session on one box that's been running since before the last slack outage 3 months ago.

>up 1511 days

What's the security update status on that?

Awful I'm sure, but doesn't need to be, all it does it run a lot of snmp queries on an isolated network, store the data, and serve static webpages to via a proxy

>Awful I'm sure

I don't really want to describe our company messaging platform to have "awful" security.

The truth is that you don’t know.

And slack has been breached before. So this is a weird argument to make. The point being made is that you can determine your own trade-offs. With slack: you simply can’t.

Because it's hard to keep services up and most shops aren't especially competent.

Having a process running for 1511 days does not mean a service has 1511 days of uptime -- there are a ton of places stuff can go wrong.

Running a simple service on a single server for a year without issue is not hard.

Running a service like slack is hard, but modern HN attitude is if you can't scale to a billion users your architecture is rubbish.

Running a service isn't hard.

Going a year without any sort of outage is. For example, if you're just serving from one place, any network outage between the users and that place will be an outage -- seamless failover is pointlessly hard and very few shops do this for their internally-hosted services.

Depends how you define a network outage. One of my transatlantic circuits has been a bit ropey - had a 2 second outage on it a couple of nights ago, does that count as a network outage?

If a user has a local network problem, that's not a loss in the service. If multiple users do that's an issue. My home broadband went out for 3 minutes at 23:33:43 GMT on May 13th, but then I wouldn't run a service needing 24/7 on my home broadband.

My frickin nasdrive has a better uptime than slack.