Ah, but as always, the million dollar question is ..... What does the PagerDuty small print say ?
My guess is the small print is not "bulletproof reliability" or five-nines. I betcha their contract is full of exclusions and get-outs.
It's a bit like the famous Verizon 100% SLA. Any idiot knows its not technically feasible, but the reason you pay the Verizon-tax is so the've got some cash to pay you the inevitable SLA claims.
You still want reliability that's far higher than your contract baselines.
What matters is the execution and word of mouth. And PagerDuty is widely known to be rock solid.
Honestly, PD is probably the service I would the least complain about in our (large org) monitoring chain.
At least, it's reliable, which cannot be said of Newrelic for example.
1: https://www.pagerduty.com/standard-service-level-agreement/
Regarding Uptime 99.99%. That's very true and also a good hint.
My current stack is very robust - tried and tested in many other products that I worked in.
Some other HN user also suggested to include an uptime status page to create this kind of confidence.
For uptime you also need to consider the availability of your hosting provider, you might even have to have a fallback installation at a different provider, something along these lines.
If your design requires that the alerting service can receive a one-off affirmative "something's broken" packet, then yes, you are asking an inherently unreliable distributed system (i.e. the Internet!) to reliably deliver a critical message at a time when you know something is broken. Good luck. :)
Instead, if you use something like a periodic heartbeat (also known as a dead man's switch, inbound liveness monitor, or outbound HTTP probe -- all of which we support at Heii On-Call https://heiioncall.com/ out of the box), you can tolerate some occasional lost messages, regardless of whose end they are on.
Real reliable systems (for example, embedded systems) use periodic heartbeats and watchdogs, and are usually designed to be lenient to the occasional missed heartbeat. If the system being monitored is truly down, then enough consecutive heartbeats will be missed that some threshold is reached and the on-call person can be alerted (or a watchdog timer can reboot a system, etc).
I sincerely wish you luck with allquiet. I just want to make very sure you are aware why people still pay for Pagerduty. To compete, you need to be looking at 99.99% uptime or better (ideally 99.999%, 5 minutes of downtime a year) where 'uptime' is defined as the ability to exercise the entire notification stack. The moment someone's site has an outage and you aren't able to deliver the notification, you lose the customer and everyone they talk to.
I also worry about in-app notifications, but that's well-covered by everyone else's comments.
Pagerduty is vulnerable. Their UI is garbage. But you need to have bullet-proof uptime to take them down. It's a tough challenge and I wish you luck!