> Do we know if this sort of message causes a person to be more or less likely to take their lives? How can you A-B test that?

Actually... you probably could train a model to detect suicidal posts. I had a friend who committed suicide a few months ago, and I didn't see it coming. But looking back, there were signs in his facebook posts (in line with this article's example of thanking everyone), but our conversations were normal. It is at the very least a wishful line of thinking for me that ML can recognize this type of pattern.

To reply to your middle paragraph, you'd probably want a longitudinal cohort study rather than an A/B test. A well-designed one can provide pretty conclusive indications in situations where A/B testing (or, testing with a double blind control as the epi community might call it) isn't possible.