That's not science :)
However, you're right: in the article, it's not clear how we decided to consider the experiment as successful: we called a meeting with all the team and asked everyone something like: "Do you think that the situation on Slack is better now?", and the majority of people said "yes".
I added a note at the end of the article to address some of the comments ;)
Looking at your graph, it's not really clear that anything has changed significantly: when did the change get implemented? Is a couple weeks' change in activity a result of a couple of loud employees going on leave? There's such variability there that it's hard to say that anything has really changed.
I agree that a formal analysis isn't always necessary, but you don't appear to done _any_ here and are then couching a suggestion to try something on Slack as having measured something. That's just bad science! :D