I think the potential for IMs to be distracting is there, but my closest friends all fully understand that I will respond whenever I have a moment, and so will they, so we don't have a sense of urgency with the thing. It's more like an extended conversation with history, so we can come back to it, catch ourselves up by reading what they wrote (sometimes hours inbetween responses) and pick right up where we left off.
I feel constant barrage of notifications and distractions simply fragment your time. Like, whether you do that communication in one short block or keep the mail/IM constantly open, you generally accomplish the same amount of "communication" in a day. But the second case is like fragmented heap - it reduces the length of consecutive free memory blocks, making you increasingly unable to allocate larger blocks of continuous memory. Instant reaction to incoming messages does the same with concentration, making it hard to perform tasks that require longer stretches of focused work.
Truth is they're not getting much done but they feel like they are.
Actually that's an over simplification. With something like Slack people get less of "their own" work done but more of certain types of group activity such knowledge sharing, forming a consensus and decision making.
In other words any kind of activity that requires extended periods (30mins+) of concentrated work will suffer if you're tapped in to Slack or otherwise. That includes things like writing code, creating a presentation, planning a project and similar...