I have regained a lot of focus by using the Unread filter. I don't think it's on by default, you have to go into settings. I use it to mark the events in my day where I go to slack. I just hit the unread filter, read through all that I need or want to, respond, and then decide if I'm going to be in slack for a while (e.g. A conversation that will be having a lot of back of forth) or if I can safely context switch back to whatever I was doing or something new. The Unread filter has keyboard shortcuts too, so you can navigate stuff quickly.

This may also be an opportunity to raise the issue. If it's not your style to be direct about it, consider just asking in your team channel for advice on this very issue. Maybe it's an important conversation to open up at your workplace.