At Mattermost we went for the do-it-yourself option and wrote a custom tool for the job [1]. After a lot of research on all the existing open-source frameworks we couldn't really find anything that would fit our use-case. We are quite happy with the result although, as the OP mentioned, there's a significant maintenance cost attached. As new features gets implemented and more API calls added you need to go back and make sure your user behaviour defining logic stays in sync with the real world. If I were to do it all over again, I'd probably give k6 [2] a chance but I am still convinced a tailored solution was the best choice.

[1] https://mattermost.com/blog/improving-performance-through-lo...

[2] https://k6.io/