Being the servicer allows for all sorts of griefing that falls completely within regulation guidelines.
My service has done dumb things, all presumable by accident, like: deposit my check into a different account, charge me a late fee and not refund; change my escrow amount 4 times in a year; not take electronic payments.
It’s annoyed me just due to stupidity. If my arch enemy owned my loan they could do stuff like change payment addresses; “lose” payments; late pay taxes and insurance.
Here's my go to doc about mortgage servicing in the USA. From 2007, but as far as I know still accurate:
https://www.calculatedriskblog.com/2007/02/tanta-mortgage-se...
If you beneficially own the economics... well, great, your arch enemy is theoretically paying you money, but you have no options to screw with them. You own an extremely regulated specialty financial product. This is similar to the misconception that stock in Google entitles you to just walk in and take a computer in exchange.
If you buy the servicing rights, you have much more surface area to be an incompetent servicer, but again extremely regulated and you’re pricing yourself to being sued by the homeowner, the GSEs et al who set up the securitization program and zealously defend it, and potentially even the entity owning the economics.