I have Jellyfin setup alongside Plex, and the one thing that's keeping me from moving fully off Plex is Plexamp [1]. It works flawlessly on my phone/laptops, includes a ton of features and is a pleasure to use - especially compared to Spotify or Apple Music.

I am aware of FinAmp [2] and music players for Jellyfin, but they aren't comparable to PlexAmp yet.

1. https://www.plex.tv/plexamp/

2. https://github.com/jmshrv/finamp