Turns out I don't much care for Jellyfin's approach to music, but for videos it's pretty solid, and I found it very easy to set up. Thanks for the recommendation!

(As for music, I ended up just running Airsonic in another container on the same box, so all set there till something better comes along. Navidrome, maybe...)