Jellyfin runs on about 700-800MiB of RAM for me. It sounds like you're hitting a memory bug if you're running out of RAM just serving files, to be honest. I've never used Plex but I doubt it runs with just 70MiB.

Perhaps it'll use much more RAM transcoding, but I'd still expect a dedicated 2GiB to be plenty for Jellyfin even in extreme cases. I doubt you'll get any decent transcoding speed out of ffmpeg if you need a buffer that size anyway.