While "portable MP3 player" implies "not a cell phone", have you considered use of an old cell phone and an open source music player (i.e. one example here: https://f-droid.org/en/packages/ch.blinkenlights.android.van...).

The "phone" does not even need to work as a "phone" anymore, it just needs to work enough to open the player app and play music.

And with the example player above, it "does not require a user hostile music ecosystem". It simply reads the phone's filesystem and catalogs all mp3 files it finds, and then lets you play those files.

It (using either an old cell phone, or your current one if you have one) will allow you to "just to be able to load and play mp3".