10-15 years ago for music I had a cheap MP3 player that also sported a radio tuner. It could take 10mins to transfer a few MB of music, but I was using that for music. It was very slim, imagine a knock-off of an iPod (the ones with the tiny no-touch screen).

Perhaps get one of these? They shouldn't cost more than £€$50 for a couple of GB

for a couple GB? You should probably get 60-100 GB for that I'd bet.

You can load and play audio files on a feature phone. Many people did this before iPhones and streaming services existed.

Cron job to copy both to a microSD card?

Reminds me of my old casio watch that synced calendar reminders via a sweet dock. Physical actions to do syncing and remembering every time I left my desk wasn't that long ago.