I concur. I've tried listening to podcasts in the background while doing anything on PC and it always results in me losing track of what has been said. So work, games and any video with graphical content (i.e. not just faces of people talking) all hinder listening to podcasts which cuts out about a half of the awake time. And in the rest - I would either listen to a music or to silence. The only viable place for podcasts I've found was driving, and not any driving (because same issue occurs - losing track of the topic when actively driving), but specifically standing in the traffic jam for an hour. There yeah, it fits.

And another problem is that after listening to chunk of a podcast and having to stop in the mid stream I almost never return to continue it. It just stops being interesting and I lose context. With some exceptions of course.