libavcodec tries to recover from missing frames (you need this for TV broadcasts) but it's actually not very good at it for H.264. You'd almost always see major artifacts.

Any idea what the exact warning text is?