> That seems like a good default, with the ability to choose an arbitrary image, or a video that will be automatically cropped to the duration of the "gap", or a segment of a video chosen by start-end timestamps.

Got it. Definitely doable ;)

> Not talking about containers. When trying to concatenate videos with different codecs, dimensions, frame rates, bit rates, etc. ffmpeg will issue warnings and output a video. Playing that video messes the players up.

If I understand correctly, that seems like a bug in ffmpeg.