I wonder though if this could be solved with edit lists, supported e.g. by ISO BMFF. Not all players support them completely, though, for example I believe mpv only supports one edit, so I guess that's not useful for general purpose use..