Re:Remove first 300M lines from 700 GB file on 1TB disk
by TheOtherHobbes on Tuesday September 22, 2020 @10:57AM
(#24555163)
Without getting into the details of video compression, you can't remove arbitrary parts of a video file without causing problems when the file is played.
Even if the video is uncompressed you will still need to remove entire frames and then - for most formats - reconstruct a file header at the start of the file.
This is a much harder problem than throwing away x% of a file without worrying about the structure of the content.
It's fairly easy to solve that problem at the disk block level. But if you're trying to keep the data structures error-free, you're going to have to do a lot more work.
Deleting a non-block-aligned segment of a file means copying the remaining data over it and then truncating the file at the end using the ancient truncate function. It's not obvious how that can be improved.