Re:Remove first 300M lines from 700 GB file on 1TB disk
by tzs on Tuesday September 22, 2020 @10:27AM
(#24554717)
The old fashioned way to deal with making sure you see your printed offsets is to actually print your offsets. You'd go to the machine room and run the command on the system console, which would typically be a hardcopy terminal like a DECwriter.
The trendy modern way, of course, would be to use the cloud. Upload the offsets to the cloud as they are produced, pausing until you get confirmation after each upload.
A more clever and/or foolhardy way is to put the crash recovery markers in the file itself. Place evenly spaced markers throughout the file, saving the data that they overwrite in a separate file. Then do your in-place file shifting. Finally, replace the markers with the saved data.
The trendy modern way, of course, would be to use the cloud. Upload the offsets to the cloud as they are produced, pausing until you get confirmation after each upload.
A more clever and/or foolhardy way is to put the crash recovery markers in the file itself. Place evenly spaced markers throughout the file, saving the data that they overwrite in a separate file. Then do your in-place file shifting. Finally, replace the markers with the saved data.