Re:Remove first 300M lines from 700 GB file on 1TB disk
by anon946 on Tuesday September 22, 2020 @10:33AM
(#24554812)
If you print the offset after a write has safely completed, then it should still work. The internal copy is idempotent, up to the number of bytes in the first 300M lines. In other words, assuming reasonable line lengths, you can safely re-do a copy operation. So if you crash, you may not have printed out the last offset, but you have recorded a recent offset. You can then just restart from that recent offset.
[ Parent
| on HN ]
There are no comments on this comment yet.