News for hackers. Stuff that matters.
Re:Remove first 300M lines from 700 GB file on 1TB disk by tyingq on Tuesday September 22, 2020 @10:27AM (#24554709)
You can do it with a single filehandle, reading the data then seek()ing backwards before you do the writes. A Perl example that reads a file with the numbers 1 to 100 then replaces the digits with X's:

  #!/usr/bin/perl
  open(my $fh,">foo") or die($!);
  for (1..100) {print $fh "$_\n"};
  close($fh);
  open($fh,'+<','foo') or die($!);
  while(<$fh>) {
    my $spot=tell($fh)-length($_);
    seek($fh,$spot,SEEK_SET) or die($!);
    s/\d/X/g;#replace digits with X's
    print $fh "$_" or die($!);
  }
  close($fh);
If his file has lines of equal size, this would be fairly easy to do in place. Would be a little more complicated if not.
Re:Remove first 300M lines from 700 GB file on 1TB disk by yencabulator on Wednesday September 23, 2020 @07:50PM (#24573669)
Number of file handles is irrelevant, all handles for the one file are accessing the same data.
Re:Remove first 300M lines from 700 GB file on 1TB disk by tyingq on Thursday September 24, 2020 @01:54PM (#24581379)
Just pointing out it's not needed. Why is that irrelevant?