News for hackers. Stuff that matters.
Re:Remove first 300M lines from 700 GB file on 1TB disk by speedgoose on Tuesday September 22, 2020 @11:20AM (#24555507)
I guess you weren't allowed to use a public cloud provider. Because you can always mount a network file system using sshfs, s3 like, or similar. Or upload the file on another server rented for one hour.
Re:Remove first 300M lines from 700 GB file on 1TB disk by Demiurge on Tuesday September 22, 2020 @11:52AM (#24555979)
I guess I could have asked for the CC information, setup a new AWS account, and have them pay for the storage and egress of 1.5TB of data. However, that very likely would have lasted quite a while as well, and in my case I had to minimize downtime as well. So, no, I would say even if you're allowed to do something, sometimes it is not the best solution.