News for hackers. Stuff that matters.
Re:Remove first 300M lines from 700 GB file on 1TB disk by Demiurge on Tuesday September 22, 2020 @10:46AM (#24555009)
Very recently I've been in a similar situation with a dedicated server. There was no way to add storage and even getting another server would have been pretty expensive. Luckily it was a raid, so I just had to double the storage temporarily :)
Re:Remove first 300M lines from 700 GB file on 1TB disk by speedgoose on Tuesday September 22, 2020 @11:20AM (#24555507)
I guess you weren't allowed to use a public cloud provider. Because you can always mount a network file system using sshfs, s3 like, or similar. Or upload the file on another server rented for one hour.
Re:Remove first 300M lines from 700 GB file on 1TB disk by Demiurge on Tuesday September 22, 2020 @11:52AM (#24555979)
I guess I could have asked for the CC information, setup a new AWS account, and have them pay for the storage and egress of 1.5TB of data. However, that very likely would have lasted quite a while as well, and in my case I had to minimize downtime as well. So, no, I would say even if you're allowed to do something, sometimes it is not the best solution.