News for hackers. Stuff that matters.
Re:One Billion Files: Scalability Limits in Linux File Syste... by dr_zoidberg on Wednesday April 06, 2016 @10:35AM (#11439006)
Example from another field: data recovery on a hard drive yielded over 1,5 million posible JPGs. Linux died whenever we tried to get on the folder to do anything at all (we couldn't even move/rename the files with perl or python). Finally, we just deleted the directory and modified the carver program slightly[0] to extract 10k files, then make a new directory for another 10k files[1]. This was a few years so it might not happen anymore (with "just" 1.5M files)

[0] this program was an in-house solution that handled some edge cases other programs available at the time didn't handle so well.

[1] which is pretty much standard behaviour in file carvers, but wasn't part of the in-house development yet.

There are no comments on this comment yet.