News for hackers. Stuff that matters.
Re:Unix Commands I Wish I’d Discovered Years Earlier by ck2 on Tuesday September 10, 2013 @03:55PM (#6362935)
I am not saying find is outputting files, I am saying it is decoding the full table entry for each file and formatting it, then passing that listing as an entry to wc. Which has to be a bunch of overhead that adds up when you have hundreds of thousands of files to count.

All we want from find is "count the number of inode entries in this tree branch that are type f". No formatting, no output, just count. That has to be much more efficient.

Re:Unix Commands I Wish I’d Discovered Years Earlier by wiredfool on Tuesday September 10, 2013 @04:24PM (#6363192)
OMM, looking at 700k files on a spinning disk, I'm getting an uncached run at 1m13s, and a cached run at 1.2 seconds.

That's "time find . | wc -l"

It's going to be disk limited unless you're in cache.

Switching to "time find . -printf '1' | wc -c" lowers overhead to .7 seconds, so roughly 1us/file.

Re:Unix Commands I Wish I’d Discovered Years Earlier by jbarber on Wednesday September 11, 2013 @04:34AM (#6366132)
There is a more important benefit to using the -prinf '1' approach - which is that it won't miscount the result if any of the filenames contain newlines.
Re:Unix Commands I Wish I’d Discovered Years Earlier by res0nat0r on Tuesday September 10, 2013 @04:05PM (#6363022)
You are going to need to rely on the filesystem authors to write a tool to traverse their inode trees directly, or probably write something like that yourself.

https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux....