Show HN: Fdir – Faster Node.js glob alternative

Re:Show HN: Fdir – Faster Node.js glob alternative by BiteCode_dev on Sunday March 29, 2020 @05:40PM (#22723017)

I took it for a ride

They all list all files recursively in a sync fashion, from the node_modules dir (like in the benchmark), excluding dirs, and print a total count.

Here are the results calculated with hyperfine (https://github.com/sharkdp/hyperfine):

    hyperfine "bash test.sh" --warmup 5
    Benchmark #1: bash test.sh
    Time (mean ± σ):       7.5 ms ±   0.5 ms    [User: 4.5 ms, System: 4.2 ms]
    Range (min … max):     6.9 ms …  10.5 ms    332 runs

    hyperfine "perl test.pl" --warmup 5
    Benchmark #1: perl test.pl
    Time (mean ± σ):      25.6 ms ±   1.2 ms    [User: 16.8 ms, System: 8.8 ms]
    Range (min … max):    24.0 ms …  30.8 ms    97 runs

    hyperfine "python3.7 test.py" --warmup 5
    Benchmark #1: python3.7 test.py
    Time (mean ± σ):      43.4 ms ±   1.4 ms    [User: 32.6 ms, System: 10.9 ms]
    Range (min … max):    40.9 ms …  46.8 ms    66 runs
    
    hyperfine "ruby test.rb" --warmup 5
    Benchmark #1: ruby test.rb
    Time (mean ± σ):      66.5 ms ±   2.0 ms    [User: 52.1 ms, System: 14.4 ms]
    Range (min … max):    63.2 ms …  70.3 ms    42 runs

    hyperfine "node test.js" --warmup 5
    Benchmark #1: node test.js
    Time (mean ± σ):      83.7 ms ±   4.0 ms    [User: 74.7 ms, System: 15.6 ms]
    Range (min … max):    79.4 ms …  95.3 ms    36 runs

Here are the results of an hello world with each runtime for comparison:

    hyperfine "bash test.sh" --warmup 5
    Benchmark #1: bash test.sh
    Time (mean ± σ):       1.2 ms ±   0.3 ms    [User: 1.1 ms, System: 0.3 ms]
    Range (min … max):     0.9 ms …   3.8 ms    1521 runs

    hyperfine "perl test.pl" --warmup 5
    Benchmark #1: perl test.pl
    Time (mean ± σ):       1.3 ms ±   0.3 ms    [User: 1.3 ms, System: 0.3 ms]
    Range (min … max):     1.0 ms …   5.3 ms    1103 runs

    hyperfine "python3.7 test.py" --warmup 5
    Benchmark #1: python3.7 test.py
    Time (mean ± σ):      19.5 ms ±   0.9 ms    [User: 16.2 ms, System: 3.4 ms]
    Range (min … max):    18.3 ms …  23.7 ms    144 runs

    hyperfine "ruby test.rb" --warmup 5
    Benchmark #1: ruby test.rb
    Time (mean ± σ):      55.2 ms ±   2.2 ms    [User: 47.2 ms, System: 8.1 ms]
    Range (min … max):    52.2 ms …  61.9 ms    51 runs
    
    hyperfine "node test.js" --warmup 5
    Benchmark #1: node test.js
    Time (mean ± σ):      55.4 ms ±   1.8 ms    [User: 49.5 ms, System: 7.0 ms]
    Range (min … max):    53.0 ms …  60.0 ms    53 runs

Now, my machine is not setup for a clean benchmark. The disk cache is warmed up. Hyperthreading is on. Other softwares are running.

Plus the scripts all found a slightly different number of files :) I suspect they all treat symlinks/dotted dirs differently, and I didn't take the time to normalize. Although I don't think this makes up for the difference.

Still the result is a bit interesting. The non JS tests are not using any 3rd party libs. Ruby and Node seems to have the same cost for VM startup.

I'm quite surprised that node is last frankly, especially on a uber optimized code. I'm expecting V8 code to be blazing fast as it's Google made and C++.

[ Parent | on HN ]

Re:Show HN: Fdir – Faster Node.js glob alternative by thecodrr on Sunday March 29, 2020 @06:17PM (#22723284)

Hey, thanks for taking the interest and the time to audit and benchmark.

You can get a rough idea of VM startup time by running a simple hello world program.

fdir is supposed to compete against nodejs alternatives so I am not surprised it comes last. Maybe the next goal should be to optimize it against other cross language platforms.

V8 does a lot of optimizations but there is still overhead. A native solution in bash has little to no overhead so no wonder it's the fastest.

Anyway, thanks again. This benchmark was insightful (also humbling).

[ Parent | on HN ]

Re:Show HN: Fdir – Faster Node.js glob alternative by BiteCode_dev on Sunday March 29, 2020 @06:43PM (#22723470)

Gonna add test with hello world to see vm startup time.

[ Parent | on HN ]

Re:Show HN: Fdir – Faster Node.js glob alternative by thecodrr on Sunday March 29, 2020 @09:42PM (#22724466)

If we subtract the vm time from total time that should (very roughly) be the time taken for fdir to crawl the directory.

So basically:

83ms - 55ms = 28ms

Not bad, not bad. :D the scale remains the same tho so not that beneficial.

[ Parent | on HN ]

Re:Show HN: Fdir – Faster Node.js glob alternative by 29athrowaway on Sunday March 29, 2020 @06:03PM (#22723187)

1) Did you take frequency scaling into consideration while benchmarking?

2) v8 doesn't make your code blazing fast by default. It has many optimizations but your code may be written in a way that v8 cannot fully optimize.

3) I recommend this tool as an alternative to using "time": https://github.com/sharkdp/hyperfine

[ Parent | on HN ]

Re:Show HN: Fdir – Faster Node.js glob alternative by BiteCode_dev on Sunday March 29, 2020 @06:14PM (#22723264)

As stated: "the times varied a lot because my machine is not setup for a clean benchmark"

So this is not a benchmark. Just a fun observation of a trend. A proper benchmark would require to have a dedicated machine with no other softwares running, hyperthreading off and run each of them a 100 times, without fs cache.

EDIT: gonna edit the original post with hyperfine results.

[ Parent | on HN ]