The stat() results are going to be cached.
What is fair for a test like this? SSD or rotational media?
At least clear the caches between runs, or give them the same cache.
https://www.thegeekdiary.com/how-to-clear-the-buffer-pagecac...
Okay, updated results (cleared all the caches and restarted shell/terminal between runs):
"time command ls -lR --color=always": 12.31 secs
"time command exa -lgR --icons": 11.32 secs
Still, no performance difference on my system. In fact exa seems to be slightly faster (I did the test multiple times and it was alwasy ~1 second faster). This is on a physical HDD and I could hear it churning equivalently on both of the runs.
"time command ls -lR -color=always": 898 millis
"time command exa -lhgR --icons": 638 millis
This is in a directory of roughly 17k files of varying filetypes. Exa also outputs underlines and icons.
---
Admittedly if I create 80000 arbitrary extensionless files "ls" takes around 700 millis while exa takes around 1000 millis, but still the performance difference is nowhere near what you demonstrated.
I don't know what job you're doing that requires "ls"-ing >20k files, but on my system there seems to be no performance penalty before an uncertain, unachievable threshold.