https://github.com/Microsoft/WSL/issues/873#issuecomment-425...
WSL2 itself uses a virtual file system.
On Linux, and other unices, there have always been important workloads where it really matters, so that itch got scratched.
Windows metadata performance is horrendously bad for code that needs to get metadata for a large number files in the same folder the way it would be done on Linux: by calling the Windows equivalent of `stat` (`GetFileAttributesEx`) for every file.
Metadata performance is much better if the code uses `FindFirstFileEx`, which IIRC has no direct Linux equivalent, to grab metadata en masse for all files in a folder and then filtering the results down the files that the code needs to know about.
Unfortunately, most code ported to Windows from Linux uses `GetFileAttributesEx`, under the assumption that it's as fast as `stat` is on Linux, but results in a poor user experience. Worse, many modern cross-platform languages (including Python) don't expose `FindFirstFileEx` without platform-specific add-ons, which means that devs have no easy way to do Windows metadata access right even if they know there's one specific way to do it.
On the other hand, if you look at e.g. Windows Explorer. Explorer will take somewhere from a few seconds to ~tens of seconds to list the contents of a SSD-backed directory with a couple thousand files. The same operation on Linux is practically instant. Copying a bunch of small files on Windows takes forever, maybe doing around a few hundred files per second. Meanwhile Linux will happily copy thousands of files per second, all on the same hardware.
Explorer might be using incorrect ways to do I/O, but if that is the case, then that would mean that the proprietary, only-ever-developed-for-Windows-by-Microsoft tool that is also the main way users do I/O doesn't do I/O properly. What would that say about Windows?
I wouldn't be surprised if it used the wrong API (in fact the lack of long path support is an example of using an outdated API). The only reason Windows Explorer appears reasonable is because people generally consider creating files you can't access with Windows Explorer a bug or misfeature.
In fact, if you use Total Commander, in the settings you will find option regarding things like "getting correct icons for files", with the option for "display icons for all files" explicitly marked as "SLOW".
Explorer always gets the icons, and possibly other metadata.
The last time I needed something along these lines the only option were os.listdir or using FindFirstFile through win32py. Os.listdir was an order of magnitude slower for my use case.