Care to explain why do you think NTFS is crap?
From my experience it's usually bad assumptions about files under windows and every application or library tries to stick to posix interface when dealing with files (open, read/write, close) which tends to block for longer periods on windows than on linux counterparts which results in significant perfomance loss .
Linux first software will always outperform windows implementation and Windows first software will outperform Linux implementations unless you provide separate code paths to properly handle underlying OS architecture and assumptions.
On Windows closing the file handle is extremely costly operation due AV checks and file content indexing [1]
I don't know if it's crap but it's much much slower than EXT4.
I remember reading a comment here that Windows in a VM on a Linux host was faster than bare metal.
Probably not true but I decided to make a test. I have a .net core app that insert data in a sqlite db (the resulting db is about 300 GB).
So I benchmarked this app on Linux (it was previously running on Windows) and IIRC it ran about 4 times faster.
In my Linux VM I had to use the time command to even have an idea of how long it took, as it seemed to return immediately. I think 5-15ms but it already a while ago.
On the Windows machine where the Linux VM ran it took several hundred ms.
Using either memory mapped files of overlapped io (IOCP).
It's tricky to use when you want to write the content since you must preallocate the file before you start with the writing. Appending to file just doesn't work under NT kernel since WriteFile blocks even if you use overlapped io.
Devs just need different mentality when it comes to Windows programming compared to Linux. Due the fact that everything under NT kernel is operated asynchronously you'll have to adapt your code to such concept. Meanwhile under Linux you had no other alternative for nearly 30 years (io_uring and friends) so if you wanted to be portable with minimum OS specific code then you had to implement things in synchronous way or write two separate code paths for each OS.
Guess which one is used in practice.
Try it on another file system if you don't believe me.
That's one of the lost opportunities that the original article laments about.
Edit: a reply to dataflow’s "try your operations on another file system" answer: I don't have to, I remember: there was a time when people used FAT formatted media much more than today -- and NTFS was slower than FAT too, under the same Windows, on the same hardware, without installing anything from third parties. It can be that on Windows 10 FAT got to be comparably slow, but in earlier times, on slower machines, FAT was visibly faster compared to NTFS. But NTFS was more robust to failures, and I preferred using NTFS for anything that is not temporary. I admit that the whole Windows infrastructure around what we consider purely "a filesystem" is also a serious part of the problem, by design. And as I wrote, it's a lost opportunity that the bottlenecks weren't identified and the changes made to the benefit of everything on some future Windows.
It's not NTFS that's slow at opening files. It's the I/O subsystem. You'll see the slowness with other file systems too.
You can already run windows on top of BTRFS if you want but it'll be painfully slow compared to linux [1].
https://twitter.com/NTDEV_/status/1327358814891470850 https://github.com/maharmstone/quibble
Ok, probably both on default windows installation due legacy and backward compatibility with dos names [1].
https://docs.microsoft.com/en-us/windows-server/administrati...
I remember this issue when creating few million of files inside one folder and it was extremelly slow because of 8dot3 name creation. It has to go through each filename to generate short name O(n) when this legacy feature is enabled.
After disabling 8dot3 there were no performance issues anymore.
> I have no experience with BTRFS, but that doesn't prove anything without knowing more details (the overhead introduced to make it work)
I tried to point out following: Ext4, Btrfs, Zfs, any other UNIX filesystem will be slow under Windows. NTFS or any Windows first filesystem will be slow on Linux. There is just too much of the differences in OS architecture between the NT and Linux.
This was more about cache coherency, and it's not an argument, it's a trait shared by every other Linux-over-X implementation (UML, coLinux). It is fundamental to solving a problem users want - seamless integration, i.e. no opaque ext4 blob. Why doubt the reasons given by Microsoft, when they match observations not just for WSL but every other system in the same class?
Yes, I've read this comment: https://github.com/microsoft/WSL/issues/873#issuecomment-425...
The argument about Windows not having a single, central dentry cache doesn't hold water. For one thing, there's no reason to think that a two-level cache would significantly decrease performance. More importantly, the fact that on Windows most VFS work occurs in the filesystem driver itself only proves that Windows could have implemented an EXT4 driver with better dentry and other semantics and permitted WSL1 environments to keep most of its files on an EXT4 volume, which is what happens with WSL2, anyhow.
WSL1 could have been better in every way. It would have required them to track a moving target, but they already accomplished semantic parity with seemingly minimal resources. Improving performance was certainly possible, and keeping up would have required significantly less work on its own. At the end of the day, I think the reason WSL1 was canned is obvious: the decision to ship WSL1 was probably accidental. Why accidental? Because anyone with the slightest business acumen would realize that a WSL1 environment with feature and performance parity to open source Linux would mean there'd be little reason to target Windows' native environment at all for new software development. And while Microsoft has done relatively well for itself with Azure and other ventures, its revenue still principally derives from Windows and Office, and the last thing Microsoft needs is to accelerate movement away from those products.
WSL2 means that Microsoft can keep its Linux environment a second-class citizen without being blamed for it, while still providing all the convenience necessary for doing Linux server application development locally. Integration with the native Windows environment will be handicapped by design and excused as an insurmountable consequence of a VM architecture. For example, with WSL1 and some obvious and straight-forward improvements it would have been trivial to run a Linux-built Electron app with identical performance, responsiveness, and behavior (not to mention look & feel, given the web-like UI). With WSL2 Microsoft will likely only ever provide GUI integration over RDP, which will never have the same responsiveness (not because it's theoretically impossible, but because nobody would ever demand it, and in any event it would require tremendously more effort than the comparable WSL1 approach).