News for hackers. Stuff that matters.
Re:Windows NT vs. Unix: A design comparison by nullindividual on Monday September 09, 2024 @04:00PM (#41492906)
The reason is due to file system filters, of which Windows Defender is always there. There is a significant delay from Defender when performing CloseFile()[0].

> As I was looking at the raw system calls related to I/O, something immediately popped out: CloseFile() operations were frequently taking 1-5 milliseconds whereas other operations like opening, reading, and writing files only took 1-5 microseconds. That's a 1000x difference!

This is why DevDrive was introduced[1]. You can either have Defender operate in async mode (default) or remove it entirely from the volume at your own risk.

The performance issue isn't related to sync or async I/O.

[0] https://gregoryszorc.com/blog/2015/10/22/append-i/o-performa...

[1] https://devblogs.microsoft.com/visualstudio/devdrive/

Re:Windows NT vs. Unix: A design comparison by cyberax on Monday September 09, 2024 @09:05PM (#41496132)
Windows FS stack is still _way_ slower than Linux. Filesystem operations have to create IRPs and submit them for execution through a generic mechanism. These IRPs can get filtered and modified in-flight, providing quite a bit of overall flexibility.

In Linux, filesystem paths are super-optimized, with all the filtering (e.g. for SELinux) special-cased if needed.

But even still, Windows also had to cheat to avoid completely cratering the performance, there's a shortcut called "FastIO": https://learn.microsoft.com/en-us/windows-hardware/drivers/i...

I wrote a filesystem for Windows around 25 years ago, and I still remember how I implemented all the required prototypes and everything in Explorer worked. But notepad.exe was just showing me empty data. It took me several days to find a note tucked into MSDN that you need to implement FastIO for memory mapped files to work (which Notepad.exe used).

Re:Windows NT vs. Unix: A design comparison by SoothingSorbet on Tuesday September 10, 2024 @04:18AM (#41498454)
That's interesting, why would notepad.exe use mmapped files?
Re:Windows NT vs. Unix: A design comparison by cyberax on Tuesday September 10, 2024 @04:26PM (#41505130)
Probably to save space when opening large files?
Re:Windows NT vs. Unix: A design comparison by nullindividual on Tuesday September 10, 2024 @10:08AM (#41500943)
But it simply isn't slower than Linux.

Robert Collins explains that performance is just as good as Linux and the performance loss on Windows is due to file system filters (Defender)[0].

This is what DevDrive intends (and does) fix.

[0] https://youtu.be/qbKGw8MQ0i8?t=1759

Re:Windows NT vs. Unix: A design comparison by cyberax on Tuesday September 10, 2024 @04:32PM (#41505198)
The last time I did FS tests was admittedly around 4 years ago, but back then Windows was several times slower on pure FS performance benchmarks (creating/listing/deleting large number of directories and files).

It used to be _much_ slower, like orders of magnitude slower, especially for directories with a large number of files.

Re:Windows NT vs. Unix: A design comparison by nullindividual on Tuesday September 10, 2024 @05:21PM (#41505715)
You're not seeing "pure" FS performance. You're seeing all of the abstractions between you and the file system.

To get more of the abstractions out of the way, you want DevDrive. And don't use Explorer.exe as a test bed which has shims and hooks and god knows what else.

Re:Windows NT vs. Unix: A design comparison by cyberax on Tuesday September 10, 2024 @07:04PM (#41506534)
I was testing the performance using plain C++ code.
Re:Windows NT vs. Unix: A design comparison by lproven on Wednesday September 11, 2024 @01:29PM (#41513486)
These days there is a full native Btrfs implementation for Windows.

https://github.com/maharmstone/btrfs

It's complete enough you can boot and run Windows from Btrfs.

https://lilysthings.org/blog/windows-on-btrfs/

I wonder how many of these limitations affect this, notably performance.

If WSL1 could natively talk to Btrfs without the performance-sapping translation to NTFS, would that resolve the poor Git performance etc?