News for hackers. Stuff that matters.
Re:63 Cores Blocked by Seven Instructions by snagglegaggle on Monday October 21, 2019 @10:53AM (#21312928)
> Marking the temporary files as FILE_ATTRIBUTE_TEMPORARY could improve things, without having to go into significant Windows kernel changes.

Having used Cygwin and MinGW (and less so WSL) NTFS is probably the main factor here. Especially when using Cygwin program compilation is very slow not just due to process creation but file access, potentially on disk.

I could see checkpointing contributing, but having used backups/file history on Windows server, you will see CPU use irregularities that I think are similar in cause to this but you do not see CPU use irregularities as bad as described in the article.

The pathological behavior of NTFS with many files is easy to prove once you encounter it.[0] This would be exposed to the kernel as well and is likely holding up NtfsCheckpointVolume. At least in my experience the problem goes deeper, and how NTFS structures are handled also contributes; for example, trying to enumerate and copy files in certain ways is extremely slow even if the files are easily enumerable.

You can say that if disabling checkpointing gets rid of noticeable slowdown it is the right thing to do, but there are people who will rightly not want to disable it.

[0] https://stackoverflow.com/questions/197162/ntfs-performance-...

Re:63 Cores Blocked by Seven Instructions by pault on Monday October 21, 2019 @02:57PM (#21315896)
Time the difference between an npm install, with all its thousands of tiny files, on NTFS and ext4. It's excruciating.
Re:63 Cores Blocked by Seven Instructions by cpach on Monday October 21, 2019 @04:25PM (#21316896)
Just copying Emacs for Windows from one folder to another takes surprisingly long time :)
Re:63 Cores Blocked by Seven Instructions by Filligree on Monday October 21, 2019 @04:45PM (#21317130)
I've definitely noticed this behaviour. It's excruciating.

There's a ZFS driver for Windows now. I haven't tried it yet, but I think I will. Can't be any worse.