WRT NTFS I don't think they can fix it without releasing another major version of it and that comes with some insane level of risk based on the early churn in NTFS. I don't have an appetite for that. They tried to fork it with ReFS but that went pretty much nowhere.

Powershell is amazingly consistent for sure but each little bit of consistency has numerous inconsistencies in it which aren't exactly stable, hence the whole web request nightmare. Also things like different methods of getting the date return timezone and non timezone aware things. They nearly had it but rushed it out of the door. Also things like WinRM are totally unreliable and don't scale well due to the runtime performance being dire (try arguing with ansible/WinRM failures on a daily basis).

I think a lot of people using object pipelines miss the major point of designing test pipelines which is the first step is to make your pipeline easily parseable. So for example dates as unix dates, colon separated etc. Then you don't need to use regex and stuff. For that little bit of compromise first you end up with something which is several orders of magnitude faster than the UTF-16 and serialisation backed powershell pipelines. Try grepping a few gig of data with it :)

On your last point, you're lucky. This comes up a lot in corporate networks and proxies and occasionally randomly out of the blue.

> I think a lot of people using object pipelines miss the major point of designing test pipelines which is the first step is to make your pipeline easily parseable.

And if developers of UNIX tools actually did that then maybe object pipelines wouldn't be nearly as appealing as they are, but we don't live in that timeline.

I use jq a lot in bash. That's a good part of an object pipeline.

curl | jq | grep/awk | jq | curl

Yeah but ip, and ls, and cat /etc/somefile.conf, and pretty much anything asides from curl don't output structured data so you can't use jq on it.

That was just an example of a pipeline. jq can parse plain text easily with split/map as well.

Also:

ls -l

ip -o

I know about `ls -l` (and have for about 25 years now, thanks). You're still scraping.

Have you seen this comment from a Microsoft employee explaining why NTFS performance is so bad? It's pretty interesting and informative, but doesn't leave me with much hope that things will ever improve.

https://github.com/Microsoft/WSL/issues/873#issuecomment-425...

I use Windows for development at work (semi-voluntarily - our Windows support was getting neglected because most developers choose to use run macOS or Linux), and I've been getting pretty fed up with how slow it is. And it's not just filesystem performance; process creation and terminal output are horrendously slow as well.

The only redeeming thing about development on Windows is that Linux doesn't have any debuggers that are as good/usable as Visual Studio.

I’ve seen that and it misses the mark by a mile because the main performance constraint is MFT contention.

You can see this because the tools suck just as bad on native windows as on WSL without that in between.

Check out 10,000 small files on (1) NT native SVN, (2) WSL on Windows, (3) Linux on ext4.

1 and 2 are within 10% of the time delta. 3 is 10x faster at average.

At this point I reckon they won’t fix NT but close the gap on NT and forget to mention the MFT problem.

I’ve dealt with this issue going back 20 years since NT4 for ref.

Very interesting post - thanks for sharing it.

Filter drivers and IRPs really are the worst architectural blunder MS made in Windows. They took what really should be a corner-case (driver interposition) and put it on the development path of every hardware driver programmer to trip over. We got KMDF eventually, but that took years...