Re:NTFS Corruption on Win10
by codetrotter on Friday January 15, 2021 @03:11AM
(#25788254)
Different bug AFAICT. The Twitter thread is talking about corruption happening from trying to open some path. Your link talks about trying to show a specific icon leading to corruption.
Re:NTFS Corruption on Win10
by josteink on Friday January 15, 2021 @03:13AM
(#25788257)
Moderators: Twitter is a terrible medium for long-format "articles". Can the main link be updated to this one, instead of the twitter-link?
Also: With twitter starting banning random accounts, like sci-hub, seemingly without any consistent justifications, shouldn't we reconsider twitter as valid source for HN news-stories?
Edit: seemingly different bug. Twitter is still all over terrible though.
Re:NTFS Corruption on Win10
by SkitterDotOrg on Friday January 15, 2021 @05:42AM
(#25789258)
Only problem with nitter.net is that 'this instance is over loaded' and so on. Nitter.net is running on some hetzner low instance and at least since beginning of 2021 nitter has said ' this instance is over loaded ' half the time you use nitter. So it goes
Re:NTFS Corruption on Win10
by jabberwcky on Friday January 15, 2021 @06:07AM
(#25789457)
Counter case: Twitter forces authors to be succinct and to the point, and encourages them write in a style similar to the classical newspaper article structure where the most pertinent information is presented first
There is a wide variety of research published only on Twitter, precisely because the publishing process is so lightweight for writers, and every single paragraph can form the leaves of a single giant conversation. A high quality Twitter thread cannot easily be replicated using alternative media
Re:NTFS Corruption on Win10
by kevincox on Friday January 15, 2021 @09:36AM
(#25791396)
You raise a good point, but an email on a mailing list lets you reply to specific sentences/paragraphs. I guess the fairly unique value that twitter is adding here is going from the specific paragraph to see if there are any replies.
Maybe webmention needs the ability to reference parts of a page, although I guess you could do that by linking to an anchor.
Re:NTFS Corruption on Win10
by alexvoda on Tuesday January 19, 2021 @04:52AM
(#25831703)
Commenting per paragraph has also been done by Medium. I'm not saying that Medium is good, it has its own problems. But that feature can be replicated outside Twitter/Mastodon/etc.
Re:NTFS Corruption on Win10
by uncledave on Friday January 15, 2021 @02:58AM
(#25788171)
I don’t think this is actually a corruption issue. I spent a lot of time with this yesterday analysing it in a VM and I couldn’t find any cases where on reboot it caused chkdsk to actually take any actions. It always found the file system clean.
What it seems to do is trigger a consistency check for the next reboot and some alarmist notifications. This is probably a simple check assertion somewhere which is seeing unexpected data passing through a call.
That is a problem but significantly less of one than actual corruption.
If anyone has evidence to the contrary and steps to reproduce it I’d be interested to try them.
Re:NTFS Corruption on Win10
by shakna on Friday January 15, 2021 @03:56AM
(#25788549)
> I spent a lot of time with this yesterday analysing it in a VM and I couldn’t find any cases where on reboot it caused chkdsk to actually take any actions. It always found the file system clean.
That is very frequently the case with these kinds of filesystem bugs. You need actual hardware, not a VM. FWIW, it does seem to be reproducible on hardware.
Re:NTFS Corruption on Win10
by rndgermandude on Friday January 15, 2021 @05:05AM
(#25789021)
I don't see why this bug would work any different in a VM than on a real bare-metal OS or disk?
It's an NTFS issue, i.e. a software issue. NTFS doesn't care if the underlying disk is virtual or bare metal. In fact, it kinda runs on virtual disks already even on bare metal, i.e. disk partitions (maybe even with a layer of encryption in between). There could be potential issues of course where you could get NTFS to trigger some hardware bugs, but that doesn't seem to be the case here, instead it is about accessing NTFS $I30 attributes.
Re:NTFS Corruption on Win10
by steerablesafe on Friday January 15, 2021 @05:31AM
(#25789175)
I'm not a filesystem developer, but assuming the corruption is caused by a race condition in the filesystem driver the difference can come down to many aspects:
* Different timings accessing a raw device compared to a virtual drive.
* The presence of the host scheduler can also alter timings.
* CPU configuration on the guest that's not typical on a bare metal (single CPU VM?)
It's not at all obvious that a bug must manifest the same way on a VM than on bare metal, although probably more often than not that's the case. If filesystem testing is also done on VMs then that would explain how this slipped through QC.
Re:NTFS Corruption on Win10
by rfoo on Friday January 15, 2021 @08:28AM
(#25790691)
Sure, but there are no corruption at all.
The bug is, if you write garbage data into a file "name" similar to those holding filesystem internal metadata, the filesystem driver detects this and mistakenly thinks the file could be the real metadata, so it does a consistency check on it, and because it fails consistency check, it sets a "volume needs fsck" flag. Windows (after 18H2) then show an alert. The alert is extremely confusing (it literally says "disk corruption") to the user, hence all the buzz.
Re:NTFS Corruption on Win10
by stallman_is_ok on Friday January 15, 2021 @06:16AM
(#25789516)
I believe you are correct. I could not see this cause filesystem corruption. From looking around in ntfs.sys, it just seem to set a 'this is corrupt' flags on that code path.
The article suggests a .url file could be found empty after the reboot. Perhaps the check assertion causes the filesystem to become read-only or discard the write cache or some other side effects that could lead to corruption?
If it's something along these lines, it may not be reproducible in a VM (where the host is responsible for waiting around for flushing delays)
Re:NTFS Corruption on Win10
by uncledave on Friday January 15, 2021 @03:46AM
(#25788484)
That’s certainly a possibility. Will go and try it. The MFT writes are transactional in theory from my understanding and small files are stored in the MFT and shortcuts are small files.
Could be that the MFT is not flushed to disk after that event? That would suggest anything after the event would be broken. Also in line with my worries that this should be a bugcheck/panic rather than a soft assertion.
Re:NTFS Corruption on Win10
by uncledave on Friday January 15, 2021 @03:11AM
(#25788251)
I ran the test case and then chkdsk 5 times each on 1903, 2004 and 20H2 and it came back clean every time. It’s definitely possible I missed something but until it’s consistently reproducible then I’m on the fence.
One complaint is I’d expect something at the level of MFT corruption to trigger a bugcheck (panic). Doesn’t seem safe behaviour to let the system carry on with a corrupt MFT.
Re:NTFS Corruption on Win10
by TonyTrapp on Friday January 15, 2021 @06:58AM
(#25789857)
As long as you don't download anything, it should be fine? The last time I checked, browser caches don't use original filenames for storing cached objects on disk (like IE did in the old days).
Re:NTFS Corruption on Win10
by 0x0 on Friday January 15, 2021 @03:06AM
(#25788223)
Does this affect windows-based web/file servers too? If a client can request an URL or similar with the pattern triggering a folder lookup on the server's NTFS volume?
Re:NTFS Corruption on Win10
by freeone3000 on Friday January 15, 2021 @11:43AM
(#25792796)
Yes, but you wouldn't normally be able to access this file as a web client. Being able to view arbitrarily files as a web client is already a security concern.
Re:NTFS Corruption on Win10
by pdenton on Friday January 15, 2021 @03:17AM
(#25788284)
The article says the path to the icon is cleared after a reboot, but what if it tries to access the icon first and fails to actually write the path to the shortcut (given that the filesystem is already hosed at that time). I wonder what happens if one creates the shortcut offline, e.g. using older MS-Windows, wine, reactOS or whatever. Then just copy the crafted lnk file to \Users\Public\Desktop and see what happens.
The stuff about icons is just using a filetype to trigger the bug quickly by forcing windows to look up the path embedded in the icon.
Also: With twitter starting banning random accounts, like sci-hub, seemingly without any consistent justifications, shouldn't we reconsider twitter as valid source for HN news-stories?
Edit: seemingly different bug. Twitter is still all over terrible though.
There is a wide variety of research published only on Twitter, precisely because the publishing process is so lightweight for writers, and every single paragraph can form the leaves of a single giant conversation. A high quality Twitter thread cannot easily be replicated using alternative media
Maybe webmention needs the ability to reference parts of a page, although I guess you could do that by linking to an anchor.
What it seems to do is trigger a consistency check for the next reboot and some alarmist notifications. This is probably a simple check assertion somewhere which is seeing unexpected data passing through a call.
That is a problem but significantly less of one than actual corruption.
If anyone has evidence to the contrary and steps to reproduce it I’d be interested to try them.
That is very frequently the case with these kinds of filesystem bugs. You need actual hardware, not a VM. FWIW, it does seem to be reproducible on hardware.
It's an NTFS issue, i.e. a software issue. NTFS doesn't care if the underlying disk is virtual or bare metal. In fact, it kinda runs on virtual disks already even on bare metal, i.e. disk partitions (maybe even with a layer of encryption in between). There could be potential issues of course where you could get NTFS to trigger some hardware bugs, but that doesn't seem to be the case here, instead it is about accessing NTFS $I30 attributes.
* Different timings accessing a raw device compared to a virtual drive.
* The presence of the host scheduler can also alter timings.
* CPU configuration on the guest that's not typical on a bare metal (single CPU VM?)
It's not at all obvious that a bug must manifest the same way on a VM than on bare metal, although probably more often than not that's the case. If filesystem testing is also done on VMs then that would explain how this slipped through QC.
The bug is, if you write garbage data into a file "name" similar to those holding filesystem internal metadata, the filesystem driver detects this and mistakenly thinks the file could be the real metadata, so it does a consistency check on it, and because it fails consistency check, it sets a "volume needs fsck" flag. Windows (after 18H2) then show an alert. The alert is extremely confusing (it literally says "disk corruption") to the user, hence all the buzz.
This seems to be a deterministic bug tho. Open "special" file path to an $I30 attribute stream, crash and burn.
This was a single test so the veracity of the outcome is somewhat debatable.
If it's something along these lines, it may not be reproducible in a VM (where the host is responsible for waiting around for flushing delays)
Could be that the MFT is not flushed to disk after that event? That would suggest anything after the event would be broken. Also in line with my worries that this should be a bugcheck/panic rather than a soft assertion.
Tested it three times. Two of the tests, chkdsk ran and Windows booted after.
Third time, after chkdsk it went into Windows startup repair, and ultimately led to a green screen where it wouldn't launch boot further.
One complaint is I’d expect something at the level of MFT corruption to trigger a bugcheck (panic). Doesn’t seem safe behaviour to let the system carry on with a corrupt MFT.
Also, it presents one of the rare cases where running chkdsk might actually do something useful.