News for hackers. Stuff that matters.
Re:Linux: Ext4 data corruption in 6.1.64-1 by Animats on Sunday December 10, 2023 @01:45AM (#38589658)
"Causes non-serious data loss". What does that mean. Only affects cat videos?
Re:Linux: Ext4 data corruption in 6.1.64-1 by Grimburger on Sunday December 10, 2023 @04:12AM (#38590281)
Most data is recoverable albeit slow to do. It takes some very bad conditions or intentional actions to make data recovery impossible. Current high level standards for military/diplomatic data sanatisation are complete physical destruction of hard disks.

Even linux tools like shred have given up saying they can actually delete data from disks due to how SSD's work these days.

Re:Linux: Ext4 data corruption in 6.1.64-1 by eh8 on Sunday December 10, 2023 @04:20AM (#38590310)
> Even linux tools like shred have given up saying they can actually delete data from disks due to how SSD's work these days.

Which emphasizes the importance of enabling full disk encryption immediately whenever you start using a new device--BitLocker if you're on Windows, FileVault on macOS, LUKS on Linux, etc. Trying to decrypt data is much harder than reconstructing deleted data on a stolen drive.

Re:Linux: Ext4 data corruption in 6.1.64-1 by Grimburger on Sunday December 10, 2023 @04:32AM (#38590349)
True, properly zeroing out the headers on an encrypted drive will make recovery impossible.

How to do that reliably is another question.

Re:Linux: Ext4 data corruption in 6.1.64-1 by WirelessGigabit on Sunday December 10, 2023 @02:14PM (#38594006)
You cannot with normal tools as writing a 0 to the SSD does not guarantee it overwrites the 0 you want. At best it does, at worst it writes the 0 somewhere else and remaps the bit (or whatever its physical storage thing is).
Re:Linux: Ext4 data corruption in 6.1.64-1 by WirelessGigabit on Sunday December 10, 2023 @02:16PM (#38594030)
This is especially a problem on macOS:

https://support.apple.com/guide/disk-utility/erase-and-refor...

> Note: With a solid-state drive (SSD), secure erase options are not available in Disk Utility. For more security, consider turning on FileVault encryption when you start using your SSD drive.

So if you set up a Mac without FileVault you can never erase everything.

At least with my Lenovo I can do the secure erase.

Re:Linux: Ext4 data corruption in 6.1.64-1 by Roark66 on Sunday December 10, 2023 @04:28AM (#38590333)
SSDs are not magic. For shredding to be impossible the drives would have to have hidden capacity (and a lot of it, more than 1%). People often say, write levelling makes overwriting useless, but what if you overwrite the entire disk? How would you recover after something like this?

Of course providing something like 10% of hidden extra capacity would extend the life of the drive significantly, but are manufacturers really doing that and are not mentioning it in their marketing materials? I never heard that they do that.

Re:Linux: Ext4 data corruption in 6.1.64-1 by Grimburger on Sunday December 10, 2023 @04:44AM (#38590408)
Sure, but very few business tools are doing that, they are playing the old Gutmann game which he himself has walked away from as not reliable anymore. Filling up the drive is certainly getting you somewhere compared to some of the snake-oil out there. Drives are only getting bigger though and rarely filled up, done some of this stuff in my time and if cost is no barrier then you'd be surprised how much can be reclaimed. Degauss and destroy is the only real method we have now and that's probably not changing in future.

>but are manufacturers really doing that and are not mentioning it in their marketing materials

SSD manufacturers have been caught out repeatedly with their in-built deletion API claims. Recovery of a significant % of files is possible nearly always without tampering having occurred.

Re:Linux: Ext4 data corruption in 6.1.64-1 by colechristensen on Sunday December 10, 2023 @04:55AM (#38590470)
> Of course providing something like 10% of hidden extra capacity would extend the life of the drive significantly, but are manufacturers really doing that and are not mentioning it in their marketing materials? I never heard that they do that.

Every SSD has considerably more physical blocks than reported blocks. They have to, SSD bad blocks are common and number of writes is quite limited compared to hard disks.

https://download.semiconductor.samsung.com/resources/others/...

Re:Linux: Ext4 data corruption in 6.1.64-1 by justinclift on Sunday December 10, 2023 @08:18AM (#38591343)
> I never heard that they do that.

Not sure how you missed it, as it's not a new thing. Straight from the horses mouth, if that helps: :)

https://www.kingston.com/en/blog/pc-performance/overprovisio...

Note that it's not just a Kingston thing, it's common across SSD manufacturers. Or at least with the major ones.

Re:Linux: Ext4 data corruption in 6.1.64-1 by LoganDark on Sunday December 10, 2023 @10:19AM (#38592076)
SSDs don't cycle blocks based on how much you've written. You can write over the entire reported capacity of a drive 100 times and there's still the chance, however small, that your encryption key and some encrypted data are still sitting around on some chip. You just can't guarantee that it's gone. For all you know, the controller has marked them as bad and isn't going to reuse them, even though there's still recoverable data on them. Since there's no way to reach them through the drive interface, there's also no way to erase them. But if someone opened up the drive and pulled the chips, there they would be.
Re:Linux: Ext4 data corruption in 6.1.64-1 by matheusmoreira on Sunday December 10, 2023 @05:37PM (#38595755)
> what if you overwrite the entire disk

People who want secure deletion generally want their storage device to remain functional afterwards. Same reason they don't simply destroy the disk to get rid of its contents: those things are expensive. Overwriting the entire SSD every time you want a file gone just isn't a solution.

Re:Linux: Ext4 data corruption in 6.1.64-1 by 15457345234 on Sunday December 10, 2023 @04:48AM (#38590423)
Bot

That reply has nothing to do with the context of the discussion

This bug involves data never being written to the disk due to a filesystem write bug. You are just replying in a general context in a way that a LLM or similar would.

Re:Linux: Ext4 data corruption in 6.1.64-1 by Grimburger on Sunday December 10, 2023 @07:11AM (#38591016)
I'm sorry what? You are calling me non-human here. A quick glance will tell that I'm very fallible for both opinions and assertions.
Re:Linux: Ext4 data corruption in 6.1.64-1 by dang on Sunday December 10, 2023 @11:26AM (#38592641)
Could you please stop posting unsubstantive comments and flamebait? You've unfortunately been doing it repeatedly. It's not what this site is for, and destroys what it is for.

If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful.

Re:Linux: Ext4 data corruption in 6.1.64-1 by lpcvoid on Sunday December 10, 2023 @05:38AM (#38590660)
I don't agree - modern NVME drives have secure erasure mechanisms. All data is by default encrypted on-the-fly within the flash memory, and when requesting a secure erase, the controller throws away the key and generates a new one.

https://man.archlinux.org/man/nvme-format.1

Re:Linux: Ext4 data corruption in 6.1.64-1 by throw0101b on Sunday December 10, 2023 @07:26AM (#38591073)
> I don't agree - modern NVME drives have secure erasure mechanisms.

Assuming the firmware doesn't lie about it does:

* https://www.zdnet.com/article/flaws-in-self-encrypting-ssds-...

Re:Linux: Ext4 data corruption in 6.1.64-1 by Grimburger on Sunday December 10, 2023 @07:14AM (#38591022)
What percentage of people on Earth use this in your expert opinion?

Also have you done any work in digital forensics?

Eagerly awaiting your response.

Re:Linux: Ext4 data corruption in 6.1.64-1 by csdvrx on Sunday December 10, 2023 @06:28AM (#38590832)
In "reduce, reuse, recycle", this is the #2 R

Please don't trash physically good drives with a hammer. It's not good for the environment (or the drives!) when you have such a simple technology at your disposal!

Re:Linux: Ext4 data corruption in 6.1.64-1 by viraptor on Sunday December 10, 2023 @02:26AM (#38589806)
I believe the Debian policy for reports says that for corruption you can recover from. Serious data loss would be losing your metadata and not being able to figure out where any of the files are, or corrupting all copies of the disk encryption key.
Re:Linux: Ext4 data corruption in 6.1.64-1 by userbinator on Sunday December 10, 2023 @02:34AM (#38589843)
I think "corruption you can recover from" seems far too vague; you can recover from any corruption with backups.
Re:Linux: Ext4 data corruption in 6.1.64-1 by viraptor on Sunday December 10, 2023 @03:21AM (#38590042)
If you're trying to be padantic and missing forest for the trees... Sure.
Re:Linux: Ext4 data corruption in 6.1.64-1 by piperswe on Sunday December 10, 2023 @02:48AM (#38589905)
The way I'd interpret it is that it's non-serious if fsck can repair it. Not sure if that's accurate though, just the way it sounds to me.
Re:Linux: Ext4 data corruption in 6.1.64-1 by LoganDark on Sunday December 10, 2023 @10:16AM (#38592052)
Sure, and you can recover from any hardware failure with a separate computer. In fact, you can recover from death simply by cloning yourself. What's the issue?

The issue is that some people don't have the time, money, and/or willpower to invest in backups. Some people only have one volume, or one copy of each volume at least, and need to recover their data using only that one volume if something gets corrupted. That's what "corruption you can recover from" means. Using a backup is not recovering from corruption; it's replacing the corruption with the pre-corrupted data. Recovering from corruption involves actually transforming the corrupted data back into non-corrupted data.

Re:Linux: Ext4 data corruption in 6.1.64-1 by remram on Sunday December 10, 2023 @11:20AM (#38592580)
I think you're agreeing with userbinator here.
Re:Linux: Ext4 data corruption in 6.1.64-1 by LoganDark on Sunday December 10, 2023 @11:32AM (#38592695)
Not really. They're arguing that backups count as recovering from corruption, where my argument is that backups don't count because the only thing you're doing is discarding the corruption, not recovering from it.
Re:Linux: Ext4 data corruption in 6.1.64-1 by tremon on Sunday December 10, 2023 @04:34PM (#38595273)
In this case, I would say that backups are not relevant protection. In order to restore something from backup, you would first need to know that a restore from backup is needed, and secondly you need to know what the last-known good copy is.

This is a silent data corruption bug (data is written at the wrong offset, IIUC) with no known method to detect the error condition except to compare the resulting data with a known-good copy. The next backup run will happily accept the corrupted file contents into the backup vault, and unless someone happens to stumble upon the corrupted data, the last-known good copy will eventually be rotated out.

Re:Linux: Ext4 data corruption in 6.1.64-1 by minikomi on Sunday December 10, 2023 @02:15AM (#38589762)
You lose your data but, like, it's just a prank man.
Re:Linux: Ext4 data corruption in 6.1.64-1 by pestatije on Sunday December 10, 2023 @03:01AM (#38589952)
yeah, practical joke for using Linux...have a laugh and deal with it
Re:Linux: Ext4 data corruption in 6.1.64-1 by jbverschoor on Sunday December 10, 2023 @04:51AM (#38590447)
So you’re a dog person?
Re:Linux: Ext4 data corruption in 6.1.64-1 by AnAaaaardvark on Sunday December 10, 2023 @07:18AM (#38591040)
I thought we all were dog people. Woof.
Re:Linux: Ext4 data corruption in 6.1.64-1 by LoganDark on Sunday December 10, 2023 @10:24AM (#38592123)
Some of us are different kinds of animals.
Re:Linux: Ext4 data corruption in 6.1.64-1 by dmvdoug on Sunday December 10, 2023 @11:34AM (#38592724)
Some of us are just dogs.
Re:Linux: Ext4 data corruption in 6.1.64-1 by lproven on Monday December 11, 2023 @07:43AM (#38600128)
On the Internet, nobody need ever know.

P.S. Squirrel!

Re:Linux: Ext4 data corruption in 6.1.64-1 by sedatk on Sunday December 10, 2023 @06:46AM (#38590911)
Reminds me of this exchange from Coupling (UK):

“How is it not serious if she died from it?”

“She made an unsuccessful recovery”

Re:Linux: Ext4 data corruption in 6.1.64-1 by riffraff on Sunday December 10, 2023 @09:30AM (#38591715)
Funny, I also thought of Coupling, except I was thinking of

- how badly can you phrase yes?

[Cuts to other scene]

- No

Such a good show.

Re:Linux: Ext4 data corruption in 6.1.64-1 by tremon on Sunday December 10, 2023 @04:45PM (#38595355)
I think they meant non-catastrophic (no pun intended). From what I can tell, individual files can get corrupted when opened with O_DIRECT, but the filesystem structure will never fail.

From the link in https://news.ycombinator.com/item?id=38591444 :

> file position is not updated after direct IO write and thus [we] direct IO writes are ending in wrong locations effectively corrupting data.

This may be a major problem for hypervisors using ext4-backed disk images. It wouldn't surprise me if qemu opens image files with O_DIRECT by default, but it would surprise me if professional shops were using ext4 as the backing store rather than a provisioning abstraction layer like dm_thin. Still, anyone using kvm on top of ext4 must not be having a good day.