News for hackers. Stuff that matters.
Re:Threadripper 3990X: The Quest To Compile 1B Lines Of C++ ... by wtallis on Wednesday February 03, 2021 @06:17PM (#26020550)
SSDs never shrink the reported capacity in response to defective/worn-out blocks. Instead, they have spare area that is not accessible to the user. The SMART data reported by the drive includes an indicator of how much spare area has been used and how much remains. When the available spare area starts dropping rapidly, you're approaching the drive's end of life.
Re:Threadripper 3990X: The Quest To Compile 1B Lines Of C++ ... by Wowfunhappy on Wednesday February 03, 2021 @07:23PM (#26021079)
I so happened to know this already, but I must say I've always found the approach somewhat weird. Wouldn't it make more sense to give the user all the available space, and then remove capacity slowly as blocks go bad? I guess they think people would be annoyed?

Imagine if we treated batteries like SSDs, not allowing the use of a set amount of capacity so that it can be added back later, when the battery's "real" capacity begins to fall. And then making the battery fail catastrophically when it ran out of "reserve" capacity, instead of letting the customer use what diminished capacity was still available.

Re:Threadripper 3990X: The Quest To Compile 1B Lines Of C++ ... by wtallis on Wednesday February 03, 2021 @08:10PM (#26021375)
Shrinking the usable space on a block device is wildly impractical. The SSD has no awareness of how any specific LBA is being used, no way to communicate with the host system to find out what LBAs are safe to permanently remove. You can't just incrementally delete LBAs from the end of the drive, because important data gets stored there, like the backup GPT and OS recovery partitions. Filesystems also don't really like when they get truncated. Deleting LBAs from the middle of a filesystem would be even more catastrophic. The vast majority of software and operating systems are simply not equipped to treat all block devices as thinly-provisioned. [1]

And SSDs already have all the infrastructure for fully virtualizing the mapping between LBAs and physical addresses, because that's fundamental to their ordinary operation. They also don't all start out with the same capacity; a brand-new SSD already starts out with a non-empty list of bad blocks, usually a few per die.

Even if it were practical to dynamically shrink block devices, it wouldn't be worth the trouble. SSD wear leveling is generally effective. When the drive starts retiring worn out blocks en masse, you can expect most of the "good" blocks to end up in the "bad" column pretty soon. So trying to continue using the drive would mean you'd see the usable capacity rapidly diminish until it reached the inevitable catastrophe of deleting critical data. It makes a lot more sense to stop before that point and make the drive read-only while all the data is still intact and recoverable.

[1] Technically, ATA TRIM/NVMe Deallocate commands mean the host can inform the drive about what LBAs are not currently in use, but that always comes with the expectation that they are still available to be used in the future. NVMe 1.4 added commands like Verify and Get LBA Status that allow the host to query about damaged LBAs, but when the drive indicates data has been unrecoverably corrupted, the host expects to be able to write fresh data to those LBAs and have it stored on media that's still usable. The closest we can get to the kind of mechanism you want is with NVMe Zoned Namespaces, where the drive can mark individual zones as permanently read-only or offline. But that's pretty coarse-grained, and handling it gracefully on the software side is still a challenge.

Re:Threadripper 3990X: The Quest To Compile 1B Lines Of C++ ... by Wowfunhappy on Thursday February 04, 2021 @12:38AM (#26022958)
That makes a lot of sense, thank you for the detailed answer!
Re:Threadripper 3990X: The Quest To Compile 1B Lines Of C++ ... by okr on Wednesday February 03, 2021 @07:48PM (#26021233)
I can imagine, OSes in general are not prepared to such reported numbers shrinking. There would be a hole.

What if a moment ago my OS has still 64G and then all of the sudden it only has 63G. Where would the data go? I think something has to make up for the loss.

For me it makes sense to report logically 64G and internally you do the remapping magic.

I wonder, how some OSes deal with a hot-swap of RAM. You have a big virtual address space and all of the sudden there is no physical memory behind it.

Hm.