News for hackers. Stuff that matters.
Re:I tested four NVMe SSDs from four vendors – half lose FLU... by shadowgovt on Monday February 21, 2022 @04:37PM (#30420942)
As a frame of reference, how much loss of FLUSH'd data should be expected on power loss for a semi-permanent storage device (including spinning-platter hard drives, if anyone still installs them in machines these days)?

I'm far more used to the mainframe space where the rule is "Expect no storage reliability; redundancy and checksums or you didn't want that data anyway" and even long-term data is often just stored in RAM (and then periodically cold-storage'd to tape). I've lost sight of what expected practice is for desktop / laptop stuff anymore.

Re:I tested four NVMe SSDs from four vendors – half lose FLU... by xenadu02 on Monday February 21, 2022 @04:43PM (#30421010)
The semantics of a FLUSH command (per NVMe spec) is that all previously sent write commands along with any internal metadata must be written to durable storage before returning success.

Basically the drive is saying "yup, it's all on NAND - not in some internal buffer. You can power off or whatever you want, nothing will be lost".

Some drives are doing work in response to that FLUSH but still lose data on power loss.

Re:I tested four NVMe SSDs from four vendors – half lose FLU... by benlwalker on Monday February 21, 2022 @08:30PM (#30423043)
A flush command only guarantees, upon completion, that all writes COMPLETED prior to submission of the flush are non-volatile. Not all previously sent writes. NVMe base specification 2.0b section 7.1.

That's a very important distinction. You can't assume just because a write completed before the flush that it's actually durable. Only if it completed before you sent the flush.

I'm not very confident that software is actually getting this right all that often, although it probably is in this fsync test.

Re:I tested four NVMe SSDs from four vendors – half lose FLU... by wmf on Monday February 21, 2022 @10:05PM (#30423596)
Is there a separate barrier command so you don't have to track all the writes individually in software?
Re:I tested four NVMe SSDs from four vendors – half lose FLU... by benlwalker on Monday February 21, 2022 @10:20PM (#30423661)
Nope. The reason this is so complex is that these devices are actually highly parallel machines with multiple queues accepting commands. It's quite difficult to even define "before" in terms of command sequence. For example, if you have a device with two hardware queues for submitting commands and a software thread for each, if you submit a flush on one queue, which commands on the other queue does it affect?

Or what if the device issues a pci write to the completion entry that passes a flush command being submitted on the wire?

I think the only interpretation that makes sense is from the perspective of a single software thread. If that particular thread has seen the completion via any mechanism and then that thread issues the flush, then you know the write is durable. Other than that, the device makes no promises.

Re:I tested four NVMe SSDs from four vendors – half lose FLU... by dathinab on Monday February 21, 2022 @05:08PM (#30421298)
> how much loss of FLUSH'd data should be expected on power loss for

0%

In enterprise you are expected to expect lost data, but only if your drive fails and needs to be replaced, or if it's not yet flushed.

Re:I tested four NVMe SSDs from four vendors – half lose FLU... by colanderman on Monday February 21, 2022 @04:43PM (#30421012)
None. If the drive responds that the data has been written, it is expected to be there after a power failure.