News for hackers. Stuff that matters.
Re:Is sequential IO dead in the era of the NVMe drive? by colordrops on Tuesday May 09, 2023 @06:30PM (#35881185)
Looks like that's the case based on a cursory google search. Seems analogous to only charging batteries up to 80%. I wish companies would factor these issues into their products and leave invisible buffers, but I guess making them cheaper is more important than being consistent and reliable.
Re:Is sequential IO dead in the era of the NVMe drive? by wtallis on Tuesday May 09, 2023 @06:42PM (#35881295)
They do leave invisible buffers, commonly referred to as over provisioning. The discrepancy between GB and GiB is ~7% hiding in plain sight, and then more expensive enterprise drives are commonly sold in (usable) capacities like 960GB and 7.68TB rather than 1TB and 8TB.
Re:Is sequential IO dead in the era of the NVMe drive? by andrecarini on Tuesday May 09, 2023 @07:28PM (#35881695)
1 TB = 930 GiB

The value difference is not because of invisible buffers! Marketing material and usable OS space are measured in different units, but the usable amount of bits in each value is exactly the same.

Re:Is sequential IO dead in the era of the NVMe drive? by wtallis on Tuesday May 09, 2023 @07:37PM (#35881750)
No, there really are invisible buffers resulting from this discrepancy. The raw flash chips have nominal capacities that align with power of two sizes, but the drives use decimal-based units. So a drive advertised as 1TB will have a usable capacity of approximately 1TB, but is assembled from flash chips with a total capacity of at least 1 TiB.
Re:Is sequential IO dead in the era of the NVMe drive? by cfinnberg on Wednesday May 10, 2023 @01:12PM (#35890300)
Yes, there are invisible buffers, but the discrepancy doesn't come from this. Drive makers count the space in 1000 -multiple units (i.e. 1GB = 1000*1000*1000 bytes) whereas in computing units are usually count on 1024 -multiple (i.e. 1GiB = 1024*1024*1024 bytes)

So 1TB = 1000*1000*1000*1000 bytes/1024/1024/1024 = 931GiB (or 0.9TiB)

Re:Is sequential IO dead in the era of the NVMe drive? by wtallis on Wednesday May 10, 2023 @05:57PM (#35894053)
Did you notice the part where I explained how drive makers use the decimal units but the chips they source to build those drives are built in binary capacities?

I don't give a shit whether your operating system likes to show you disk usage in binary or decimal units, because I'm not talking about software at all. I'm explaining how the hardware is built. I especially don't need another person to try to explain what the binary and decimal units are, after I've repeatedly used both correctly.

Re:Is sequential IO dead in the era of the NVMe drive? by Osiris on Tuesday May 09, 2023 @09:50PM (#35882665)
Some flash based drives will even report this value in SMART, the amount of the over-provisioned space that has been used.

Modern QLC has very low write endurance so those drives need to have spare space to use when it starts to wear out.

Re:Is sequential IO dead in the era of the NVMe drive? by wtallis on Tuesday May 09, 2023 @11:29PM (#35883297)
I think there are two things you may be referring to. First, some drives support thin provisioning of namespaces, and can report the current amount of storage actually being used to back the namespace(s), relative to the maximum user-accessible capacity.

Second, in the SMART/drive health data, there's usually a counter tracking how many reserve/spare blocks remain usable as in not worn out and retired. That's a one-way counter; deleting data won't un-retire defective or worn-out blocks.

It's almost unheard-of for a drive to directly expose a realtime counter of how many blocks are currently unallocated, not being used to store data, and in the erased state making them available to accept new writes.

Re:Is sequential IO dead in the era of the NVMe drive? by Waterluvian on Tuesday May 09, 2023 @06:35PM (#35881235)
Sales and marketing will never leave anything on the table that might make their job easier.
Re:Is sequential IO dead in the era of the NVMe drive? by wongarsu on Wednesday May 10, 2023 @03:14AM (#35884613)
Most consumers make purchasing decisions against these buffers.

If you have two identical electric cars for the same price, one with a range of 350 miles and one with a range of 280 miles, which one would you buy?

If you have two identical phones, one with a 4300 mAh rated battery and another with a 3400 mAh battery, which one would you buy?

If you have two equally priced SSDs, one with 500GB capacity and one with 480GB, which one would you buy?

To enterprise customers the sales rep will explain "look, this device has a bit less space, but the write endurance number quoted here is much better, over 5 years and 5000 drives this will lower your TCO by X". In the consumer space you build your entire brand around one price-reliability tradeoff and stick to it, and usually the cheaper one with the bigger headline stat sells the most.