News for hackers. Stuff that matters.
Re:Modern HDDs have gotten somewhat better by OskarS on Wednesday June 22, 2022 @01:36PM (#31839177)
It's absurd to claim they're doing this for any other reason beyond deceptive marketing. It's like an advertising version of a dark pattern: a decision which has a sheen of reasonableness, but is clearly designed to deceive and exploit your customers.

Nowhere else in the computer is the base-10 version standard, and it shouldn't be. The sentence "a 32-bit CPU register can address up to 4 gigabyte of memory" is clearly reasonable, the sentence "a 32-bit CPU register can address up to 4.29497... gigabytes of memory" is not. It's not just ram though: all basic data types (bytes, ints, floats, pointers, SIMD registers, whatever) are sized to powers of 2, having a storage unit that is not is insanity. Or, to bring it back to hard drives: what's the max file size of a file stored on a FAT32 filesystem again?

It's just bad and deceptive, and hard drive manufactures should not be allowed to play these kinds of games. You should stop carrying the water for their shitty practices.

Re:Modern HDDs have gotten somewhat better by adwn on Wednesday June 22, 2022 @02:51PM (#31840204)
> Nowhere else in the computer is the base-10 version standard

Well, you're simply wrong. CPU frequency is measured in GHz, that is 1,000,000,000 Hz, not 1,073,741,824 Hz. Consequently, my CPU has a nominal frequency of 3.0 GHz, not 2.793967724 GHz. Since frequencies are denominated in powers of 10, bandwidths are as well: The L1 cache in my CPU can read 64 bytes/cycle (I think), so it has a read-bandwidth of 192 GB/s, that is 192,000,000,000 B/s. My Ethernet device can transfer 1 Gbit/s, that is 1,000,000,000 bit/s, because it's clocked at 125 MHz (that's 125,000,000 Hz) and transfers 8 bit per clock cycle.

> all basic data types (bytes, ints, floats, pointers, SIMD registers, whatever) are sized to powers of 2

Yeah, and it pretty much ends with basic data types. Most of my structs aren't powers of two in size, even though their individual elements are 1, 2, 4, or 8 bytes in size. Almost none of my files have sizes related to powers of two. My SSD has a size of 931.51339 GiB – so tell me again how powers of two are more correct or more useful here?

Re:Modern HDDs have gotten somewhat better by colanderman on Wednesday June 22, 2022 @03:07PM (#31840436)
> Almost none of my files have sizes related to powers of two.

They are all rounded up to 512 or 4096 bytes by your filesystem.

> My SSD

SSDs operate using erase blocks sized in mebibytes, and expose sector sizes of 512 B or 4096 B to the OS. [1] HDDs a similarly low-level-formatted with 512 or 4096 B sector sizes.

[1] https://spdk.io/doc/ssd_internals.html

Re:Modern HDDs have gotten somewhat better by dspillett on Wednesday June 22, 2022 @08:44PM (#31843786)
> They are all rounded up to 512 or 4096 bytes by your filesystem.

Not always. Some filesystems allow small files to be compacted into other structures: the MTF can hold small in NTFS, ext* and btrfs support similar inlining of data in inodes for small files, other filesystems (ZFS, Reiser, btrfs again) support more general sub-block allocation (with techniques such as tail packing or block sub-division). Sometimes this is only for small files not “oddly” sized larger files, but things like tail packing can apply to the latter too.

Re:Modern HDDs have gotten somewhat better by adwn on Wednesday June 22, 2022 @03:36PM (#31840802)
> They are all rounded up to 512 or 4096 bytes by your filesystem.

That's an implementation detail, and neither I nor my programs care about it. We do care about the actual length of the file, which isn't related to powers of two, except in niche cases.

> SSDs operate using erase blocks sized in mebibytes, and expose sector sizes of 512 B or 4096 B to the OS.

While true, this is also an implementation detail, and I don't think about the size of my SSD in multiples of sectors or pages or kibibytes – "976762584 kiB" just isn't a useful number for me, but "1000.2 GB" is manageable. Specifying it as "931.51 GiB" isn't more correct, so it makes sense to default to powers of 10 instead of powers of 2.

The situation would be different if HDDs and SSDs always or typically had exact sizes like 512*2^30 (= 512 GiB) or 3*2^40 (= 3 TiB) – you know, like RAM – in which case using powers of 2 would make sense. But that's not the world we're living in.

Re:Modern HDDs have gotten somewhat better by colanderman on Wednesday June 22, 2022 @05:40PM (#31842204)
> The situation would be different if HDDs and SSDs always or typically had exact sizes

I just checked two of three SSDs available to me. Two were exact multiples of power-of-two gigabytes -- 200 GiB, 120 GiB. One was 500.3 GB.

SI units might be more useful to a consumer (and I am not arguing they aren't). But the argument that storage systems have "nothing to do" with powers of two is not valid.

Re:Modern HDDs have gotten somewhat better by qilo on Wednesday June 22, 2022 @06:30PM (#31842686)
HDD sizes are set by manufacturers according to the IDEMA "LBA Count for Disk Drives Standard"[0] formula:

  LBA_count = 97696368 + 1953504 * (Reported_Capacity_in_GBytes - 50)
So, for 120 GB HDD:

  (97696368 + 1953504 * 70) * 512 / 1024^3 = 111.79 GiB
  (97696368 + 1953504 * 70) * 512 / 1000^3 = 120.03 GB
For 200 GB HDD:

  (97696368 + 1953504 * 150) * 512 / 1024^3 = 186.31 GiB
  (97696368 + 1953504 * 150) * 512 / 1000^3 = 200.05 GB
For 500 GB HDD:

  (97696368 + 1953504 * 450) * 512 / 1024^3 = 465.76 GiB
  (97696368 + 1953504 * 450) * 512 / 1000^3 = 500.11 GB
Doesn't look like powers of 2 at all. Are you sure about your sizes?

[0] http://www.idema.org/wp-content/downloads/2169.pdf

Re:Modern HDDs have gotten somewhat better by ThePowerOfFuet on Thursday June 23, 2022 @07:21AM (#31847531)
Logical Block Addresses were simple back when all blocks were 512 bytes. Now we have 520 (T10-PI), 4096, etc, so your math falls apart.
Re:Modern HDDs have gotten somewhat better by qilo on Thursday June 23, 2022 @08:08AM (#31847900)
For disks with sectors of 4096 bytes, the formula is the same except the constants are accordingly 8 times smaller.

Regarding sectors of 520 bytes, usable bytes are still 512. Quote from the above referenced document:

> HDDs formatted with T10 PI (Protection Information) have 8 bytes added to the end of each sector. However, the user usable sector size will remain at 512 bytes or 4096 bytes. The 8 bytes of PI will increase the overhead of the disk sector density similar to overhead of ECC bytes in a given sector.

Re:Modern HDDs have gotten somewhat better by wmf on Wednesday June 22, 2022 @02:44PM (#31840106)
Nowhere else in the computer is the base-10 version standard

As the parent said, networking. (Which also suffers from an unfortunate bits/bytes mismatch.)

The real problem with storage IMO is that software used base-2 measurements and vendors used base-10 and eventually the discrepancy got big enough to be noticeable.

Re:Modern HDDs have gotten somewhat better by Akronymus on Thursday June 23, 2022 @04:53AM (#31846557)
For me, the egregious part isn't even the difference in base 2 and base 10 but rather that windows uses base 2 but displays it as base 10 units. (Internally uses Gibibytes but displays the unit as Gigabytes, for example)
Re:Modern HDDs have gotten somewhat better by tyrfing on Wednesday June 22, 2022 @02:57PM (#31840296)
You have to be very familiar with tech to have any idea about base-2 vs base-10. Universal standards to make computing units understandable would very clearly go with base-10, which is the status quo in units elsewhere (metric). Base-2 is just an implementation detail, and one that the vast majority of the world simply does not care about.
Re:Modern HDDs have gotten somewhat better by WalterBright on Wednesday June 22, 2022 @05:15PM (#31841950)
> very familiar

Only slightly familiar. Binary number systems are explained in introductory first books on computers. I knew binary nearly 10 years before I got my hands on a computer.

Re:Modern HDDs have gotten somewhat better by jopsen on Wednesday June 22, 2022 @06:59PM (#31842984)
Yes, but let's not allow the Americans to "reinvent" the SI-prefixes for computers..

If you want base-2 then use kibiBytes (kiB) and move on..

Re:Modern HDDs have gotten somewhat better by mlyle on Saturday June 25, 2022 @03:33PM (#31878045)
You need to have some domain knowledge to know -- did someone think the power of 2 was important here?

For communications they mostly haven't (we tend to deal with decimal clock rates, bandwidths, etc-- and even when we have 128-QAM or whatever that's going to not result in powers of 2). For memories and caches we generally like them to be powers of 2 to use all the bits of our addressing and decoders well. For secondary storage, ... we tend to have a non-round number of lumps that are a small power of 2 sized, and we might as well use decimal.

This is subtle and has been woefully inconsistent. It would be better to just use powers of 10.

(Me of 20 years ago would be very disappointed in myself for writing this comment).

Re:Modern HDDs have gotten somewhat better by gjvc on Wednesday June 22, 2022 @04:31PM (#31841500)
Maybe not today. But they did in previous decades.
Re:Modern HDDs have gotten somewhat better by robocat on Wednesday June 22, 2022 @07:29PM (#31843249)
Redefining meanings is like defining π = 3.

SI metric prefix standards really really matter to an engineer. The creation of a new indistinguishable unit which varies depending on what was measured (so that 1k != 1k) is mind-bogglingly idiotic in this old engineer’s opinion.

As a child I too felt the pain of feeling “shortchanged” by hardware manufacturers’ marketing practices with hard drive sizes, and I deeply understand the glamour of using 1024 based decimals for declaring total memory. However the cost of differing values for metric prefixes (k, M, G) is confusion, mistakes, and arguments. I also think that any card carrying geek knows their measurements, and cannot be easily deceived. Another argument for why software engineers are not. Disclaimer: last gig was JavaScript.

Re:Modern HDDs have gotten somewhat better by walrus01 on Wednesday June 22, 2022 @02:51PM (#31840208)
I may not be the biggest fan of it ever but it is a fact that this is how hard drives have been measured since the days when a 5.25" full height MFM 10MB or 20MB hard drive in an IBM XT or clone was state of the art technology.

your comment seems to insinuate that this is some new sneaky marketing practice of HDD manufacturers to trick the consumer.

Re:Modern HDDs have gotten somewhat better by fragmede on Wednesday June 22, 2022 @03:00PM (#31840333)
Right. It’s an old sneaky marketing practice of HDD manufacturers to trick consumers, which is still a sneaky marketing trick, just not a new one.
Re:Modern HDDs have gotten somewhat better by kennywinker on Wednesday June 22, 2022 @04:39PM (#31841579)
Give it a little more time and it'll be like 2x4s. At one point they were actually 2" by 4", but now they're 3.5" by 1.5" and that's the standard size and we're stuck with this dumb little detail where the name doesn't match reality and everybody just accepts it - but it confuses virtually every single person on the planet at LEAST once or twice in their life.
Re:Modern HDDs have gotten somewhat better by selectodude on Wednesday June 22, 2022 @04:46PM (#31841664)
2x4s are 2" by 4" before they're finished. If you search S4S (sanded 4 sides) 2x4s, they will actually be 2" by 4". If you get the wood with rounded corners, they take a quarter inch off each side to "finish" it. It's been that way for 100 years.
Re:Modern HDDs have gotten somewhat better by kennywinker on Wednesday June 22, 2022 @06:17PM (#31842555)
I don't think that's right.

> The limited availability of lumber and the rapid pace of housing construction made other methods like concrete-block housing viable. This pressured further compromise because thinner 2x4s were a way to compete in price with wood alternatives. Size standards, maximum moisture content, and nomenclature were agreed upon only as recently as 1964. The nominal 2x4 thus became the actual 1½ x 3½, imperceptibly, a fraction of an inch at a time.

http://www.harvarddesignmagazine.org/issues/45/nominal-versu...

So it seems the reduction was was driven by cost saving, not a product of the finishing process, and it's actually only been standardized that way for 58 years.

Re:Modern HDDs have gotten somewhat better by mritun on Wednesday June 22, 2022 @05:23PM (#31842038)
Sneaky, eh?
Re:Modern HDDs have gotten somewhat better by thedougd on Wednesday June 22, 2022 @06:25PM (#31842629)
https://en.wikipedia.org/wiki/Lumber#Dimensional_lumber

"Dimensional lumber is available in green, unfinished state, and for that kind of lumber, the nominal dimensions are the actual dimensions."

Re:Modern HDDs have gotten somewhat better by kennywinker on Wednesday June 22, 2022 @06:36PM (#31842744)
> For example, a "2×4" board historically started out as a green, rough board actually 2 by 4 inches (51 mm × 102 mm). After drying and planing, it would be smaller by a nonstandard amount. Today, a "2×4" board starts out as something smaller than 2 inches by 4 inches and not specified by standards, and after drying and planing is minimally 1+1⁄2 by 3+1⁄2 inches (38 mm × 89 mm).[8]

Your quote just means when you're buying green wood and it says it's 2"x4" it will be exactly 2"x4". It doesn't mean a 2x4 started out as a 2"x4" green board.

Re:Modern HDDs have gotten somewhat better by thedougd on Saturday June 25, 2022 @12:58PM (#31876819)
> Historically, the nominal dimensions were the size of the green (not dried), rough (unfinished) boards that eventually became smaller finished lumber through drying and planing (to smooth the wood). Today, the standards specify the final finished dimensions and the mill cuts the logs to whatever size it needs to achieve those final dimensions.

I was originally pointing out that the naming is simply a standard and convention, 'dimensional lumber'. All that matters is that if I buy a '2 by' almost anywhere in the United States, it will be close to the same size and characteristics.

Re:Modern HDDs have gotten somewhat better by walrus01 on Wednesday June 22, 2022 @04:46PM (#31841657)
if you really want to see something that's irretrievably stuck in american measurements, take a look at the national engineering standards and vendor sources for telecom tower components... for anything from a light duty 40' aluminum self support you might put in your yard for ham radio all the way up to gargantuan 2000' guyed broadcast towers.

canada has no hope of doing anything in metric here, everything is in US units, and probably will be until the year 2200

Re:Modern HDDs have gotten somewhat better by The_Colonel on Wednesday June 22, 2022 @04:48PM (#31841678)
What is 2x4?
Re:Modern HDDs have gotten somewhat better by fragmede on Wednesday June 22, 2022 @10:02PM (#31844339)
It refers to pieces of wood used for building in North America, which comes in 2 inch by 4 inches by x feet, where x is generally 8 or 10.
Re:Modern HDDs have gotten somewhat better by Akronymus on Thursday June 23, 2022 @05:00AM (#31846591)
So, basically a 50x100mm piece of lumber?

(I rounded those values because if the US were to go metric, the dimension would slightly change, rather than being exactly 50.8 by 101.6 mm)

Re:Modern HDDs have gotten somewhat better by kennywinker on Thursday June 23, 2022 @10:26AM (#31849550)
Except, per my original point - it’s called a 2x4 - but it’s not actually 2 inches by 4 inches, which would be as you say 50ish x 100ish mm. In reality it’s actually standardized to be 38 x 89 mm
Re:Modern HDDs have gotten somewhat better by Akronymus on Thursday June 23, 2022 @10:44AM (#31849802)
Guess 40 by 90 then.

What I tried to convey, in a stupid way, is that there wouldnt be an exact translation of the measurements into metric, but that the dimensions qould slightly change in switching over.

and now I realize that that point was irrelevant anyways. Sorry for derailing the thread with useless replies.

Re:Modern HDDs have gotten somewhat better by walrus01 on Wednesday June 22, 2022 @05:36PM (#31842166)
a north american measurement of wood
Re:Modern HDDs have gotten somewhat better by babypuncher on Wednesday June 22, 2022 @06:29PM (#31842683)
It wasn't sneaky way back then. A "byte" has not always been defined as "exactly 8 bits"; it used to be hardware-dependent. 6-bit bytes were common in systems with word lengths that were multiples of 6 instead of the now-common 8.
Re:Modern HDDs have gotten somewhat better by walrus01 on Wednesday June 22, 2022 @03:30PM (#31840726)
at this point I think it's something we will have to deal with, like, I'd prefer to buy carpet by the square meter, but fact is in canada it's still sold by the square foot just like in the USA, with weird non metric units.
Re:Modern HDDs have gotten somewhat better by perennialmind on Wednesday June 22, 2022 @02:19PM (#31839798)
The sentence "a 32-bit CPU register can address up to 4 gigabyte of memory" is sloppy, the sentence "a 32-bit CPU register can address up to 4 GiB of memory" is not.
Re:Modern HDDs have gotten somewhat better by hulitu on Wednesday June 22, 2022 @02:31PM (#31839934)
> The sentence "a 32-bit CPU register can address up to 4 gigabyte of memory" is sloppy, the sentence "a 32-bit CPU register can address up to 4 GiB of memory" is not.

"Up to" is marketing. From my experience "up to" starts from 0.

Re:Modern HDDs have gotten somewhat better by mwint on Wednesday June 22, 2022 @02:42PM (#31840073)
32 bits can address 4GiB. That's how 2^32 works. It can also address 2GiB, or 1GiB. I think the wording is fine.

Like saying a car can go up to 200mph, no one thinks the car is unable to go 50mph.

Re:Modern HDDs have gotten somewhat better by dspillett on Wednesday June 22, 2022 @08:26PM (#31843646)
No, it is absurd to claim that something which predates using base 2 based counts being common is designed to con people who work in that sort of unit.

Communications theory used 1,000s and such to state transfer rates as with any other scientific unit long before modern drives. That stuck when things were recorded to mediums such as tape, and every disk ever has been measured on 1,000s or 1,000,000s or …

There have been some oddities along the way: punched cards were all over the place and were measured in characters not bits/bytes, many comms methods too were measured in characters/symbols or modulations rather than bits & bytes, many comms and storage (tape, disk, and RAM/ROM) formats are not nice round powers of 2 in raw form due to check bits, parity bits, and stop bits.

The one and only occurrence I can think of where 2^10 was used officially for a physical storage measure is the 1.44Mb floppy format which was a strange bastard of multiple divisions: it was 1440KiB, so 1.4410001024, so really was either ~1.38MiB or ~1.47MB.

> Nowhere else in the computer is the base-10 version standard, and it shouldn't be.

Nowhere else in science nor enginerring is 2^10 the standard instead of 10^3, and it shouldn't be. In computing we've adopted 2^10 that because some things are easier for us to think about that way, and 2^10 and 10^3 are close enough for many estimating or quick calc purposes, but why should science that predates that practise being common be forced to change simply for our convenience?

Yes I support moving from imperial to metric where that hasn't already been done, but not because I encountered metric first (I didn't) so the other is unfamiliar but because it makes sense. You might argue that to programmers or less technical tech users 2^10 makes more sense, but they are less important in terms of science and engineering then, well, the rest of science and engineering.

> what's the max file size of a file stored on a FAT32 filesystem again?

2GiB, or approximately 2.147GB. Many people incorrectly calling it 2GB does not change that G as a prefix means 10^9 and not 2^31 - otherwise we have to accept other things like “u” being a correct spelling of “you”!

It isn't a grand conspiracy to defraud us. We chose to give those prefixes a double meaning, it isn't the rest of science & engineering's fault that we sometimes get confused by that choice that we made and they didn't.

Re:Modern HDDs have gotten somewhat better by taffer on Wednesday June 22, 2022 @02:44PM (#31840098)
I wonder why RAM manufacturers are so generous that they don't engage in deceptive marketing, while everyone else does.
Re:Modern HDDs have gotten somewhat better by babypuncher on Wednesday June 22, 2022 @06:55PM (#31842951)
It goes back to the '60s and '70s.

Back then, a "byte" was not necessarily "8 bits", but whatever the smallest unit of addressable memory was. This was the same as or an even fraction of the word length of the computer. Common word lengths back then included 12, 18, 24, and even 30 bits. 8-bit bytes would only fit evenly into one of those.

Storage devices like hard drives are not addressed like memory, and the logical segmentation of bits into bytes/words/clusters was a function of the software, not the hardware. One storage device could be built for use in an 8-bit or 12-bit computer, resulting in very different numbers if you tried counting its size in bytes on these different machines.

Re:Modern HDDs have gotten somewhat better by vikingerik on Wednesday June 22, 2022 @04:39PM (#31841587)
> Or, to bring it back to hard drives: what's the max file size of a file stored on a FAT32 filesystem again?

That is not about hard drives, that is about the abstraction layer of FAT32, which uses a 32-bit quantity for size.

FAT32 could be used for a storage device that stores things based on powers-of-10 sizes. It's just an implementation detail if anything happens to do it with powers of 2 underneath.

Re:Modern HDDs have gotten somewhat better by colanderman on Wednesday June 22, 2022 @05:47PM (#31842274)
> FAT32 could be used for a storage device that stores things based on powers-of-10 sizes

Not without wasting lots of space. FAT32 only allows sector sizes of 512, 1024, 2048, or 4096 bytes.

[1] https://www.cs.fsu.edu/~cop4610t/assignments/project3/spec/f...

Re:Modern HDDs have gotten somewhat better by BeetleB on Wednesday June 22, 2022 @05:11PM (#31841913)
> It's absurd to claim they're doing this for any other reason beyond deceptive marketing.

Your comment is a good example of "I so want this narrative to be true!" without any backing.

Back in the 90's, my computer studies textbook had 1 KB as 1024 bytes, and 1 MB as 1,000,000 bytes. It explicitly stated that it is not 1024 KB. What's their angle?

Re:Modern HDDs have gotten somewhat better by throw0101a on Wednesday June 22, 2022 @07:02PM (#31843013)
> It's absurd to claim they're doing this for any other reason beyond deceptive marketing. […] It's just bad and deceptive, and hard drive manufactures should not be allowed to play these kinds of games. You should stop carrying the water for their shitty practices.

The confusion of binary and decimal prefixes has been around for decades:

* https://en.wikipedia.org/wiki/Timeline_of_binary_prefixes

This is not some recent conspiracy to boost profits or whatever.

Re:Modern HDDs have gotten somewhat better by Stratoscope on Wednesday June 22, 2022 @07:22PM (#31843182)
If this troubles you, wait until you learn how many bytes are on a 1.44MB floppy disk.

It holds 1,474,560 bytes.

That is 1.44 * 1000 * 1024.

Re:Modern HDDs have gotten somewhat better by mistrial9 on Wednesday June 22, 2022 @07:24PM (#31843200)
zero people in 2022 use floppy disks (and yes I still have mine)
Re:Modern HDDs have gotten somewhat better by Stratoscope on Wednesday June 22, 2022 @08:25PM (#31843645)
Anecdata: I have a lovely old Canon Cat in working condition that I want to sell to someone who is interested in this piece of computing history. But I have somehow misplaced the box of floppy disks that came with it!

I know they are here somewhere. Looks like I have some housekeeping to do.

When I find them, that number will become nonzero...

Re:Modern HDDs have gotten somewhat better by vel0city on Wednesday June 22, 2022 @02:36PM (#31839988)
How many meters are there in a kilometer? 1024?
Re:Modern HDDs have gotten somewhat better by remedan on Wednesday June 22, 2022 @02:54PM (#31840255)
There are 1000 meters in a kilometer. However, there are 1024 meters in a kibimeter.
Re:Modern HDDs have gotten somewhat better by bombcar on Wednesday June 22, 2022 @03:44PM (#31840917)
Kibi, the prefix that comes from kibitzing.