I'll toss in on that anecdata. This has happened to me a several times. In all these cases we were dealing with drives with more or less sequential serial numbers. In two instances they were just cache drives for our CDN nodes. Not a big deal, but I sure kept the remote hands busy those weeks trying to keep enough nodes online. In a prior job, it was our primary storage array. You'd think that RAID6+hot spare would be pretty robust, but 3 near simultaneous drive failures made a mockery of that. That was a bad day. The hot spare starting doing its thing with the first failure, and if it had finished rebuilding before the subsequent failures, we'd have been ok, but alas.
Like, 25+ years ago I would've bought hard drives for just my personal usage in a software raid making sure I don't get consecutive serial numbers, but ones that are very different. I'd go to my local hardware shop and ask them specifically for that. They'd show me the drives / serial numbers before I ever even bought them for real.
I even used different manufacturers at some point when they didn't have non consecutive serials. I lost some storage because the drives weren't exactly the same size even though the advertized size matched, but better than having the RAID and extra cost be for nothing.
I can't fathom how anyone that is running drives in actual production wouldn't have been doing that.
I mostly just buy multiple brands from multiple vendors. And size the partitions for mdadm a bit smaller.
But even the same model where it's 2 each from bestbuy, Amazon, newegg, microcenter, seems to get me a nice assortment of variety.
Of course, learned experience has value in the long term for a reason.
I rescued what could be rescued at a few KB/s read speed and then checked the serial numbers...
I just get 1/3 Toshiba, 1/3 WD, 1/3 Seagate.
Moving to software storage systems (ZFS, StorageSpaces, etc.) has saved my butt so many times.
Don’t know about disk batches, though. Took used old second hand drives. (Many different batches due to procurement timelines.) Half of them was thrown out because they were clicky. All were tested with S.M.A.R.T. Took about a week. The ones that worked are mostly still around. Only a third of the ones that survived S.M.A.R.T. have failed so far.
Whats funny is that about a year ago I ended up installing FreeBSD onto the same Microserver and ran a 5 x 500GB mirror for my most precious data. The drives were ancient but not a single failure.
As someone who never played with hardware raid ZFS blows my mind. The drive that failed was a non issue because the pool it belongs to was a pool with a single vdev (4 disk mirror). Due to the location of the server I had to shut down the system to pull the drive but yeah I think that was 2 weeks later. If this was the old days I would have had to source another drive and copy the data over.
Every time I think I might need a feature in a file system it seems to have it.
The drives are spaced apart by empty drive slots and have a 12cm case fan cranked to max blowing over it at all times.
It is in a tower though so maybe it was bumped at some time and that caused the issue. Being in the top slot this would have had the greatest effect on the drive. I doubt it though.
Usage is low and the drives are spinning 24/7.
Still I think I am cursed when it comes to Seagate.
The NVMe drives in my servers have these little aluminium cases on them as part of the hotswap assembly. They manage the temperature differential by using a conductive pad for the controller, but not the flash.
By the time Backblaze has a sufficient number of a particular model and sufficient time lapsed to measure failures, the drive is an obsolete model, so the report cannot really inform my decision for buying new drives. These are new drive stats, so not sure it is that useful for buying a used drive either, because of the bathtub shaped failure rate curve.
So the conclusion I take from this report is that when a new drive comes out, you have no way to tell if it's going to be a good model, a good batch, so better stop worrying about it and plan for failure instead, because you could get a bad/damaged batch of even the best models.
Worked in a component test role for many years. It's all of the above. We definitely saw significant differences in AFR across various models, even within the same product line, which were not specific to a batch. Sometimes simply having more or less platters can be enough to skew the failure rate. We didn't do in depth forensics models with higher AFRs as we'd just disqualify them and move on, but I always assumed it probably had something to do with electrical, mechanical (vibration/harmonics) or thermal differences.
I am becoming more and more convinced that hard drive reliability is linked to the batch more than to the individual drive models themselves. Often you will read online of people experiencing multiple failures from drives purchased from the same batch.
I cannot prove this because I have no idea about Blackblazes procurement patterns but I bought one of the better drives in this list (ST16000NM001G) and it failed within a year.
When it comes to hard drives or storage more generally a better approach is protect yourself against down time with software raid and backups and pray that if a drive does fail it does so within the warranty period.