> > ban SSD from critical systems.
> That's an over-reaction if I've ever seen one.
No, it's really not. SFJulie's point is that we don't know enough about the way SSDs fail to build a good failure model for them, and that we currently don't seem to have a good way to even detect that they are failing. If you can't build a failure model for something, you can't build a good risk model including it. That's kinda her point.
FWIW, I don't know that I fully agree with that point: it seems to me that cryptographic integrity checks and backups would suffice, but … maybe not?
> If it's something that has to guarantee high performance under massive load, SSDs are what you need, period.
We seemed to do alright four years ago, no? SSDs are not a requirement: one can get equivalent system performance, with predictable failures, by spending more money on more hard drives and CPUs to talk to them.
Lots more money. Many, many more hard drives, with failure rates to match. And you're still not going to get there.
We started running a 240TB all SSD array a few months ago and we get sub-millisecond latencies on most reads and writes. When we see an excursion to 10ms that's cause for concern; 50ms and we do a root-cause. We can peg multiple 16GBit fiber channels and it's okay. And it fits in half a rack; try that with disk.
I cannot imagine doing this with spinning rust.
To me the three scariest words in engineering are "common mode failure". I personally never buy more than one hard disk at a time, so that I'll hopefully avoid something like both haven been shaken too much in their shipping container (the best explanation anyone was able to come up with for a Seagate failure pattern some time ago).
So, yes, the above, to detect and recover, but plenty of services need very up to date backups, so if I was designing such a system, I'd make sure the (near-)real time backup would be to SSDs that share nothing in common with my production ones, different manufacturer, chip sets, etc.
Or if the load could handle it ... ah, I see why Seagate is still selling real SAS enterprise disks (last time I checked, if I remember correctly, <= 1TB and up to 15K RPM). We understand those, not that they won't also fail on you.
Basically all you need to define is the expected output. "99.9pct of my reads are within 1ms" + "The read response is correct 99.9999% of the time" and then verify. Eject the disk if that doesn't hold, restore from replica. The rate of replacements is a factor of how close you are to the true failure model.
Ok, it may be RAID1+0 but, what about data corruptions?
If primary fails, and secondary is "ok" what will be the state of your data? You don't roll back an image when everything is green.
The point is that in critical systems according to the SLA or the critical aspect of the mission (flight recorder, medical instruments...) silent failures, silent data corruption may not be ok.
However for 99% of the business that is making money out of SSD right now in the world (advertisement, marketing) this is not really a concern. They don't aim at an absolute reliability. Most industry just aim at good enough.
Some small islands of IT aims at doing a legally accurate book-keeping. Like banking, accounting, legal electronic archiving for instance...
This topic is covered in Signal Processing in the first chapter of the cost of False Positive and False Negative.
Security and risk management norms force to enumerate and evaluate the costs and predicting the outcomes of failures in terms of costs.
Here, the paper is saying SSD works pretty much fine all the time. But for failure, there is a lot we don't know.
How do you evaluate the cost of a risk you cannot compute?
And how does another solution that maybe slower rates in risk management with a better failure model that can be better predicted?
The scarriest part in the study is the implication that their could be propagation of failures by coupling in certain environment and that some failure could be snowballing.
SSD are affected by heat/memory usage, CPU load is affected by failures, heat rises, SSD are more likely to fail ....
They are implying that failures can propagate or spread or happen like a contamination.
1 SSD failing alone is already a problem, even small chances of coupling in failures is making the potentiality for a massive failure.
Would you take that risk with a SLA of 99.999%?
As I said, it depends on what your definition of "critical system" is. For aviation, for instance, there are many other factors (shielding etc); no hardware is perfect for all uses.
> How do you evaluate the cost of a risk you cannot compute?
Easy: you come up with a worst-case scenario, make plans to address it, and the cost of implementing such plans is your cost. Say you assume you could get silent errors: you build test routines for actual data integrity and implement test systems for continuous checks. The cost of building and maintaining all that is the cost of (accepting) your risk.
Point-in-time recovery is a snap compared to that domain.
Aviation standard goal is 9 nines of reliability, and many systems get there.
If SSDs make your business 2x as productive or 2x as remunerative, the cost of an extra backup or two is simply a blip.
> ban SSD from critical systems.
That's an over-reaction if I've ever seen one. I guess it depends what a "critical system" is for you. If it's something that has to guarantee high performance under massive load, SSDs are what you need, period. If it's something that has to guarantee 100-year data storage under a mountain, then there might be better choices, but they might not involve spinning drives either.