News for hackers. Stuff that matters.
Re:Best CPUs for Workstations: 2017 by Retric on Thursday October 05, 2017 @10:00AM (#15409180)
ECC has no where near 30% performance overhead.

In terms of speed it's ~2–3 percent slower worst case, and memory is only one component of overall performance lowering the impact further.

More importantly, bit errors are not evenly distributed with some users seeing vastly more errors.

Re:Best CPUs for Workstations: 2017 by keldaris on Thursday October 05, 2017 @02:03PM (#15411180)
The performance penalties I'm referring to are memory performance penalties. If your workload isn't memory-bound (and most aren't), you'll measure a difference of 2% or less, sure, but that's not really relevant when talking about memory performance. In extreme cases (such as, for instance, lattice Boltzmann simulations, where the streaming step is completely memory bound) I've seen over 50% performance degradation, but much of it can be attributed to the notoriously loose timings on most ECC DRAM.
Re:Best CPUs for Workstations: 2017 by dijit on Thursday October 05, 2017 @07:24PM (#15413329)
I feel I have the relevant real world expertise to have a say about that.

I am one of the principle engineers working on a game that was able to support over 1.2 million players concurrently, so I feel that I have some merit behind what I'm about to say.

ECC Memory does not have such an extreme overhead. Our gameserver is memory bandwidth bound. To put that into perspective a change from DDR3 to DDR4 saw a jump in performance of 34%~. Using non-ecc memory (which is obviously very available to the development team) only gave a penalty of roughly 3% in the worst case. (both DDR4 systems).

Why are we memory bandwidth bound? because we allocate 250GB of "game world" and phases and make teeny tiny modifications to them when you run around or pick up loot and shit. It's as taxing as you can be on memory bandwidth since you can't batch set or load anything.

Re:Best CPUs for Workstations: 2017 by eightysixfour on Thursday October 05, 2017 @07:53PM (#15413475)
I think what he is trying to say is there isn’t a lot of DDR4 out there at 3200mhz with tight timings like there is with non-ECC memory, not just that ECC vs non-ECC at the same specs is 10-30% slower.
Re:Best CPUs for Workstations: 2017 by keldaris on Friday October 06, 2017 @03:21PM (#15419615)
That's right, I'm sorry if I worded it ambiguously in my previous posts. ECC RAM is considerably slower than non-ECC RAM, but nowadays it's mostly because of looser timings and lower frequencies. I'm well aware that there are people who claim that frequencies beyond the ECC standard are liable to cause data loss (for reasons usually left unspecified), but I've never seen any reliable data to support that claim, as long as the frequencies and timings tested by the manufacturer are maintained (not to be confused with people who overclock RAM way beyond its tested specification and experience instability).

On a further note, while the implementation of ECC itself is very fast for modern RAM, the same is not true of VRAM on GPUs. At least as implemented in NVidia Tesla GPUs, enabling ECC has a huge performance penalty by itself. [1]

[1] http://geoceano.fr/wp-content/uploads/2015/01/cpu-gpu-lbm-in...

Re:Best CPUs for Workstations: 2017 by Retric on Friday October 06, 2017 @07:02PM (#15421075)
I think some of this breakdown is from motherboard support. You can MB that supports a single CPU and 128GB for DDR4 4400 which is significantly higher than ECC.

When you start looking at multi CPU computers with 1+TB of ram the story changes.

PS: Some of this stuff is also really hard to test. Without ECC it's hard to data in terms of stability for petabytes of ram over years. And no extrapolations don't work very well.

Re:Best CPUs for Workstations: 2017 by Retric on Thursday October 05, 2017 @02:43PM (#15411558)
[Citation Needed] With identical memory ECC and non <2% is the actual memory overhead.

Now, non ECC memory might have better specs / a higher overclock rate etc, but doing either of those increases the rate of errors. And sure you might not notice the problem directly, but that does not mean things are ok.

Re:Best CPUs for Workstations: 2017 by TrainedMonkey on Thursday October 05, 2017 @12:26PM (#15410425)
Can you elaborate on that distribution?
Re:Best CPUs for Workstations: 2017 by Retric on Thursday October 05, 2017 @12:42PM (#15410573)
The main causes are altitude/background radiation, temperate, overclocking, faulty parts etc. But, memory can have 100+x the normal failure rates without being particularly noticeable in day to day operations.

The value of ECC is often more to detect errors than to correct them. Remember, if you have 10 errors an hour then you unlikely to see them in a 20 second memory test.

In practice out of 10,000 machines you might have 1/2 your errors on 10 of them. Critically, if you have ECC memory and you both detect and replace those 10 machines the apparent rate of failures then drops.

PS: This is a little dated (09) and only from a single set. But, still an interesting introduction: https://www.cs.toronto.edu/~bianca/papers/sigmetrics09.pdf "Around 20% of DIMMs in Platform A and B are affected by correctable errors per year, compared to less than 4% of DIMMs in Platform C and D. ... About a third of all machines in the fleet experience at least one memory error per year (see column CE Incid. %) and the average number of correctable errors per year is over 22,000. "

Re:Best CPUs for Workstations: 2017 by keldaris on Thursday October 05, 2017 @02:00PM (#15411155)
As I understand it, the situation is even better (as in, less errors) than that with DDR4. Unfortunately, all the published data I could find is woefully out of date. Still, even if you generously assume something on the order of one correctable memory error per machine year, how many of those errors do you actually care about? Unless you have 100% memory utilization for 24/7 critical tasks, most of those errors probably don't even matter.

EDIT: the least out of date reference I could find is a blog post by Jeff Atwood, where he concludes that based on the available literature ECC-correctable errors are quite rare. [1]

[1] https://blog.codinghorror.com/to-ecc-or-not-to-ecc/

Re:Best CPUs for Workstations: 2017 by Retric on Thursday October 05, 2017 @02:39PM (#15411524)
The median performance for non ECC memory is very good. The problem is worst case performance is hard to detect and much much worse than that.

"The median number of errors per year for those machines that experience at least one error ranges from 25 to 611." "We find that for all platforms, 20% of the machines with errors make up more than 90% of all observed errors for that platform."

A minority of that 20% again have even more issues.

The worse case is literally 10,000x more than 1 bit error per year on a single machine. Without using ECC you really can't tell other than some vague not sure what's wrong just replace it.