News for hackers. Stuff that matters.
Re:Puget Systems' Perspective on Intel CPU Instability Issues by Aurornis on Sunday August 04, 2024 @10:43PM (#41157722)
> Have I been living in a fantasy bubble where CPU's do exactly what you asked of them (and errors come from not holding it right)?

These are gaming CPUs clocked right up to the threshold of stability (and in some cases, past it)

Server CPUs with ECC RAM are significantly better.

However, if you haven’t experienced much CPU instability, you may not have operated at scale where it appears. Get into the scale where operations occur across 100,000s of CPUs and weird things happen constantly.

Re:Puget Systems' Perspective on Intel CPU Instability Issues by userbinator on Monday August 05, 2024 @01:38AM (#41158421)
One of the previous discussions of this instability noted that it was happening to the server equivalents (which are after all the same die) at stock settings as well.
Re:Puget Systems' Perspective on Intel CPU Instability Issues by Dalewyn on Monday August 05, 2024 @03:23AM (#41158869)
I remember hearing that server motherboards also played a role in overclocking out of the box, which is frankly fucking stupid. I don't recall anything about Raptor Lake-based Xeons suffering from degradation.
Re:Puget Systems' Perspective on Intel CPU Instability Issues by userbinator on Monday August 05, 2024 @05:17AM (#41159427)
Unless they disable Turbo Boost (which is horrible for performance, but great if you want benchmark consistency), the CPU will automatically overclock until it reaches the limits, adjusting both voltage and frequency.

All the evidence I've seen points to electromigration as a cause of this degradation, and IMHO excessively aggressive automatic overvolting by Intel's microcode is to blame.

There is actually a simple experiment which can determine whether that is true --- remove the fan from the heatsink, or even let the CPU run without a heatsink. As the CPU will automatically throttle once it reaches its designed maximum temperature (and AFAIK that is a hardcoded limit), it will lower its frequency and voltage to maintain that temperature. If this results in a stable CPU, while the one that has great cooling becomes more unstable, it confirms the hypothesis.

There are numerous stories of machines where the heatsink was not in contact with the CPU for some reason, yet they remained perfectly stable (but slow) for many years. I can also say that I've had an 8th-gen i7 running at 100% 24x7 with all power and turbo limits disabled, with its temperature constantly at the design limit of 100C, and it has also remained stable for over 5 years.

Re:Puget Systems' Perspective on Intel CPU Instability Issues by cesarb on Monday August 05, 2024 @07:39AM (#41160269)
> There are numerous stories of machines where the heatsink was not in contact with the CPU for some reason, yet they remained perfectly stable (but slow) for many years.

I once had a laptop, which came from the factory with the four screws which hold the heatsink to the CPU missing. It was very slow, and shut down after a few minutes (the reason being thermal shutdown in the BIOS event log helped diagnose the issue). After the four screws were replaced (each screw came in its own large individual cardboard box), it worked fine for many years, BUT after a couple of years (still under warranty), the motherboard failed with a short in the power input. I suspect that all the extra heat from when the CPU was without a working heatsink went to the power supply components through the motherboard ground plane, and cooked them, significantly shortening their useful life.

Re:Puget Systems' Perspective on Intel CPU Instability Issues by gradschool on Monday August 05, 2024 @07:35AM (#41160244)
That's a lot of cpu time. Maybe they just don't make them like they used to. If there's some crazy complicated numerical or combinatorial problem you've been trying to crack, do tell.
Re:Puget Systems' Perspective on Intel CPU Instability Issues by Dalewyn on Monday August 05, 2024 @08:37AM (#41160770)
>Unless they disable Turbo Boost (which is horrible for performance, but great if you want benchmark consistency), the CPU will automatically overclock until it reaches the limits, adjusting both voltage and frequency.

Turbo Boost (and Thermal Velocity Boost if applicable) frequencies are according to specifications, it's not an overclock.

Re:Puget Systems' Perspective on Intel CPU Instability Issues by sirn on Monday August 05, 2024 @05:04AM (#41159362)
I feel like the entire fiasco has been multiple issues being lumped together as one, and muddied to the point that even a bluescreen out of an attempt to run XMP at extremely high MT/s are now being claimed as degradation. From what I can make out of this mud, there seems to be (1) a failure caused by high current due to some boards unlocking IccMax/PL1/PL2 by default, and (2) high voltage during a single-core boost (TVB). The former is caused by overclocking, and the latter seems to be Intel's failure to validate the CPUs at low load/long period of single-core boost, where IccMax/PL no longer matters as much (since single-core boost never exceeds PL1 anyway).

Most Raptor Lake "server boards" right now are W680 with client CPUs because the C266/Xeon E-2400 took a long time to come out. The one intended for workstations typically has overclockable settings or is even overclocked by default, which means it's likely to get hit with the failure (1). The one intended for servers do have more conservative settings, but can still be hit with failure (2) under some conditions.

Buildzoid released a video on the Supermicro W680 blade a bit ago that were having issues after running a single-core load 24x7, which is essentially 24x7 boost[1] (aka issue (2)). Xeon E-2400 _could_ be affected in this scenario, although even the highest clock E-2400 SKU (E-2488) is only running at 5.6GHz without Thermal Velocity Boost, and most others are ranging from 4.5 to 5.2 GHz boost (rather than the 5.8 to 6 GHz boost some client SKUs do). I feel like the actual B0 Xeon E-2400 would be a lot less prone to both failures (1) and (2) due to this (but it could happen, though there's no reports of such).

But then the conversation gets muddied enough that "even servers and Xeons are affected" becomes the common narrative (while the former is true, the circumstances needs to be noted; and for Xeons, it's a _maybe_ at most, since right now there's no report of Xeon E-2400 failing).

[1]: https://www.youtube.com/watch?v=yYfBxmBfq7k

Re:Puget Systems' Perspective on Intel CPU Instability Issues by userbinator on Tuesday August 06, 2024 @12:11AM (#41167702)
Looking around, I'm seeing reports of 1.4-1.5V core voltages using Intel's stock profiles, with some even going to 1.7V. That's insanely high for a 10nm process and I'm not surprised about the degradation. For comparison, in the 45/32nm days 1.2-1.3V was the norm, with some extreme overclockers (who don't expect CPUs to survive for more than a few minutes, using liquid nitrogen etc.) hitting ~1.5V, and 1.4V was a commonly quoted safe upper limit for 24x7 operation.
Re:Puget Systems' Perspective on Intel CPU Instability Issues by sirn on Wednesday August 07, 2024 @10:36AM (#41181927)
This is why I think it's going to be much harder for Xeon to be affected by this, as they're normally running in a more conservative voltage settings. I don't have Xeon E-2400 to look at, but Raptor Lake should be able to do 5.6 GHz at 1.3-1.4V-ish, which should be within a safe voltage range. (even the "power hungry" w9-3495X only runs at ~1.25V during 4.8 GHz TVB, and ~1.15V at non-TVB 4.6 GHz boost.)