Good server CPUs should survive 7 years of use without problems.

Nevertheless, their expected lifetime is not much greater than that.

I normally replace CPUs because they become obsolete much earlier than they should show age problems.

Nevertheless, I had a pair of Opterons that were used continuously for about 10 years.

At the end, they developed a very high leakage current, leading to an idle power consumption several times higher than in the beginning.

Increased leakage current is one of the most frequent signs of aging in semiconductor integrated circuits.

How do you assess that increased consumption?

Server BMCs provide power consumption timelines, which are measured in real time by the PSUs themselves. Also processors can assess their power consumption and report it to the OS and to the platform.

Querying and recording these as time series provides very nice insights over time.

Nice I'll look into that, I log those but never looked at them

What temp you got on those machines?

Since you haven't tried the 7nm CPUs yet I guess we'll have to see?

7 years sounds really short to me, I'm planning on running my machines for 100 years!

Just looked to a bunch of Xeon Gold 6258Rs. They're running around ~70 degrees Celsius per die. The machine is under full load. I'm sure its fans are at full speed.

That CPU's reported temperatures are 81 for high, 91 for critical (in degrees Celsius).

We get new systems almost every year, so we have a rolling set consisting many systems. So, we have a cross section of systems to observe.

Server parts are big chips (or a bunch of chiplets) running at fairly moderate clock speeds. For example your XCC CPU should have a die size in the 600-700 mm² neighborhood for a power density of around 0.3 W/mm² (which neglects hotspots because probably half the CPU is cache, which does not consume half the power). Desktop parts like the 10900K can chug down 200+ W on a 200 mm² die, though that's basically impossible to cool. Even more reasonable CPUs like AMD's 5000 series run something like 40-50 W through an 80 mm² die, these are still hard to cool.

This is a different and refreshing perspective. I've never looked from the perspective of W/mm². OTOH, the number I've given is calculated by eyeballing the per core internal thermistors, provided by lm_sensors (hence by CPU itself).

So, per die sensors are probably reading somewhat lower numbers, but the core's cooking at 70 degrees C internally. Nevertheless, bigger die surface inevitably allows better heat conduction and reduces internal stresses considerably, when compared to a desktop part.

Desktop use is also very uneven, leading to a lot of large changes in temperature (35 -> 80 and back), which causes more material stress, than when everything is loaded evenly all the time. The same is true for electrical transients.

If you have a good cooler with lots of thermal capacity and good fan (like an Arctic Cooling or Noctua, nothing at the extreme end), the increase is very gradual. Also, short spikes in loads are well absorbed with minimal temperature changes.

Nevertheless the numbers you mention are neither unrealistic, nor impossible in stock cooling and/or sustained load scenarios.

5950X with the top of the line noctua cooler jumps from 35C to 80C very quickly when a single core is loaded at 100% (this is the worst case, because it leads to max voltage being applied to the CPU cores (because of boost to almost 5GHz), compared to loading all 16 cores at 100%, which will result in just 60˚C temperature sustained over long periods of full load). So interestingly, compiling the Linux kernel with 32 threads is less thermally straining on the CPU than browsing the web and a single page loading 10 different animated ads.

This mirrors my experience with Zen 2 and 3 as well. Multi-core loads result in just a few W per core and decent temperatures, while few or single core loads push per-core power into the 12-16 W region and the temperature rise/fall is basically instant (brick-wall at 10 Hz update rate, though it's unclear what filtering is applied and how the CPU derives a single temperature from its probably numerous sensors), which suggests that they're not limited by the cooler, but rather by the thermal impedance from the active area to the cooler. The core itself is really tiny (iirc around 4 mm²)...

This is my conundrum when pondering buying a Threadripper 3970x / 3975x Pro workstation: aren't those chips cooked very aggressively when loaded, even with very good coolers? Doesn't that mean that one day the CPU might just burn?

I really want to buy a proper serious workstation and I can afford it in a few months but I keep wondering: is now the best time for it?

I wouldn't really worry about it. I have an overclocked 3970x and a Noctua air cooler keeps it much cooler than the Intel 6950X I had before (also aircooled, with an even larger Noctua cooler). Air cooling is more of a limit to how high an overclock you can get; normal operation will be no problem for any cooling system designed for Threadripper.

There's never a good time to buy a new computer. It's always going to be obsolete before you open the box. Though in this case, one might wait for the Zen 3 Threadrippers.

Thanks for the feedback, it's reassuring. As other posters said, I am worried because it's a workstation which means it won't have consistent load, and material stress is much more destructive in the thermal conditions of expanding and contracting. So if in your ~8h workday your station gets stressed for 2 or so hours in irregular intervals, how destructive is that, objectively?

--

As for the Zen 3 TRs, well, to be fair, I am not looking to blow $50k on a workstation. :) I am more interested how -- and if -- they will drive down the prices of the TR 3900 SKUs...

It probably won't drive down prices that much. You won't want the old thing, and the incremental cost between generations is not the expensive part of a workstation. I have bought old parts before (to replace broken parts), and my experience is that the price didn't change much. What was a $130 motherboard when brand new, is $130 when a generation old. Does it make any sense? Nope. But that seems to be how it is.

You won't spend $50,000 on a workstation just by using current-generation parts. I think when you see a workstation that costs that much, it's because it has multiple GPUs in them. Pro GPUs are always artificially overpriced, and given the GPU shortage, they're now even more overpriced.

I did a quick pcpartpicker expedition and found that using last-gen parts saves you about $1000 on a $6000 32 core Threadripper workstation. I compared last-gen SSDs, consumer GPUs, processor, and motherboard, and picked relatively high-end parts. You will save more money by dropping to 24 (or heaven forbid, 16) cores, not getting an extreme motherboard, getting 64G of RAM instead of 128, etc.

This could all be invalid in a few months. It is hard to separate "market is always crazy" from "this whole COVID thing is going on". Building a workstation during the pandemic was a pain -- I bought a used GPU, and didn't get ECC memory because nobody would sell me any. If you wait a year, that is likely to improve, and there will be newer hardware. But, if you need to do some computing between now and then, you don't have much choice but to buy what's available now, and it certainly makes for a very good computer.

Thanks for the check! Much appreciated, it did put my mind at ease.

I realized I was looking at several custom-made TR Pro workstations and they of course have a hefty markup on top -- custom cases, custom cooling, pre-added several PCIe NVMe riser cards, plus style price tag I suppose, etc. Should have looked in PC Part Picker indeed.

And yeah, last-gen tech very rarely gets discounted even by normal people who just post ads in local Craigslist-like websites (like OLX). Stuff that's 2 or more generations ago is discounted, but not last gen. Puzzling indeed, especially having in mind that this last gen tech is very soon going to be "two gens ago". Oh well.

As for ECC RAM, I hear you. I was unable to find such anywhere officially but lucked out that one guy in the local OLX was having loads of it but just didn't post the ads due to being very busy (we communicated because of other ads) and then I just bought 64GB ECC DDR3 RAM from him for a home NAS that I am gradually expanding.

I'm definitely very interested in having a TR Pro workstation; started getting sick of Macs and their artificial slowness. I got the iMac Pro and granted, I have it connected to my TV where it plays Twitch streams all day but hell, a lot of stuff on the terminal (that's not Python) just works slower than it does on a meager i3 Linux machine that I have lying around.

So I do want a TR Pro machine but I am very curious about TR 5000. A release is expected in August which isn't that far away. On the other hand, the TR Pro 5000 might take several more months on top. Hmmm. Decisions, decisions. :)

I wouldn't focus too much on the Pro SKU. It has a slower base frequency and boost frequency, with the upside that it has 8 memory channels and supports 2TB of RAM. If you need 2TB of RAM and 8 channels, it's what you need, but quad channel memory is still very good. Most desktop-class machines are still 2 channel.

There is something to be said for the prebuilt workstations from reputable vendors. It shows up in a box, and starts working. I have strongly considered that angle; being a system integrator is tough. If something doesn't work, it's a week cycle time where you find a new part, organize the RMA of the faulty part, etc. and there is a potentially unbounded amount of time spent tweaking. Meanwhile, HP or Lenovo just ships you a computer; they tested it and it works. You pay several thousand dollars for the privilege, but it might be worth it. (And if you want a TR Pro, you have no choice. AMD doesn't sell them to consumers.)

My rule of thumb is "if hardware is too hot for you to hold your finger on it indefinitely" it's too hot and it's going to break very soon!

Temperature kills hardware, you need to bring those temperatures down, and the only way to do that is to lower the wattage!

Atom is the perfect design, no crap and low power!

> "if hardware is too hot for you to hold your finger on it indefinitely" it's too hot and it's going to break very soon!

From my experience, that doesn't happen like that in enterprise hardware. Either wrong voltage (inside the system) or defective design causes premature death. If the server BMC says it's fine, it's fine.

> Temperature kills hardware, you need to bring those temperatures down, and the only way to do that is to lower the wattage!

High Performance Computing doesn't work like that, unfortunately :)

> Atom is the perfect design, no crap and low power!

I'm sure it has its own uses and can accomplish a lot, but in HPC, it won't cut it. I use small SBCs at home to do and try a lot of fun and useful stuff, but it has limits.

Your rule of thumb is wrong. The thing what kills hardware is temperature fluctuations. Keeping a processor running consistently at 90 degrees is much better than switching it off and on all the time. Running a CPU at full load has a very minor impact on its mechanical properties. Even after running for 10 years. Much less than storing it in your drawer for the same amount of time.

Temperature fluctuations damage the interfaces between the different materials inside an integrated circuit, so they will greatly lower the lifetime.

Nonetheless, there is a continuous aging of the metal traces (electromigration) and especially of the insulating layers, e.g. the MOS transistor gates, due to the difusion of atoms.

This continuous aging is accelerated by steady-state high temperatures and it eventually results in either open circuits or short circuits somewhere, destroying the device.

Good MOS integrated circuits are designed for a lifetime at their maximum specified temperature of at least 10 years or even 20 years or more for the better of them.

Nevertheless, this is puny in comparison with the lifetime of many semiconductor components produced 40-50 years ago, before the continuous shrinking of the active device sizes, which could have lifetimes of hundreds of years, when free from fabrication defects.

Yes, so I'm betting on these 14nm Atom boards because they are very performant per watt (I get twice the server juice out of these at 25W than I get out of my 6600 desktop at 65W!!!!!!!!!) and I think they will be more reliable at 42 celcius full blast than the still non existent 10 or smaller nm.

I even think like you that my 45nm D510MO probably will outlive these! But it's so underpowered (like 1/10 of the perf. at 15W) compared to these that I'm willing to take the risk!

The risk of new boards being so much better that I'll have to throw these away ever is zero at this point, memory being the bottleneck!

> Temperature kills hardware, you need to bring those temperatures down, and the only way to do that is to lower the wattage!

The best way to keep your CPU intact for a long time is keep it powered off. But then you have probably no use for this CPU…

> it's too hot and it's going to break very soon!

“very soon” on your scale from now to 100 years, probably. But most people prefer having a CPU working at full capacity for a few years than having a useless brick of silicon sitting around for a hundred year.

> "if hardware is too hot for you to hold your finger on it indefinitely" it's too hot and it's going to break very soon!

Are you talking about junction temperature, package temp, or heatsink temp? The only CPU I own with a package temp that's cold enough to touch is inside my phone.

Meanwhile some of my data center machines are a decade old and run at 75C all day. I've never had a CPU fail before the machine became obsolete

linux sensors command, where ever that is.

That data can't really be trusted, because various sensors often have large offsets or are just bogus on specific hardware. E.g. lm-sensors reports both a sensor with 0 °C and one that's usually something like 90-100 °C on my desktop, while also misreporting the CPU temperature due to not taking the AMD offset into account (might be patched in recent versions).

In terms of Intel and/or enterprise hardware, the values are reliable. You can cross-check them via IPMI Sensors (provided by BMC itself) or Intel powertop (by crosschecking cpu throttle states). Intel also writes relevant support code (cpufreqd, thermals, etc.) themselves.

The biggest offender in my experience is OrangePi Zero, but I need a thermal probe to verify it.

>if hardware is too hot for you to hold your finger on it indefinitely

That's 50C (bit less)... which is quite low operating temperature and very far from anything that damages silicon. Temperature fluctuations are far worse due to thermal expansion/contraction.

Does temperature kill hardware? I wasn't very careful with my first laptop (a 2007 model with a dual-core CPU and a discrete GPU), and often used to run it on my lap with the fans blocked. It regularly got up to a sustained temperature of 96c. And it ran fine for 5 years (used for several hours most days) before the screen gave out. It actually still runs fine now with an external display, although it rarely gets used.

> "if hardware is too hot for you to hold your finger on it indefinitely"

That's only like ~45 °C or so.

More like 48, but yeah.

Enterprise networking chips happily run 80C+ junction temperature (much too hot to touch) for the part lifetime - which is 10+ years.

There are plenty of things that damage chips, especially those at 7nm and beyond but this too-hot-to-touch "rule" is rubbish.

Is this an exaggeration, or do you genuinely want to run something on the same hardware for the next 100 years?

If so, why? I can't imagine a workload that is so unchangeable in its nature, especially considering the world we live in and how often storage media changes that I'd want to plan ahead for 100 years.

A museum obviously

Things are about to not change as fast.

My drives are from 2011, peak SSD at 50nm (100.000 writes per bit).

My CPU/motherboards are from 2017, I think 2Gflops/watt is about where peak CPU is at.

Time will tell... but there is no point waiting for improvements now, time to go all in!

Why would that be “peak SSD” or “peak CPU”? Your CPU is already way behind the latest processors in both performance and performance per watt.

> I'm planning on running my machines for 100 years!

Heh, that's an interesting thought that i've also sometimes had.

For doing something like that, the hardware would need to be way more resilient and have failover for various components - for example RAID for the HDDs/SSDs and some mechanism for clustering and failover for apps and other stuff you'd want to run on them. But even with those in place, i'm not sure that the hardware that's available to us wouldn't just die long before the 100 year mark.

Anyone have any idea what are the oldest computers presently in continuous use? Best i could find was this, but it was just turned on for once, instead of being continuously working: https://www.smithsonianmag.com/smart-news/watch-the-worlds-o... Apart from that, all i can think of are mainframes and such.

I really doubt that any piece of currently modern hardware could last 100 years without becoming some hard to understand mess that's incredibly out of touch with the OSes and paradigms of the future. What would you even run on it? Debian? FreeBSD? Haiku? Would there be anyone to debug Python 2 or Python 3 in 100 years? What about Java, .NET, PHP, Golang, Rust, C++ or even C? Considering that many of those ecosystems are more and more migrating to integration with the Internet, especially for the dependencies, what are the chances that any of the software will survive that long?

I'm going to not use raid and instead mount the drives "manually" (via fstab) because I heard som nasty things about raid.

Instead I made my own async. distributed database: http://root.rupy.se

That way I can just fix the drives as they fail.

As for OS/languages I'm 100% sure linux + Java is the final server platform for eternity.

Nothing has improved for 5+ years since NIO got epoll!

The io_uring and potentially user network/disk stuff to work around the kernel waste (in my case ~30% of the CPU) might happen but also might not...

> Anyone have any idea what are the oldest computers presently in continuous use?

Voyager 1 and 2 are good candidates. In the end, they'll probably shut off because their generators won't provide enough power to operate their computers after 50 years or so.

I still have Pentium 4 running at one location. Sometimes the temperature is scary to look at, but it doesn't care. Those things were made to last

What temp. you get?