I feel like the entire "PC enthusiast" review space has dropped the ball on hardware vulnerabilities. Reevaluating performance between microcode and OS patches is an afterthought, and when a new CPU hits the market the numbers are presented without the obvious disclaimer that these performance gains may evaporate within months.
Some even perpetuate the "only relevant to datacenter" myth despite the fact that security researchers have shown to be able to exploit these vulnerabilities with JavaScript in the browser.
Even though I'm personally in the fortunate position not to have any reasonable exposure to these vulnerabilities, I wouldn't be particularly worried even if this wasn't the case. It's been well over a year since Meltdown and Spectre came out and there still hasn't been a single case of anyone successfully using these vulnerabilities to productive ends in the wild that I know of. Obviously, cloud computing vendors need to pay attention and there are legitimate business concerns that are affected by this, but insofar as personal computing goes? If people persist in the ridiculous notion that constantly running completely arbitrary code in naive sandboxes is a great idea, I imagine there will eventually be issues, but so far the issue seems to be vastly overblown in the popular media.
I fully support a users right to bypass these mitigations, and you're correct that your typical desktop user, at least today, isn't a target. But it seems odd that websites dedicated to performance computing have a blindspot to how automatically installed updates will impact performance.
It's quite easy to sensationalize benchmark results even unintentionally. The average reader of PC hardware review sites is totally willing to latch on to a microbenchmark result that shows a 20% performance drop and claim that it's disastrous for performance, even if the actual added delay to real-world operations is a fraction of a millisecond and thus will almost never cause the result of your user input to be delayed by even a single frame. There's a certain degree of irresponsibility in publishing results that you know will be taken out of context by almost everyone who reads them. I've discontinued benchmarks in the past because it was frustrating seeing readers pretend like they show a meaningful difference between products when the reader's workload never comes close to the workload represented by that benchmark.
That would kind of defeat the point.
If they published a benchmark in the past and don't bother to correct the benchmark when it becomes out of sync with reality - that is just bad journalism.
Nobody is saying you should go and cherry-pick benchmarks after the mitigations hit, but you should definitely check the benchmarks you already published once.
These sites can and should expect an informed reader.
In any case: Leaving wrong information up uncontested helps neither "experts" nor laymen.
No, and you should know better.
> If they published a benchmark in the past and don't bother to correct the benchmark when it becomes out of sync with reality - that is just bad journalism.
Proper practice is to publish the full test conditions, including software, firmware and nowadays also microcode versions. The availability of newer versions does not make older results any less true.
At AnandTech, we make all reasonable attempts to keep a thorough database of older hardware tested on newer benchmark suites, but the time this requires means we cannot re-test everything multiple times per year. I have over 200 SSDs and counting in the collection, and that test suite is over 30 hours long. The collection of CPUs is much larger. GPU reviews typically have fewer back-catalog hardware entries because updating to new drivers a few times a year is often unavoidable. You can browse the results for current and previous test suites at https://www.anandtech.com/bench/
> These sites can and should expect an informed reader.
You don't read the comments as often as we do.
This behaviour angered me no end. I wasted significant time looking for workarounds, and deleting the microprocessor driver was the only way. I wonder what fixes I've now nixed, but there was seriously no need for Intel to kill my overclock.
On a couple of occasions Intel have pushed updates which have reset my fix. Dear $deity .. my next PC will be AMD for sure.
Considering we are referring to attacks that can bypass your PC's security, "prudence" is a better word than hysteria.
Yes, if they are left alone, it is the "end of the world".
They can be used to make any modern OS and browser as full of holes as Windows 98.
That statement can be made about any vulnerability whatsoever. The merit of any mitigation can only be determined by a cost/benefit analysis that takes into account the potential impact of the vulnerability as well as the very real costs of mitigating it.
> Yes, if they are left alone, it is the "end of the world".
No offense, but this is exactly why the word "hysteria" seems far more appropriate than "prudence". Not a single one of these vulnerabilities has been used to cause any measurable damage anywhere that we know of, whereas the mitigations deployed have significant costs that everyone must pay. Despite this, emotional "the sky is falling" type pronouncements are far more common in the media - even the ostensibly technical press - than attempts to rationally weigh the costs and benefits of any particular approach to the problem.
That's like saying: "nobody was drowned that we know of, whereas there was a significant cost to building the dam that everyone paid". (And also not dissimilar to arguments about doing no major industry/lifestyle changes regarding climate change).
It's exactly because there were mitigations relatively quickly deployed that we didn't have a "hack em all" exploit doing the rounds in hundreds of millions of devices. The difficulty of exploiting also gave some leeway to deploying those mitigations.
It is very dissimilar indeed - the sentence you quoted does not constitute an argument by itself. It is an observation regarding the present state of affairs (which you have not disputed), which to me indicates a need to take a breath and do a reasoned cost/benefit analysis as opposed to the hysterical "this must be fixed at any cost, externalities be damned" mindset that is fairly common in many circles.
If you really want a climate change analogy, though, consider this - however many mitigating workarounds you invent, as long as speculative execution exists there will always be side channel attacks, and eventually some of them will probably succeed to some extent. Perhaps, as you noted, some major industry/lifestyle changes are indeed in order - people could stop living in the delusion that a perfect sandbox is possible and realize that arbitrary code execution will always entail risks. Rather than turning every website into a potential security risk, perhaps it is our approach to software (rather than hardware) that needs re-evaluation.
That's putting it lightly. Exploiting Spectre to get private data is difficult. Turning that into a privilege escalation is exponentially harder, so any "hack em all" exploit on hundreds of millions of devices would have needed an entirely unrelated mechanism for spreading.
No need, if you really want to disable all mitigations, including future ones, use mitigations=off.
https://www.phoronix.com/scan.php?page=news_item&px=Spectre-...
Why would the attackers let people know they've successfully exploited?
If you want useful benchmarks that show the performance impact, go to phoronix.
"PC Enthusiast" websites care about gaming performance and single user desktop performance, and always have. This has been the same since I started following things when the fastest CPU available was a 300 MHz Pentium 2. Imagine how amazed we all were by the 1 GHz Slot A Athlon.
Because Phoronix cares is outside the argument.
Edit: I took a look and it seems they don't ask/record that info. I apologise for the offence this idea seems to have caused someone.
'PC enthusiast' is such a blanket term to start with, so applying a blanket statement to such a group is obviously doomed to failure from the very start.
From what i've seen web browser teams have taken the recent risks extremely seriously - I have sure had a worse track record of infection via downloading and installing software versus visiting sites with JS running
Not really. They didn't even properly apply band-aids.
Chrome and Firefox disabled number of features, that allow Javascript code to create high-precision timers. This makes exploiting slightly more difficult, but the gaping hole is still there — there is infinite number of ways to create a high-precision timer, just not as obvious as closed ones.
Chrome has enabled Site Isolation on desktop, but haven't done it on Android (presumably, because of associated increase in memory consumption).
All major browsers still allow Javascript to run in background, create CPU threads and consume unrestricted amount of CPU time. I don't believe, that any of them have mounted instruction-based defenses (lfence etc.), but I may be mistaken here.
The recent CPU vulnerabilities aren't uniquely bad for Javascript specifically. They're bad for wanting to run unprivileged code. Javascript in regular web pages just happens to be the most obvious example of sandboxed code in desktop computers.
In fact, I would bet that most professional software engineers could not correctly explain Spectre, Meltdown, and Zombieload without making at least a few mistakes.
Anand has been working in Apple for a few years now, I wonder what has he been up to.
Microcode benchmarking is not the hill Intel wants to die on.
And if two competing processors are close enough in performance for these mitigations to change which one comes out on top of benchmark charts, then other factors like price, power consumption and IO capabilities are probably a much bigger deal at that point than minor CPU performance differences.
Most if not all of our benchmark suites have been updated to include at least the early Spectre/Meltdown/et al. mitigations, and new CPUs are being tested with the microcode they launch with.
https://www.tomshardware.com/news/intel-cpu-microcode-benchm...
If you don't sign that NDA you can't buy the CPUs from intel to resell them. Even if you are able to buy the CPUs from them, there's no guarantee that you'll buy from the list price or get the discounts for big, prestige projects which require tenders.
It's a deep and ugly rabbit hole.
And a minute of googling brings up multiple articles about Spectre/Meltdown including two comparisons. https://www.anandtech.com/show/13659/analyzing-core-i9-9900k...
https://www.anandtech.com/show/12566/analyzing-meltdown-spec...
What about the Ryzen 3000 line up? The benchmark leaks make it seem like it is going to be a huge improvement and AMD isn't susceptible to Zombieload.
Looking forward to a less complex architecture even if it means cutting me off at the knees with execution speed (for a few years):
RISC-V
Because the future is looking increasingly in-order high-core-count with hard partitioning between security contexts.
It's typical for news sites to report individual announcements, with little or no analysis; this is fair, as long as the post clearly specifies its nature (which, in this case, does).
Anandtech did something very interesting actually on the Intel subject, which I didn't see on other sites - it made an article about the performance of the i9-9900k locked at its nominal TDP (95W), which showed very significant losses.
They also had the Intel series 6 launch where they praised the 6600K and compared it to the 2500K to show “massive” improvements over the years. This while all the other websites noted “minimal speed boost for too big price”. Perhaps both true but the spin on it makes all the difference when showing the intention.
AT shows quite the Intel bias. And it’s not the Intel part that bothers me, it’s the bias part. They go out of their way to make Intel look better without outright lying, just selectively presenting the truth in a way that shines a much better light on Intel. This for me casts doubt on other articles.
I’m glad Andrei Frumusanu’s mobile reviews still have a home, being the best I have seen on the entire internet. But that’s the only segment on AT where I can be reasonably sure about impartiality.
The Spectre and Meltdown vulnerabilities made quite a splash earlier this year, forcing makers of hardware and software to release updates in order to tackle them.
https://www.anandtech.com/show/13400/intel-9th-gen-core-i9-9...
Do you run everything with administrative privileges?
https://www.tomshardware.com/news/intel-cpu-microcode-benchm...
Then there is the "Intel" benchmarks as usual [1] on GPU. Trying to suggest the 2 CPU were both running at 25W TDP to give a "fair" comparison, without mentioning the Ice-Lake U CPU were running with 50% more memory bandwidth vs the AMD Ryzen. And we know Graphics Benchmarks do depend a lot on memory bandwidth. The memory used was somehow mentioned in Toms or Other Sites but not Anandtech. ( Although none of them had mentioned the bandwidth difference, it was up to the reader to work them out )
Anyway none of these Consumer CPU upgrade interest me anymore ( Although any improvement to iGPU would be great ) I am eagerly waiting for a 2S - 128 Core EPYC 2 on a Server or AWS to play around with it.
[1] https://www.anandtech.com/show/14405/intel-teases-ice-lake-i...
Edit: And the lesson here, never trust a single news source. Always have a few option opened and fact check yourself. ( If you have the time )