News for hackers. Stuff that matters.
Re:Hertzbleed Attack by MrWiffles on Tuesday June 14, 2022 @02:02PM (#31743387)
Brilliant approach, really. Never occurred to me to try something like this!

Are you affected? Very likely. What can you do about it? Nerf your CPU performance by disabling "turbo boost" or equivalent. Should you do it? Probably not unless you're particularly vulnerable (journalist, human rights activist, etc.)

One thing I found interesting that may get changed later, so I'm documenting it here, is in their FAQ they say:

> Why did Intel ask for a long embargo, considering they are not deploying patches? > > Ask Intel.

So Intel did ask for a long embargo, then apparently did nothing about it. My guess is they investigated "can we actually mitigate this thing with a microcode update?" and arrived at the conclusion after actually trying - or possibly after external influences were exerted (you be the judge) - that no, there's not much you can really do about this one.

Later in the document another FAQ says:

> [...] Both Cloudflare and Microsoft deployed the mitigation suggested by De Feo et al. (who, while our paper was under the long Intel embargo, independently re-discovered how to exploit anomalous 0s in SIKE for power side channels). [...]

Which is again telling us that there indeed WAS a long embargo placed on this research by Intel.

Only mentioning this here just in case the PR spin doctors threaten the researchers into removing mention of Intel on this one. Which honestly I hope doesn't happen because my interpretation is that Intel asked for that long embargo so they could investigate really fixing the problem (state agencies have more methods at their disposal and wouldn't need much time to exert influence over Intel if they decided to). Which speaks well of them IMO. But then again, not everybody's going to come to that same conclusion which is why I'm slightly concerned those facts may get memory-holed.

Re:Hertzbleed Attack by Terretta on Tuesday June 14, 2022 @07:49PM (#31747480)
> So Intel *did* ask for a long embargo…

> … again telling us that there indeed WAS a long embargo placed on this research by Intel.

These are worded as if this wasn’t clear? No guesswork, the article states it plainly:

”We disclosed our findings, together with proof-of-concept code, to Intel, Cloudflare and Microsoft in Q3 2021 and to AMD in Q1 2022. Intel originally requested our findings be held under embargo until May 10, 2022. Later, Intel requested a significant extension of that embargo, and we coordinated with them on publicly disclosing our findings on June 14, 2022.”

Re:Hertzbleed Attack by gralx on Wednesday June 15, 2022 @10:16AM (#31753264)
You missed this part:

> Only mentioning this here just in case the PR spin doctors threaten the researchers into removing mention of Intel on this one.

Re:Hertzbleed Attack by Terretta on Wednesday June 15, 2022 @04:54PM (#31759356)
I didn't.... Continue the quote:

> Only mentioning this here just in case the PR spin doctors threaten the researchers into removing mention of Intel on this one. Which honestly I hope doesn't happen because my interpretation is that Intel asked for that long embargo...

Key part of that closing paragraph:

> ... my interpretation is that Intel asked for that long embargo ... and ... not everybody's going to come to that same conclusion...

If mentioning here to mitigate spin doctoring, the important thing to record in case Intel made them remove it would be the explicitly not open to interpretation paragraph I cited above.

Re:Hertzbleed Attack by synctext on Wednesday June 15, 2022 @04:49AM (#31750565)
The scientific article URL is broken, see correct link:

https://github.com/FPSG-UIUC/hertzbleed/pull/4

19 pages of detail, "Hertzbleed: Turning Power Side-Channel Attacks Into Remote Timing Attacks on x86 (USENIX 2022)"

Re:Hertzbleed Attack by xoa on Tuesday June 14, 2022 @02:36PM (#31743896)
>What can you do about it? Nerf your CPU performance by disabling "turbo boost" or equivalent.

Eh? Doesn't this require an attacker to actually be able to talk to the targeted system? You mention classes of actor ("journalist, human rights activist, etc") that aren't online public service providers or at least perhaps shouldn't be. Private devices connecting out to the greater net are effectively universally behind firewalls and often (CG)NAT as well and not unilaterally addressable the other direction. For private services, access should be exclusively via VPN (direct WG or mesh like nebula or something) with PSK alongside public keys and perhaps something like port knocking in front as well.

Same as with other hot class side channel attacks, the attacker does in fact need to be able to get the targeted system to run something somehow. So the most basic thing to do about it is simply not allow attackers to do that. The players for whom this is fundamentally impossible are general public online service providers, but that isn't the class most people or businesses fall into. If attackers are getting sensitive servers to respond to arbitrary code then they have already gotten access credentials of some kind.

Re:Hertzbleed Attack by BoppreH on Tuesday June 14, 2022 @03:20PM (#31744630)
> the attacker does in fact need to be able to get the targeted system to run something somehow

Unfortunately that includes Javascript, and now that affects virtually everybody. Speculation: if you can find a Javascript call that uses protected keys, you might be able to extract secrets from that route.

Re:Hertzbleed Attack by xoa on Tuesday June 14, 2022 @03:58PM (#31745209)
OK, but can you walk me through the threat model here? This isn't a rhetorical question, it's easy to see how servers in general and shared hosting, colocated VMs etc in particularly might theoretically face a threat here, I'm just trying to get a better understanding of how GP would be correct for end user devices. The individual in question on the smartphone or computer specifically chooses to initiate a connection to a desired web server, and you're imagining said server was hacked or otherwise compromised/untrustworthy and begins running Hertzbleed via JS, and that JS has the precision in practice for it to work. So vs other RCEs and such what is the path here? What short secrets (AFAICT these attacks aren't about pulling gigabytes of RAM but a few bits of key stuff) is it going after that, after having obtained them, represent a serious compromise by themselves or allow further escalation? Nothing done on the user device is going to affect the network hardware they're going through, and this attack is about reading not writing, so what is the attacker getting that will then let them break out of the web broswer/OS sandboxes and start pulling more the other direction?

I can see that if other persistent access credentials were sitting unencrypted in memory that could be an issue if the attacker can use those to hop into other services the user has access to, but that's manageable via care with device usage, proper MFA etc right? Or I can see how true external IP address might be a dangerous short secret for someone trying to hide (vs merely secure) via VPN. But I think in those cases externalizing the VPN to the network level is a good idea anyway since being 100% sure of zero leaks in a modern OS/application stack is challenging, and then the user device doesn't have to know anything about the real IP at all. JS also already allows a huge amount of fingerprinting, so if someone is allowing JS to run and is deadly worried about privacy they must be thinking about mitigations there already.

Again, not at all denying that incredibly tricky stuff can be done using little leaks like this to escalate in surprising ways. But for a dedicated web browsing device with only outgoing and session access to WAN, likely though a VPN but not necessarily on-device, what new threat scenario is this such that completely disabling dynamic frequency is the only response? Although I suppose for a dedicated web browsing device using tor browser or the like disabling that might not actually be a big deal anyway.

Re:Hertzbleed Attack by BoppreH on Tuesday June 14, 2022 @04:39PM (#31745742)
The exploit comes from a hacked server, a bad ad, social engineering, etc.

As for the attack, imagine a browser that encrypts local storage with a system key. If I understand correctly, by storing different patterns of bits, Hertzbleed might be able to extract the system key from the timings to save data.

This might sound very theoretical, but modern OS'es (and password managers) have lots of keys like that. There's a good chance one or more of them are reachable from Javascript. And that's just what popped in my mind in two minutes, I'm sure red teams will have better ideas.

The scary part is that this is another attack in the same ugly class as Meltdown and Spectre, where the antidote is nearly as damaging as the poison.

Re:Hertzbleed Attack by lossolo on Tuesday June 14, 2022 @06:51PM (#31746988)
> imagine a browser that encrypts local storage with a system key. If I understand correctly, by storing different patterns of bits, Hertzbleed might be able to extract the system key from the timings to save data.

Don't you need precise clocks for this in JS? The ones that were disabled in browsers after Meltdown/Spectre.

Re:Hertzbleed Attack by mintplant on Wednesday June 15, 2022 @12:43AM (#31749199)
A lot of that has since been rolled back as different mitigations have been deployed against Spectre.
Re:Hertzbleed Attack by slimsag on Wednesday June 15, 2022 @01:15AM (#31749371)
Source? The MDN is very clear about precise timing not being available due to Spectre mitigations.
Re:Hertzbleed Attack by thepangolino on Wednesday June 15, 2022 @04:44AM (#31750539)
>by storing different patterns of bits, Hertzbleed might be able to extract the system key from the timings to save data.

Isn't that practically impossible given the amount of random software running on a regular computer nowadays? This is not based on hdd write speeds but processor timings which are affected by every single cat video you are watching.

Re:Hertzbleed Attack by neurostimulant on Tuesday June 14, 2022 @05:50PM (#31746499)
Many password managers are using end-to-end encryption and have browser extensions written in js. It would be bad if hertzbleed can be used to extract keys used by those password manager extensions.
Re:Hertzbleed Attack by mgiampapa on Tuesday June 14, 2022 @05:20PM (#31746242)
On most shared infrastructure would be even harder to exploit. In an ideal world you are sharing the infrastructure to maximize CPU and other resource utilization. When running workloads on VMWare for example, the best practice is to disable deep C states and not allow the CPU to dynamically scale down. This prevents all kinds of clock drift issues in guest VMs that expect a CPU cycle to be relatively constant.
Re:Hertzbleed Attack by angry_octet on Friday June 17, 2022 @09:31AM (#31777874)
This is not about power saving, it is dynamically, on the granularity of individual instructions, balancing the thermal dissipation available across cores. When some cores are relatively idle (e.g. waiting on cache fill) it uses the available amps to run other cores at "turbo boost" frequencies.

For some single threaded code we force other cores to idle so that one thread can get maximum cache and maximum frequency. A similar approach could be used to minimise the power analysis signal.

Re:Hertzbleed Attack by progval on Wednesday June 15, 2022 @02:08PM (#31756881)
untrustworthy server that runs malicious JS: potentially half the links one clicks on HN or Reddit

short secrets which aren't mitigated by MFA: session cookies, TLS client certificates, secret keys of e2ee IM apps (eg. Element), Zerobin URLs, ... Maybe even TLS session keys?

Re:Hertzbleed Attack by userbinator on Tuesday June 14, 2022 @08:11PM (#31747642)
Just like Spectre/Meltdown, it assumes you have an idea where to extract the secrets from, and more importantly what they secure. A string of random bytes is worth nothing to someone who doesn't know what they're the key to.

...and someone who is running JS is probably also running tons of other JS, adding even more noise to what already exists.

Re:Hertzbleed Attack by bobkazamakis on Tuesday June 14, 2022 @03:49PM (#31745063)
> the attacker does in fact need to be able to get the targeted system to run something somehow

>Unfortunately that includes Javascript, and now that affects virtually everybody.

Debatable -- not because of the goofballs who insist on never running javascript, but because it doesn't have the necessary clock precision available.

Re:Hertzbleed Attack by fpoling on Tuesday June 14, 2022 @05:34PM (#31746369)
When Spectre came it turned out that it was very straightforward to implement the relevant attacks in JS. A script can use workers with shared memory access to monitor execution and get a timer with less than 100ns resolution. As the result the shared memory were disabled. Later under the presumption that relevant issues were mitigated, the shared memory was re-enabled again.

So I wonder if the shared memory will be disabled again as it may allow to monitor frequency changes.

Re:Hertzbleed Attack by jabart on Tuesday June 14, 2022 @06:56PM (#31747028)
My understanding was that the timer precision was limited and that was never re-enabled.

From MDN.

"It's important to keep in mind that to mitigate potential security threats such as Spectre, browsers typically round the returned value by some amount in order to be less predictable. This inherently introduces a degree of inaccuracy by limiting the resolution or precision of the timer. For example, Firefox rounds the returned time to 1 millisecond increments."

Re:Hertzbleed Attack by josefx on Wednesday June 15, 2022 @06:42AM (#31751172)
If I remember correctly they also had to disable shared array buffers because a thread just incrementing a shared value can be used as timer.
Re:Hertzbleed Attack by contravariant on Tuesday June 14, 2022 @07:34PM (#31747349)
Could anyone explain how rounding the timing protects against just running the thing you want timed 1000 times to get microsecond precision?
Re:Hertzbleed Attack by thristian on Tuesday June 14, 2022 @10:56PM (#31748628)
Same as any security: making an attack more expensive to mount means people are less likely to try it. If high-resolution timers allow you to mount an attack in the three minutes the target takes to read a listicle page, then rounded timers require the target to keep the page open in the foreground for 3000 minutes, or 50 hours. That's much more difficult to do.
Re:Hertzbleed Attack by vajrabum on Tuesday June 14, 2022 @10:05PM (#31748324)
Isn't rounding pretty much the same thing as throwing away some precision in this case? So if I drop some (enough) bits and add .5 then no amount of averaging is going to recover the lost precision. Or maybe I misunderstand?
Re:Hertzbleed Attack by danielheath on Tuesday June 14, 2022 @11:22PM (#31748774)
Example: You need to know how whether an operation takes 30 or 31 milliseconds, but your timer is rounded to the nearest 100ms, so you just get 0 or 100.

If you repeat the operation 100 times and time how long that takes, you should either get 3000 or 3100 milliseconds.

Re:Hertzbleed Attack by usrusr on Wednesday June 15, 2022 @03:26AM (#31750076)
..and a dramatically increased likelihood of something unrelated going on in the system ruining your results. It's easy to dismiss mitigations like this as "not a solution", but at least those that add uncertainty to sidechannel attacks seem to complement each other quite nicely. Uncertainties don't add, they multiply.
Re:Hertzbleed Attack by SOLAR_FIELDS on Tuesday June 14, 2022 @09:51PM (#31748243)
I know next to nothing about the internals of how the rounding code works, but couldn't randomly choosing to round up or down mitigate this somewhat?
Re:Hertzbleed Attack by arpafaucon on Wednesday June 15, 2022 @02:09AM (#31749660)
If you round up and down with the same probability, it will get canceled on the long run.

I suppose this does mitigate the risk a little, but risks breaking other things. For example, you don't have the simple logic sequence: a < b => round(a) < round(b)

So I'd say "not worth it"

Re:Hertzbleed Attack by rswail on Wednesday June 15, 2022 @04:31AM (#31750474)
Minor correction:

iff |a| < |b| then round(|a|) < = round(|b|)

Re:Hertzbleed Attack by icedchai on Wednesday June 15, 2022 @05:40PM (#31759805)
There were proof of concepts. Were there ever any actual real world attacks?
Re:Hertzbleed Attack by jnordwick on Thursday June 16, 2022 @02:36AM (#31763065)
never. nothing ever recovered in a root kit. nothing seen for sale. even the original POCs were horrible biased and didn't work. simple moving the targeted buffer around in memory would have make it virtually impossible to exfiltrate it.
Re:Hertzbleed Attack by smaudet on Wednesday June 15, 2022 @03:03AM (#31749963)
I don't have a problem running JS but it is getting to the point where, if you can't prove you are JS worth running perhaps the browser should refuse to.

Why have we gone from 'oh only run programs you trust not anything from the web' to 'oh just run every bit of bloatware out there any time you move around the web?

99% of JS should not exist, 0.9 % of it does anything useful and the other .1% is straight up malicious...

Re:Hertzbleed Attack by joveian on Wednesday June 15, 2022 @03:26PM (#31758116)
I do this with uBlock Origin. It is easy and I'd say less impactful than blocking third party domain (de-CNAMEd) CDNs by default (that I also do). Some sites just don't show anything without javascript, though, so it works best if you are often willing to just ignore those sites (not common but not all that uncommon either).
Re:Hertzbleed Attack by hulitu on Wednesday June 15, 2022 @02:35AM (#31749794)
> So Intel did ask for a long embargo, then apparently did nothing about it.

It was not for them but for their 3 letter customers. 0 day unpatched for 1 year, what can be better than that ?

Re:Hertzbleed Attack by Proven on Wednesday June 15, 2022 @04:33AM (#31750487)
Doesn't make any sense - there's no benefit to Intel (why would anyone order more CPUs?) and there's much more downside from all other customers.
Re:Hertzbleed Attack by cestith on Wednesday June 15, 2022 @10:03AM (#31753047)
It gives certain customers time to mitigate - and certain others time to exploit - the issue before the embargo is lifted. It maintains the customer/vendor relationship. It also allows Intel, even if current products are still impacted, a head start on R&D for future products before disclosure. Remember Intel also has their own Linux, has their own compiler suite, and directly supports development of Linux and Windows (and probably macOS for Apple) for their industry partners. So they could have been working on figuring out software mitigations during that time which they can now share.
Re:Hertzbleed Attack by taneq on Wednesday June 15, 2022 @07:24AM (#31751432)
Revolving door? Or even just someone looking to retire or otherwise cash out son who didn’t want the stock price to drop yet?
Re:Hertzbleed Attack by mr_toad on Tuesday June 14, 2022 @08:11PM (#31747641)
> Are you affected? Very likely.

How long is it likely to take to apply this to cryptographic protocols that people actually use? The protocol they attacked is rather obscure: https://en.wikipedia.org/wiki/Supersingular_isogeny_key_exch...

Re:Hertzbleed Attack by willbudd on Wednesday June 15, 2022 @02:59AM (#31749936)
To demonstrate the effectiveness of the attack they deliberately chose SIKE as a protocol that was thought to be very resilient against side channel shenanigans.

If anything, real-world contexts are likely to be lower hanging fruit.

Re:Hertzbleed Attack by api on Tuesday June 14, 2022 @02:04PM (#31743417)
> What can you do about it? Nerf your CPU performance by disabling "turbo boost" or equivalent. Should you do it? Probably not unless you're particularly vulnerable (journalist, human rights activist, etc.)

The most likely to be targeted (and probably easiest to target) systems are probably cloud hosts. This might be an argument for disabling frequency scaling and fixing clock speed on cloud VM hosts or bare metal servers.

Less of a performance hit there too since those tend to run at a sustained max anyway, and turbo boost can be problematic in those environments due to heat. It can reduce overall throughput in a sustained load scenario.

Re:Hertzbleed Attack by jessermeyer on Tuesday June 14, 2022 @02:33PM (#31743849)
There's so much variation (read, noise) intrinsic to response times for network requests to be satisfied on most cloud hosts anyway that I'm very skeptical about any practical attacks being made in the short term.
Re:Hertzbleed Attack by wmf on Tuesday June 14, 2022 @02:40PM (#31743973)
For one request, yes. For statistical analysis of many requests, no. People keep extracting secrets from very noisy and weak signals.
Re:Hertzbleed Attack by mike_hearn on Tuesday June 14, 2022 @03:36PM (#31744887)
In lab conditions, yes. In this case:

"Our attack is practical; an unoptimized version recovers the full key from a CIRCL server in 36 hours and from a PQCrypto-SIDH server in 89 hours ... The target server and the attacker are both connected to the same network, and we measure an average round-trip time of 688 µs between the two machines."

Note that:

• The server in this case does absolutely nothing except use the cryptographic library. Would it work on a real server that actually does something useful with the requests? We don't know, the paper doesn't try that.

• We aren't told if it works if other people are using the server simultaneously.

• They show the attack against obscure post-quantum algorithms nobody actually uses (as far as I know). Why not RSA or ECDSA or something more standard? Presumably they don't have a technique that works on those, as otherwise it'd have been a big upgrade to their paper.

• What about if you aren't running your attack physically right next to your target? Is <1msec of latency what people think of when they hear "remote attack"?

I'm not hugely surprised Intel has limited themselves to issuing guidance. This paper continues a trend that's emerged since the first Meltdown/Spectre breaks in 2018 in which attacks become ever more convoluted, theoretical and unlikely to work outside of a lab yet they're all presented as equally important by the academics who develop them. I used to follow this area of research quite closely but eventually got sick of it. Way too many papers had some bizarre caveat buried deep in the paper, e.g. eventually I noticed that a lot of attacks on Intel SGX that claimed to leak cryptographic keys turned out to be using an extremely specific version of GnuTLS. I got curious why that might be and discovered that it was absolutely ancient, dating from many years before the papers were written. They were using it because that version had no hardening against side channel attacks of any kind whatsoever. Was that a realistic assumption to make for these attack papers? Probably not, but to notice this sort of trick you had to read well beyond the headlines.

I also remember some years ago, Google researchers got worried people weren't taking Spectre seriously enough, so they released a demo that claimed it would show a Spectre attack in action inside the browser. I was keen to see this because so many posited attacks seemed to rely on extremely specific situatons that didn't seem particularly plausible in the real world. I visited it in Chrome on macOS, i.e. one of the most predictable hardware and software environments the developers could have, and it didn't work. Checked reddit, it was filling up with people saying it didn't work for them either.

In the ~5 years since these attacks came out and started being patched in software and hardware, have there been any real world attackers found using them? Maybe but I don't remember hearing about any. State sponsored attackers seem to be sticking with more conventional techniques, which probably says a lot.

Re:Hertzbleed Attack by api on Thursday June 16, 2022 @07:21AM (#31764708)
The obvious target for these is the cloud, especially second-tier cloud vendors more likely to be using "stock" KVM/XEN and therefore easier to target. The obvious target within these clouds would be cryptocurrency nodes.

I feel like if this had been exploited in the wild you would have already heard stories of people using it to zark someone's bitcoin off a Digital Ocean or Vultr node.

Re:Hertzbleed Attack by mike_hearn on Thursday June 16, 2022 @01:17PM (#31768558)
I don't think there's a way to apply this to cryptocurrency nodes, because they won't sign messages given to them by third parties over and over with their private keys (usually at least).
Re:Hertzbleed Attack by jessermeyer on Tuesday June 14, 2022 @02:49PM (#31744130)
This is not inconsistent with what I said.
Re:Hertzbleed Attack by thfuran on Tuesday June 14, 2022 @03:23PM (#31744689)
Isn't it? It probably wouldn't require any novel statistical techniques.
Re:Hertzbleed Attack by jessermeyer on Tuesday June 14, 2022 @04:06PM (#31745332)
Novel statistical techniques is a different concern to practical attacks. (And I appreciate the relativity in what is meant by 'practical' -- nation state resources are in a distinct category of capability)

But I would like to see some statistical expectations on 'how long you'd have to wait on an average open network for each key bit to reach 95% confidence'.

Re:Hertzbleed Attack by FabHK on Tuesday June 14, 2022 @03:21PM (#31744660)
> Hertzbleed is a real, and practical, threat to the security of cryptographic software. We have demonstrated how a clever attacker can use a novel chosen-ciphertext attack against SIKE to perform full key extraction via remote timing, despite SIKE being implemented as “constant time”.
Re:Hertzbleed Attack by jessermeyer on Tuesday June 14, 2022 @04:37PM (#31745727)
Please. If you actually read the paper you'll come to learn that "practical" here means "we've conclusively shown under strict laboratory conditions that this works".
Re:Hertzbleed Attack by gpm on Tuesday June 14, 2022 @02:19PM (#31743627)
> since those tend to run at a sustained max anyway

Really? I've never been on the cloud-provider side of cloud computing, but every application I've developed that ran on the cloud was rarely if ever running at a sustained maximum of the resources allocated to it. We always wanted a buffer to be able to absorb load spikes and users performing unusually expensive actions.

Re:Hertzbleed Attack by bombcar on Tuesday June 14, 2022 @02:32PM (#31743832)
Dynamic scaling involves bringing the CPU frequency down but not off - you can get almost as much power savings for some loads by using the old HLT instruction, so your CPU/core is either at full speed or basically off.
Re:Hertzbleed Attack by depereo on Tuesday June 14, 2022 @02:54PM (#31744219)
I am on the cloud provider side; we would sometimes limit the upper and lower range of frequency but completely disabling scaling would be very unusual.
Re:Hertzbleed Attack by href on Tuesday June 14, 2022 @06:02PM (#31746608)
Same. Our compute hosts are generally not using 100% of their cores at all time.

There’s computes that are not full, computes that run rather diverse tenants, and even the fully utilized computes responsible for CPU optimized VMs have enough variance in their workload for frequency scaling to occur.

Re:Hertzbleed Attack by staindk on Tuesday June 14, 2022 @02:26PM (#31743757)
I think this means you were paying for the over-provisioning i.e. paying for a full CPU or baremetal server?

"The Cloud" is all about vCPU - "2 vCPUs" feels somewhat standard for a base-tier VPS... and 2 vCPUs means "2 virtual CPUs" or rather "roughly equivalent to 2 CPU cores" I think. I understand that jargon to mean they are always cramming 11 x 2vCPU clients onto 20 physical cores.

Re:Hertzbleed Attack by phamilton on Tuesday June 14, 2022 @02:31PM (#31743828)
Nah, vCPU is generally just hyperthreading. See https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/cpu-opti...

So you get 2 vCPUs per core.

Re:Hertzbleed Attack by staindk on Tuesday June 14, 2022 @02:44PM (#31744033)
Thanks for the link, that's great to know about AWS.

I don't think all other VPS providers are that good about things - googling around for some other definitions of vCPU (in VPS context) I see a lot of examples of 16 thread server CPUs handling "128 vCPUs".

Re:Hertzbleed Attack by ceeplusplus on Tuesday June 14, 2022 @02:32PM (#31743840)
No, 2 vCPUs is 2 logical threads, which is equivalent to a single physical core on x86. So yeah, they are cramming 11 x 2 vCPUs onto 20 physical cores. In fact, it's more like 20 x 2 vCPUs.
Re:Hertzbleed Attack by sofixa on Tuesday June 14, 2022 @02:31PM (#31743825)
Depends on the cloud - OVH and Digital Ocean yes, but AWS give you whole uncontested cores ( bar the t* instance line, but that's it's whole thing).
Re:Hertzbleed Attack by rhn_mk1 on Tuesday June 14, 2022 @02:31PM (#31743816)
That only means you were making space for others to phase in and use the remaining resources that you spared. You're not the one deciding which process sits on which resource, after all.

The only reason why resources might be left unused are usage spikes that all customers share.

Re:Hertzbleed Attack by sofixa on Tuesday June 14, 2022 @02:30PM (#31743813)
Depends on your workload, spot batch jobs would run at sustained maximum.
Re:Hertzbleed Attack by gpm on Tuesday June 14, 2022 @05:46PM (#31746472)
Absolutely, my observation is in a way that "most workloads don't seem (to me) to be like that".

The number of servers serving interactive queries (frontends, rest api servers, databases, etc) seems (to me) to greatly outnumber the number of batch jobs, and I've always seen those intentionally "over" provisioning CPU because otherwise you get latency issues if load increases at all.

I don't actually know that cloud providers don't either have some clever way around this (e.g. spending spare CPU cycles on some other form of work), or that it isn't the typical usage pattern, but I strongly suspect it.

Re:Hertzbleed Attack by bayindirh on Tuesday June 14, 2022 @02:58PM (#31744298)
> What can you do about it? Nerf your CPU performance by disabling "turbo boost" or equivalent.

A server running a multithreaded load is probably disabling turbo boost anyway because of the thermal load on the package already. Instead, you should disable speedstep and set your systems to maximum performance. However, this will increase the heat and your power bill considerably.

Re:Hertzbleed Attack by mcronce on Tuesday June 14, 2022 @03:48PM (#31745042)
I was thinking about busy servers running mixed workloads. I would think that, with the CPU running a bunch of workloads on different cores, context switching, etc, it wouldn't be a practical attack. Maybe that's incorrect.

Mostly idle servers are a different story, obviously.

Re:Hertzbleed Attack by bayindirh on Tuesday June 14, 2022 @04:03PM (#31745292)
Sometimes response critical VMs are pinned to the cores at the hypervisor level, and intel's chips support independent (frequency) scaling of CPU cores for some time.

In that scenario, mixed loads won't help. You'll have at least one pinned core, and it can scale relative to the VMs load (considering you also pin hypervisor cores, etc). So, it's possible to hit that pinned core and execute the same timing attack.

I know it's a niche scenario, but it's not an impossible or implausible one. Another possibility is the servers which fill critical roles, but they're idle or in a constant low-load state to have headspace for high loads. Again, attacking these servers are plausible. Considering these servers are not open to internet most of the time, we're bordering on corporate espionage, but it's not the subject here.

Re:Hertzbleed Attack by sterlind on Wednesday June 15, 2022 @01:18AM (#31749391)
unless you do something special, a lot of interrupts are handled by CPU 0. there's techniques like Receive-Side Scaling to balance this load across the cores but that's specific to NICs.
Re:Hertzbleed Attack by robocat on Wednesday June 15, 2022 @02:05AM (#31749639)
Is that because CPU 0 tends to be scheduled by default? Or is that because the CPU usually uses core CPU 0, and Linux schedules to different cores?

Redhat Linux docs[1]:

The /proc/interrupts file lists the number of interrupts per CPU per I/O device. It displays the IRQ number, the number of that interrupt handled by each CPU core, the interrupt type, and a comma-delimited list of drivers that are registered to receive that interrupt.

The default value for smp_affinity is f, meaning that the IRQ can be serviced on any of the CPUs in the system. To view use cat /proc/irq/32/smp_affinity for interrupt 32 as example. Setting this value to 1, like echo 1 >/proc/irq/32/smp_affinity, means that only CPU 0 can service interrupt 32.

[1] https://access.redhat.com/documentation/en-us/red_hat_enterp...

Re:Hertzbleed Attack by bayindirh on Wednesday June 15, 2022 @01:48AM (#31749530)
However, I can pin a VM to far away cores to CPU0 (e.g. socket 3, cores 20-23), hence isolating it from all the interrupt handling, and attack that VM instead, at least in theory, no?
Re:Hertzbleed Attack by mjevans on Tuesday June 14, 2022 @04:02PM (#31745268)
A spike performance up, random delay in down might be sufficient mitigation on a multi-tenant platform.
Re:Hertzbleed Attack by bayindirh on Tuesday June 14, 2022 @04:07PM (#31745345)
You can tune the normal on demand governor with a high hysteresis to keep the frequency up for a long time. On demand governor is already trigger happy enough to jump to maximum frequency with a slight increase in load, so one needs to add more delay in calming down step.

A random hysteresis is bad from a system responsiveness aspect, because frequency scaling is not free (in terms of time) when observed from the CPU perspective.

Re:Hertzbleed Attack by mackmgg on Tuesday June 14, 2022 @02:31PM (#31743821)
I've always turned off Turbo Boost on my Intel laptops anyway, because of the heat/battery hit. If I was doing something that I really wanted the extra speed and I was plugged in, I would turn it back on for that task, but I never really felt I was missing anything by having it off.

Considering how easy it is to turn on/off (on macOS at least I used Turbo Boost Switcher which added a button in the menu bar to toggle it) I don't think you would have a noticeable performance hit by keeping it off except when you need it.

Re:Hertzbleed Attack by bayindirh on Tuesday June 14, 2022 @03:03PM (#31744386)
Turbo Boost is not something you can reliably kick in any load spike due to its operation being constrained by the thread and thermal load of the CPU. Also, it's affected by the CPU instructions you're already running. AVX family is esp. power-heavy and thermal-heavy.
Re:Hertzbleed Attack by 2OEH8eoCRo0 on Tuesday June 14, 2022 @06:20PM (#31746747)
You can configure your CPU to always boost if you're really paranoid. It shouldn't run much hotter since the load doesn't increase, just the frequency.
Re:Hertzbleed Attack by zx8080 on Tuesday June 14, 2022 @10:32PM (#31748478)
Tell that to my intel macbook.

Seriously, using a popular Turbo Boost Switcher makes it a lot colder and live longer on a battery

Re:Hertzbleed Attack by throwaway742 on Wednesday June 15, 2022 @01:59AM (#31749601)
Then why isn't that the default setting?
Re:Hertzbleed Attack by smaudet on Wednesday June 15, 2022 @02:58AM (#31749933)
I wonder if this is such a widespread issue - frequency attacks mean the cpu must actually change speed frequently in order for any attack to occur.

For a laptop how I thought it worked was cpu frequently was constant depending on performance level - so cpu should shift from low to high depending on idle state...

For a server trying to aggressively save power, frequently changing speed per operation could leak this information.

Turbo boost I thought was a setting overlong periods of time to change power levels, if you change too frequently you don't actually save much power.

Re:Hertzbleed Attack by 2OEH8eoCRo0 on Wednesday June 15, 2022 @05:42PM (#31759815)
I didn't say temps or power won't rise, just that it's not a huge hit.
Re:Hertzbleed Attack by hsuduebc2 on Tuesday June 14, 2022 @07:44PM (#31747441)
I'm not getting the part how disabling turbo boost lowers your cpu performance. It should only significantly increase power consumption right?
Re:Hertzbleed Attack by STRML on Tuesday June 14, 2022 @07:49PM (#31747476)
Turbo will move the processor above the rated TDP when there is thermal headroom to do so. Turning it off means you'll max out at the rated TDP.

Now, TDP used to mean the max power of the chip, but as Intel's process failures left them holding the bag with no significant performance updates to speak of, they started overclocking their chips more and more so they could claim that the new gen was faster than the last.

Try turning off Turbo Boost on a 2020 i9 Macbook Pro - you actually get a usable machine with reasonable battery life with it off, instead of the hot toaster with 2hr battery life that Intel gave you. But it'll max at something like 2.2GHz when you paid for just over 4.

Re:Hertzbleed Attack by hsuduebc2 on Tuesday June 14, 2022 @08:14PM (#31747673)
Wow. How to they get their score in benchmarks? You can't "temporarily overclock" your cpu for too long right?
Re:Hertzbleed Attack by STRML on Wednesday June 15, 2022 @09:40AM (#31752729)
Correct. This makes benchmarks, at least on thermally limited machines like laptops, very unreliable. High-quality review sites like notebookcheck spend a lot of time dealing with this by doing prolonged benchmarks and measuring thermals.

And there's an honest question to ask: how do you use your computer? If you're just browsing the web 95% of the time and occasionally opening Word/Excel, then short bursts of high power when you need it is perfect. But if you run longer tasks like many programmers or artists do, these machines simply fall down in sustained use.

This is one reason why the M1/M2 architecture has been such a revelation for professionals who primarily work on laptops. It can run full-bore for hours, because the lower-end chips (which are faster than any Intel released at the time) barely hit 10W at max load.

Re:Hertzbleed Attack by Dylan16807 on Tuesday June 14, 2022 @10:21PM (#31748414)
You can keep it up if everything can handle the current and heat.

A typical Intel chip on default behavior will go to maximum boost, limited by watts, for about half a minute. Then it will drop to a lower number of watts. Note that base clock gets ignored here; in this mode the base clock is just a minimum promise.

Many desktop motherboards easily or even automatically remove the time limit.

Re:Hertzbleed Attack by jakogut on Tuesday June 14, 2022 @07:54PM (#31747520)
Disabling turbo boost/frequency boosting would actually _decrease_ power consumption, as well as performance. The idea with boosting is to allow certain cores to exceed the maximum frequency, so long as certain parameters such as package temp, core temp, and power usage are within certain thresholds. This allows workloads that don't push the entire CPU to its limits to run faster, as the few cores that are in use can run at higher frequencies and increased performance per core, at the cost of higher power usage per core and lower efficiency.
Re:Hertzbleed Attack by hsuduebc2 on Tuesday June 14, 2022 @08:13PM (#31747661)
Thank you for clarification. :)
Re:Hertzbleed Attack by Melatonic on Tuesday June 14, 2022 @07:06PM (#31747098)
Is that all you need to do? Because many overclockers permanently disable Turbo boost anyways so that they can run a higher clock ratio all the time (cant have turbo occidentally crashing your system once you have really overclocked it a lot). This does not of course disable the low power states for idle or low low load. I probably have Turbo disabled right now!
Re:Hertzbleed Attack by yunohn on Tuesday June 14, 2022 @02:42PM (#31744009)
> > Why did Intel ask for a long embargo, considering they are not deploying patches?

> > Ask Intel.

Indeed, I really found this unnecessarily snarky on their part. I don’t think Intel was acting in bad faith.

In my experience, security researchers are very /particular/. They like telling everyone that no matter what you do, you are vulnerable for umpteen reasons, whether practical or not.

Re:Hertzbleed Attack by AlexCoventry on Tuesday June 14, 2022 @06:08PM (#31746649)
If they didn't tell you about the risks and you got hacked, you'd also be asking why.