What people need to realize is that a P cores takes around 4 times the die area/transistors of an E core, and use probably 3 to 4 times the power of an E core, but performance is only around double that of an E core. So with the same area, you get of the 8P+8E you can get 10P cores, not 12, and the question is if a 10P (rather than a 16P or even 12P) core Alder Lake is better than a 8P+8E. (Btw, these ratios are pretty similar as those for the Firestorm/Icestorm cores in Apple's M1.)
If your workload is very parallelizable you get more bang from your silicon and Watts with E cores than with P cores. If you have only embarrassingly paralleled workloads, you're probably better off with 32E cores than with a 8P cores.
If you watch a video while writing a document in word, that's trivially parrallelizable.
That's a great point. At any point in time there are already dozens of processes running, of which only a fraction are tied to a GUI. Even GUI apps are currently resorting to launching multiple processes. For example, Firefox announced last year their new site isolation security architecture where they load each site as an independent process.
https://hacks.mozilla.org/2021/05/introducing-firefox-new-si...
Also, here's a link to a recent HN discussion on Firefox's recent announcement.
https://security.googleblog.com/2018/07/mitigating-spectre-w...
(Or it should - browsers are not always very good about this but the tools are there, they just don’t want to pay licensing fees to use them.)
I always cringe a bit when people insist they need 6 cores “so they can have discord and Spotify open while they game”. People don’t generally have a very good intuitive sense of just how little cpu time it takes to run those (a lot of times the “at the desktop” measurements are not even with the processor clocked up, it’s running in the lowest active power state), and there is almost always a little cpu headroom due to non-perfect game scaling. Context switching is cheap, your OS is running thousands of threads at any given time, but people don’t have a good intuitive sense of the overhead of context switching either.
Playing a video and writing a document is something that can be trivially done on a core2duo era processor, or the modern equivalent of a low-end dual-core nuc with e-cores (atom cores) like a J4005. You don’t even need performance cores for that.
The tradeoff Intel is making is that some tasks need performance cores because they can’t really be parallelized all that well, and embarrassingly parallel stuff is more efficient to implement with lots of smaller cores. In principle this is also more power efficient, like big.LITTLE in phones, but Intel actually seems to have fairly poor power scaling - they are space efficient but not all that power efficient. This is also partially due to the fact that the little cores share a core voltage with the big cores - the big cores need higher voltage to clock up and that kills efficiency on the little cores. At that point you may as well clock the little cores up too since they have to share the voltage anyway, and that’s an inefficient way to run them.
As Agner Fog mentions, the AVX-512 is also present in silicon yet disabled, which hugely blows up core size for almost no benefit (it’s dark silicon which does have some minor thermal benefits). Alder Lake is in many ways not that well-designed at a high level, but it is magnificently fast on those performance cores and the e-cores would be very nice if they weren’t conjoined awkwardly to the p-cores. Gracemont-based server and atom chips are going to be nifty when they finally arrive.
Apple is still way ahead with Icestorm though - those cores are absolutely absurdly tiny for the performance they give. They are 0.5mm2 each iirc, even considering the node shrink that is teeny-tiny, they are half as many transistors as a Zen3 core and almost the same performance (meaning much higher IPC) or something along those lines. Firestorm is neat but Icestorm didn’t get nearly the attention and it is really insanely fast for how few transistors it uses.
Playing a video and editing a document, sure. But change that to being on a video call and having a few moderately intensive web apps open (say Gmail, Google Docs and some kind of project management software) and your average dual core processor from a few years ago is going to start to struggle.
I do ALL my work on a t25 which is just about 5 years old now. This includes daily calls with dozens of video participants (ms teams and WebEx mostly) while I have chrome and Firefox open with multiple tabs including outlook and Gmail and slack, couple of MS words, couple of PowerPoints, Visio, etc. That's a dual core processor from more than couple of years ago and it's nowhere near capacity with that workload.
It does have lots of ram and Ethernet to residential Fibe 1gbps. But CPU just hasn't been an issue and I plan to work on this another 5 years
The real question is, are you one of the video participants too? Or are you just watching others but not sending any video of your own?
Heck I've run full PeopleSoft multi vm setups on my ancient t420 back in the day. A little CPU really goes a longer way than people realize.
I’m just using Nehalem and Conroe as a touchstone that people understand in terms of cpu performance. That is what you get in general cpu performance, but it’s also got better media support and crypto acceleration and so on, so it’s more usable today. Actually a closer look at the numbers puts it just below Sandy Bridge i5 actually.
AMD cpu users (non-apu) and RX 6500/6400 owners obviously need to look elsewhere however. NVIDIA’s 1030 chips are ancient pascal based crap but AMD dropping encoder support on a chip designed during the pandemic era of work-from-home is a bit tone-deaf.
(Yes, 6500 and 6400 were designed during the pandemic, the architecture took longer to develop but 6500 wasn’t the lead ship in its architecture, and taping out a new chip configuration takes much less time than designing a whole architecture from scratch… AMD definitely knew it would be used as a desktop chip too. It was a deliberate space-saving move in the face of their wafer shortages, not something they got blindsided by.)
My 2015 MacBook Pro with Broadwell i5 would indeed work "fine" for videoconferencing (in my case Google Meet), but it would also use most of a CPU core to do it. Which is fine if that's all you're doing. But if I wanted to share my screen and work on something else while doing so then it could quite quickly get to the point where I needed to start shutting down other programs, and even then it could cause lag when typing in less well-implemented apps.
My new MacBook with M1 Pro cpu it still uses most of a cpu core for videoconferencing, but it's a single E core and I have 9 others cores. Videoconferencing now makes zero difference to system responsiveness.
Most people won't need 8 cores, but I reckon the average office worker in 2022 would benefit from 4.
It browses the web perfectly fine, plays YouTube, and runs VS code, all that the same time. The only website I've notice it struggles on is Google maps, but Google maps has a 2D mode than runs great.
It did actually surprise me how well it runs and now I've been using it as my main machine for a few weeks.
I mean, they're not wrong. I have Discord sitting open at the login screen and it's burning 10% of a core. Run a few Electron apps and your core2duo suddenly becomes a core1uno.
> Playing a video and writing a document
Using what? VLC + notepad, or Chrome + YouTube + Google Docs? There's a huge difference.
Discord gets a core, Spotify gets a core, Chrome with a few webapps open gets a few cores. That leaves 1 or 2 for a game if you're lucky. God forbid you also try to stream to Twitch. I'll cut the rant short here. You're right though; you should cringe if they insist on 6 cores, instead of 8 or more.
Bloated software is the problem. The worst part is that efficient alternatives exist (Discord -> Ripcord, Spotify -> spotify-tui) but they're against ToS and can get you banned from both services. From what I hear Discord is a lot more likely to ban you. I've not heard of any Spotify bans. But as a user, even if you know the problem, your only choices are to risk an account ban, or buy an "overpowered" CPU to run your chat client.
EDIT: Am I wrong? If you're skeptical, go find a Core 2 Duo and try watching a YouTube video while you're in a Discord call, and let me know how it goes. Those are the kinds of apps people use these days, like it or not. Don't shoot the messenger.
That is what I said about “people look at the measurement at the desktop and freak out about single-digit utilization without even realizing their processor is sitting in the lowest frequency state”.
Is your processor even boosting all its cores when that 10% measurement is taken? In my experience usually it is not. Maybe different for Zen2/Zen3 but cpu measurement gets very complicated in general for those architectures since it can move between boost states so fast - it can boost so agilely that you can end up with garbage numbers because it’s moving between boost states faster than the task manager can enumerate the threads. It can move between boost states between the time you take the frequency measurement and the time you start enumerating the cpu load. This was a whole thing a couple years ago when people started looking at Zen2 and it turns out most third-party tools can’t handle this and only Ryzen Master gives you an accurate picture of what it’s doing.
In general it is such a small amount of cpu that it’s not worth caring about.
Also, typically while you game you’d have the overlay up and not the actual UI and that’s even faster - showing the login screen is likely slower than actually in game.
Load up a game and the cpu will be loaded up harder and will stay boosted and then take your measurement of discord.
People who are casually making observations about how intensive discord are never do that, it’s just “I see it using 6% at the desktop” [with the processor in the idle state rendering a full ui instead of just the overlay].
You're right that I'm probably not benchmarking rigorously enough. But people don't care about any of this stuff. They care about "my FPS gets lower when Discord is open, pls fix". And they don't want to micromanage CPU usage either. Make sure you minimize Discord after starting the call, make sure you close that gmail tab during the game then open it back up when you're done so you don't miss notifications? No thanks. I'd rather just pay for a few more cores.
I guess what triggered me is you saying you cringe at the end users. They don't deserve that. They just want the stuff they buy to work reliably. And if they need a bunch of extra headroom for that to happen, well, that's just the sorry state of the industry. If you must cringe at someone, direct your cringe at developers.
I posted this comment here because I also appreciate your name.
Even then it’s not like they’re paying per unit. All of the patent pools and licenses I know of have a cap and AMD, Intel, Nvidia, etc are easily already there.
Can you blame browser makers for this? When was the last time you saw someone pay for a browser? Pre-2005 Opera?
Chrome is free and youtube notoriously doesn't make a profit (unless this changed in 2021).
Their money comes mostly from search.
If those two tasks aren't themselves multithreaded (which they arn't if you're doing them in your browser) then you need at most 3 cores.
All the other background processes in your browser and OS are not using any significant CPU time, and can all comfortably share a fraction of a single core.
I have ADHD and I'm capable of doing work while focussing entirely on two tasks, often I'm looking at one screen where I'm writing code while reading an article on another screen.
Sometimes I feel like "reality" is too slow and I've already figured out the code for the unit test (or whatever I'm working on) in my head and I'm impatiently waiting for my hands to catch up while filling in a boilerplate template. I often focus on two things at once because the main work I'm doing isn't stimulating enough.
A lot of people question how I'm able to do this without making constant mistakes, but before I started on my medication my brain and the world around me was so incredibly noisy. Not necessarily sound, it's just that everything around me was a distraction. When I'm taking my medication, the distractions don't exist and there's almost so little happening that I find it unsettling.
I don't really know how to describe it properly so this probably made little sense. Just imagine you have a really angry baby screaming at you 24/7, it would probably effect your work? Now the baby is suddenly being quiet and you're able to work really effectively, but you got so good at coping with the baby distracting you that now you have all this spare CPU time in your brain which used to be devoted to denoising.
I was worried at the start I would end up on 70mg, but it just seemed to work from day one onwards.
Do you get the reduced appetite? I basically don't eat from 11am until 8pm most days, which has worked quite well for weight loss. I wasn't super heavy (18st, now 14.5st) but I had put on quite a lot during COVID.
As an example: Mirtazapine is an antidepressant used to treat major depressive disorder, but that doesn't mean you can give it to an otherwise emotionally healthy person and it'll give the same effect delta to them.
If you put a crutch on a healthy leg, it doesn't suddenly become a better leg.
My medication allows me to experience the world without crippling anxiety and constant distraction, it just so happens that 30 years living with those issues means that a large portion of my attention and time was devoted to dealing with them.
So, yes, my ADHD medication is performance enhancing, but that's an incredibly reductive way to look at it. That doesn't mean that anyone can take it and perform better, in fact studies of studies have found that the results of stimulative medication for people without ADHD (such as those who are using it to try and improve cognition during studying) can be similarly triggered by placebo effects:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3489818/
As it remains unclear whether stimulant medication has the same effect on healthy individuals as for those with ADHD, it is possible that many reported effects of prescription stimulants in healthy individuals may stem from placebo effects. Looby and Earleywine (2011) examined whether placebo effects influence reports of subjective mood and cognitive performance among college students who endorsed several risk factors for prescription stimulant misuse (e.g., low grade point average, fraternity/sorority involvement, binge drinking). Interestingly, participants believed that they had better ability to focus and persevere, particularly for a sustained amount of time, when they expected to receive MPH (Looby and Earleywine 2011). This is similar to circumstances in which participants may engage in nonmedical-stimulant use to study or cram for extended hours. On the other hand, when experimental participants did not expect to receive MPH, their attention appeared disrupted resulting in inconsistent reaction times throughout the CPT.
If my medication is performance enhancing, then literally any antidepressant, antianxiety, anticoagulant, anti-inflammatory, antihistamine, antibiotic, etc., is also performance enhancing by the same thought. What we need to consider is the baseline it lifts you from and the place it takes you to.
It's probably worth mentioning that I'm in the UK, not the USA, where access to stimulant medication is a _lot_ stricter and it can be very difficult to get a diagnosis of ADHD. I had to go through several hour long sessions with specialised psychiatrists to receive the diagnosis after spending years being tested for other conditions like Mixed-Anxiety Disorder.
An excerpt from my remission notice is here showing the clinically recognised reduction in symptoms, it mentions that the decision was based on both subjective and objective justifications: https://i.imgur.com/RJQCAs6.png
The diagnostic screening questions for the ASRS1.1 are here showing the areas where I subjectively improved: https://add.org/wp-content/uploads/2015/03/adhd-questionnair...
On a side-note, your comment came across very harshly. It's very easy to make a short, pointed statement like that about a topic you aren't very familiar with, but you should consider how informed your statements actually are - if they're backed in data or long term subjective beliefs. Sometimes, brevity imparts a lot of information in few words, sometimes it just comes across as rude.
Apple's strategy with their in house chips has always involved optimizing for performance and power draw, and not so much for die area. They throw execution hardware at the problem and run the clocks slower for power efficiency.
For instance, going back to the 2012 A5x:
>the A5X is approximately 2x the size of NVIDIA's Tegra 3, and even larger than a dual-core Sandy Bridge desktop CPU
While it's technically possible for Apple to have ramped up CPU clocks in pursuit of higher performance, Apple has traditionally been very conservative on CPU clock frequency. Higher clocks require higher voltages (especially on the same process node), which result in an exponential increase in power consumption.
https://www.anandtech.com/show/5688/apple-ipad-2012-review/1...
Caring less about die area makes sense when you are selling the whole widget and not just the chip itself.
If this works that way in practice is anyone's guess though.
It's clear that we need something in between regular quad core CPUs, and GPUs connected through PCIe. Some kind of parallel compute unit should be standard in pretty much all CPUs (or SoCs) eventually.
So, an iGPU accessing unified RAM?
Denverton is actually a very interesting platform for the wattage - that’s 16 Goldmont cores in a 32W package (despite being 14nm!) with full RDIMM support and QuickAssist acceleration. And as mentioned there is a new atom-based server platform with newer cores (I think they’re not the Gracemont but actually the next iteration?) coming in the next year or two.
https://en.wikichip.org/wiki/intel/cores/denverton
If you want something with a lower core count now, look for something with a J3405, J5005, or Tremont-based processor. Intel has some NUCs with those chips and there are also itx-form factor motherboards from other companies like Asrock and gigabyte.
Actually a lot of the 2-bay synology NASs come with those chips, I believe it’s the ones ending in J. They have fairly nice video encode/decode support for their price bracket which makes them nice for plex/etc.
1. It has roughly the IPC of Skylake. 2. It’s the first Atom with AVX2. Even in 2021 Intel released brand new Atom cores without AVX2, which was introduced with Haswell (Haswell New Instructions). And in most tests, merely enabling AVX2 will yield 10% higher performance. That’s why Clear Linux has higher performance as they’re all tuned for Skylake. That’s also the reason why Red Hat couldn’t target x86-64-v3 with AVX2 for their new releases.
All previous Atom cores are slower than you imply:
Tremont (Atom 4th gen) has the IPC of Sandy Bridge. Goldmont (Atom 3rd gen) has the IPC of Core. It’s not exceeding Core like you imply. And Atom first gen is dog slow. It is like running a Pentium II/III CPU or a first generation Raspberry Pi. I find them an unsuccessful attempt of Intel of trying to create a mobile chip. In essence they just brought back a Pentium era chip.
This new Atom is fantastic though.
There is obviously a dearth of actual practical benchmarks of real tasks apart from STH and a few others, but if you look at Passmark or other generic benchmarks, like I said, it’s certainly faster than core2quad enthusiast desktop, which is completely reasonable and even impressive given its power budget. Actually in Passmark it’s even faster than a full Nehalem Core i7 with SMT, it actually works out just a bit under a Sandy Bridge i5 of similar clock - so I was a bit conservative there. And Passmark corroborates my “3104 is about 50% faster” guesstimate.
(UserBenchmark’s actual measurements - not the speed rating - is likely more accurate than Passmark but I know I’ll get my head bitten off for it!)
https://www.servethehome.com/intel-pentium-silver-j5005-benc...
https://www.cpubenchmark.net/compare/Intel-Pentium-Silver-J5...
(Do remember that Intel progress through that era wasn’t really as bad as people say… I’ve seen people say “5% a generation” and the only generation close to that low was Broadwell. Skylake is a ton faster than Sandy Bridge clock for clock, and J5005 still does fall a bit below Sandy Bridge. There’s your two generations of progress since Goldmont - Atom went from below Sandy Bridge to matching Skylake during the Tremont/Gracemont generations.)
The commentary UserBenchmark provides is terrible and the “effective speed” composite rating is terrible. The actual int/fp benchmarks are perfectly fine and actually tend to be more reflective of actual benchmarks like spec than Passmark has been, in my experience. Passmark does sometimes have oddities, UserBenchmark hasn’t.
But bandwagoners can’t accept that, they just see “UserBenchmark” and their vision goes red and they shake as they struggle to type out “nobody should EVER use UserBenchmark”. I do apologize for being colorful but that’s how it is.
Like I said, there’s a reason I didn’t link it, no matter what you say, there’s a significant number of people who just can’t act mature enough to handle separating the data from the editorial positions.
It would be nice if we had good Spec2017 benchmarks for everything. I would prefer that. If you want broad comparisons of completely obscure hardware “what does this ultra-budget Xeon Bronze look like against this atom vs a desktop processor from 2007” the options are limited. You have Passmark, you have UserBenchmark, and you have Geekbench. And people still lose their shit over Geekbench too.
Even Passmark has only seen that xeon a total of 14 times.
The situation is even worse for GPUs where UserBenchmark is the only reasonably broad benchmark database available for truly ancient stuff that 3Dmark won’t run on. What if you want to compare a GTX 9800 to a Vega 7 and a Intel HD 605? It’s UserBenchmark or nothing. It sucks but it is what it is, and people need to just grow up. Including the owner of UserBenchmark. But the effective speed is just a composite benchmark and he’s never actually tampered the underlying int/fp benchmarks.
Phoronix is also not particularly good and has some serious methodological problems too. But nobody else does linux benchmarks apart from STH and Phoronix, and Phoronix pumps them out like crazy and just survives on volume even if his testing is kinda shit. Look at his “Linux games” test and he’s mixing multiple resolutions into a single result set, including the same game multiple times, etc, and then pulling out averages, everything is framerates and not frametimes (meaning, averages and not minimums), etc. From what I’ve heard his application testing isn’t any better but I don’t remember specifics.
Just goes to show: you can be scum and put out good data, and you can be a pillar of the community and put out bad data. Reddit tone police don’t change that.
There is a awful lot of incredibly useful data in the world that came from people who are extremely disagreeable on a personal level - or much, much worse.
Kind of like the "Vista debacle", I think Atoms were just unsuited to running Windows 7. Maybe I'm too forgiving or forgetful, but I used that netbook for many years, and it wasn't that slow with Linux. (7 was pretty sluggish, though)
...I do remember intensely disliking the introduction of early gnome 3, while running Fedora on that netbook. Not kind to the integrated graphics :-)
In that form factor, and costing what it did, I was massively impressed.
Lxde is great though and sure id believe that. I ran lxde on a single-core Phenom low power thing and with a SSD it was marginally usable even with that complete shit hardware.
I actually would not be surprised if, at the extreme low-end like the V140 or the Atom netbooks, that a SSD actually produced measurable increases in CPU performance. Not only do you have iowait which generally doesn't count as "idle" time according to linux (and probably windows) cpu utilization metrics, but even if you account for iowait, swapping off for another task while you iowait likely does have a performance hit in terms of caches that are no longer hot and so on, so performance is likely diminished when you come back to it too. And with no SMT, and only a single thread, context switching does become comparatively much more expensive in that fashion.
1st gen Atoms with the chipset (945?) that was bigger and hotter than the processors was kind of amusing.
That Intel can't help itself from making confusing product names is pretty inexcusable. If a 'Pentium' is an Atom or not makes a big difference in what the specifications mean, and it's not clear to regular people that a dual-core mainstream Intel is about performance equal to a quad-core Atom (assuming similar age chips).
Laptop makers love to combine atom CPUs with amazingly slow spinning hard drives, and they're usually pretty small too. Windows 10 performance on a spinning drive is trash, and it's even worse when it's a low rpm laptop drive. Doesn't help that they're often as little ram as possible and soldered down (although even with 8GB ram, Windows 10 will flog a spinning drive just sitting at the desktop; who knows what it's doing).
It would be interesting to see a 4p+24e or 6p+16e setup for parallelized loads though. Not sure there's currently a need for that on the desktop though.
That's pretty neat, but I'm thinking about, say, a 60W-95W package that would make sense in a normal desktop.
How many E-cores could you fit in such a package? For highly-parallel workloads, would it stomp all over anything else on the market? (I'd guess yes)
[1] https://www.servethehome.com/intel-sierra-forest-the-e-core-...
What about massively reduced energy consumption and associated loss of head dissipation/noise and increased battery life? Isn't that why Apple and mobile phone SoCs introduced that design in the first place?
I hope their drivers would work properly with the OS. Else, the user would be locked on either the performance cores or the efficiency cores - it happens with laptops with dual graphics card (intel inbuilt and high performance Nvidia or AMD graphics)
It's 90% the process node. There is no magic design sauce that makes those chips better.
The big/little design change did seem to improve performance quite a bit for Intel, even using the same manufacturing node as before
That is because there is a huge design gap. Intel architecture and design teams have always assumed on the inevitability of their own manufacturing superiority and process advantage and thus never had to really consider where to place transistors for the most benefit, or think really hard on better ways to improve single-thread performance instead of just turning a knob that was invented two decades ago.
In any case, your statement is just plain wrong. Dennard scaling has been dead for a decade now. Planar area decreases with new processes come with increased power density, i.e. new process nodes reduce die sizes but the transistor performance does not improve the way they used to.
source: worked at Intel for CPU design for a decade, then worked on the same things elsewhere using TSMC process
Nobody in CPU design cares about unconstrained power performance. Even at Intel, they never use the PL2 power level as the design landing zone. All modeling is done at much lower power target, even below PL1.
That is to say: unconstrained power performance is not an engineering effort, it is a marketing and SKU differentiation effort. Your “more powerful” statement is a matter of how much power Intel/Apple/AMD/whoever is willing to shove into their piece of silicon, which in turn dictates the form factors and customers they can try to satisfy.
Apple only needs a few watts per big core and still get the peak perf it needs, and put that chip in all the form factors it sells. Obviously if M1 sucked down the kind of power as any Intel chip, it cannot go into an iPad, or a fanless macbook air, without totally compromising performance.
Intel on the other hand needs grossly more watts to hit that same performance, and therefore is losing badly on form factors that care about power: laptops and servers. Some form factors/customers just don’t care, which is why Intel cooks up marketing targeting the gamer market, or the 8-pound laptop market. That’s all fine, just don’t pretend Intel is not getting absolutely hammered on the CPU technical specs for the money-making markets that matter.
BTW, the DIY gamer CPU market does not matter. The 8-pound laptop market does not matter.
The fanboyism runs so deep it's amazing. Let's compare PS5 to GameCube next and talk about Sony's great engineering advantage
Go compare the perf/power curves of the alder-lake cores (both P/E) versus the last-gen Apple cores from TSMC n7+. TSMC n7+ is roughly comparable to Intel 10nm ESF in transistor performance, so it is a fair comparison.
Do your research then get back to me about how the process is entirely responsible for every perf/power difference and we (the silicon design teams) are just identical robots doing the same thing across the board in every company.
I do like how you claim there is no design magic sauce, but then Apple will lose big when Intel/AMD come out with their 5nm equivalents, as in, they found some magic sauce? WTF?
The same MacBook Air can do hour long Skype video calls with ~9% charge consumption, with no discernible heat difference on the body too.
M1 is a different beast.
Why is everyone insistent on comparing Apple’s latest CPUs to Intel’s parts from multiple generations back?
Intel is on 12xxx series now. AMD also has new generation laptop CPUs that perform very well. Let’s compare to those.
The M1 is power efficient but much of that is due to the TSMC process they used. I really don’t understand this desire to make arbitrary comparisons to old Intel CPUs.
Will these be meaningful for you?
The thing is, M1 on air is a relatively low clock speed CPU with passive cooling. 10200K is actively cooled, unlimited beast of its generation. The libraries I use are X86 optimized. Yet the code I’ve written is 25% faster on ARM.
This is significant.
Apple knew this so the M1 has an extremely fast memory system.
Intel’s memory architecture can keep up with the load until I utilize all cores. When I start to use HT cores, neither cache, nor memory can keep up.
I do the both tests (Intel and Apple) both with physical core counts to keep it fair.
So it’s not some run of the mill code.
Curious what you meant here since that part doesnt exist, and if it did it would be near the bottom of the stack for the generation (101xx and 103xx are the low core count i3 and i5 parts).
Written that comment on mobile on a plane, before take-off, so I either typo'd and didn't see, or auto-correct changed it deeming 10200K is more correct than 10700K.
If you look to the original comment where I sparked the discussion, I've mentioned it as 10700K already.
Being able to compete against a consumer high end desktop, even if it's an older one, it's a feat all by itself
> M1 is a different beast.
The M1 has other advantages which are decidedly not rooted in the hardware: software. Apple seems to do profoundly better than Windows and Linux at applications actually using hardware accelerators. This makes a gigantic difference in performance and power efficiency.
On a Windows or Linux system it is not at all a given that even very easily hardware-accelerated tasks, like video decoding, actually use the hardware that's there.
Working on HPC field and with generations and generations of Intel hardware, I can tell you that two node difference (alone) never gives you 25% real world performance in the realm of Intel, let alone theoretical.
Real world performance pretty much always lines up with these estimates. It's the main reason why we have exponential growth in computing power in the first place.
It would be a more fair comparison to compare the M1 to AMD's ZEN 4, which should be coming out next quarter.
[1] https://www.tsmc.com/english/dedicatedFoundry/technology/log...
This is jumping from Intel's heroically old 14nm+++++++++++++++ to a brand spanking new TSMC process. It might be on the order of 4 to 6 times denser.
We've seen how the latter went until AMD came back to the game.
TSMC 5nm is better, but AMD is not there yet.
I have a list of instructions per clock from SPECINT results which I compiled from Web sources. (I didn’t verify the results myself.) I added in rumored future processors with their highest rumored performances:
Cortex-A73 (ARMv8-A) 64.4
Pentium M 69.0
Bulldozer (AMD FX) 76.2
K10 (Phenom II) 76.7
Goldmont Plus (Atom 3rd gen+) 77.5
Core 80.0
Bobcat 81.3
Nehalem 90.0
Cell (PS3) 93.5
Jaguar (PS4) 93.5
Sandy Bridge 100.0
Tremont (Atom 4th gen) 100.7
Ivy Bridge 105.8
Cortex-A76 (ARMv8.2-A) 114.7
Zen 115.9
Haswell 117.6
Zen+ 119.3
Gracemont (Atom 5th gen) 120.8
Broadwell 121.5
Skylake 124.0
Zen 2 139.8
A10 (ARMv8.1-A) 139.8
Cortex-A77 (ARMv8.2-A) 141.0
Sunny Cove Ice Lake 146.3
Cypress Cove Rocket Lake 147.6
Zen 3 167.4
Golden Cove Alder Lake 175.6
A11 (ARMv8.2-A) 186.0
Cortex-X2 (ARMv9-A) 190.2
Rumored Golden Cove+ Raptor Lake (’22) 193
Rumored Zen 4 (’22) 207
A12 (ARMv8.3-A) 213.8
Rumored Ocean Cove (’23) 223
A13 (ARMv8.4-A) 235.0
A14/M1 (ARMv8.5-A) 236.7I find the different balances of performance and efficiency cores interesting. I’d love to have a lot more Icestorm cores on a Mac to play with background tasks that don’t need the firestorm ones.
On the Intel side, I’m also disappointed by not having the option to implement the missing instructions in software.
But even being one behind, it outperforms M1, with lower efficiency. Thus reasonable to speculate that a process node advancement would at least put Intel's chip on par with M1 on both perf and efficiency
We'll see, but I highly doubt that Intel-at-equivalent-of-TSMC-5nm will be performance/watt competitive with the M1. Apple gets that power efficiency via an extremely wide-and-deep design that intentionally trades off a surprisingly low max clock ceiling (3.2GHz) for power efficiency while maintaining an absurdly high IPC (something like 75% higher than Intel's IPC at only one process node smaller) that makes it much more competitive on performance than conventional industry wisdom ever expected (there is and has been for years a strong belief that a design as wide as Apple's would be grossly inefficient because you could not possibly extract enough instruction level parallelism to keep so many decoders/execution units saturated -- hence we've spent over a decade with 4-5 wide frontends being the "practical maximum" that allows them to still target ~5GHz turbo speeds, which was believed to be basically the best/only path to high performance within what was believed to be the limit of "achievable" IPC. The M1 blows these fundamental assumptions out of the water).
So Intel makes conventional design decisions that make up the performance of their lower clock-for-clock IPC by turboing well north of 5GHz, at the cost of consuming something like 7x the max power of the M1. A single node shrink isn't going to buy them a 75% increase in IPC (their last 4 or 5 node shrinks combined haven't bought them that much IPC increase), so Intel@equivalent-of-TSMC-5nm still going to have to burn power like crazy to stay competitive on performance, and that's going to cost them efficiency that the node shrink just can't paper over.
It will lead in absolute perf, but its not going to be nearly as efficient.
Intel has had 256-bit dual-issue FMA since Haswell (2013); that's 32 flops/cycle/core. AVX-512 doubled the vector length for Xeon Phi and was generally available in 2017 Skylake.
[1] https://www.tomshardware.com/uk/news/ryzen-7-5800x3d-vs-core...
They have a patent for something similar at least: https://wccftech.com/amd-big-little-processor-cores-show-up-...
Not saying x86 can’t ever replicate that, but the ISA makes it harder.
- some LAPACK-style libraries select slow routines for AMD cpus
- better understanding of / support for performance counters and suchlike
Once Intel is matching AMD for density, Intel will be able to jam far more small cores into a multi chip architecture than AMD will with just "big" cores.
Big/Little is a very important step forward towards core density (and idle power consumption, switching off big cores under load is a huge saver in laptop power usage).
It also avoids scheduling conflict between housekeeping / background tasks and "user" stuff, which means more predictable and stable throughput and better cache utilisation (your cache doesn't get blown because the system needed some time to zero memory or update something).
wtf do you need more cores for??? Wat are you doing?
> not interested in actually pushing HEDT forward since
consider buying a server.
I have a bunch of embarrassingly parallel workloads which I have to run regularly (text analysis, machine learning which is not easily accelerated on a GPU, etc.).
> consider buying a server.
I looked into it; not really an option. Those are vastly more expensive, really loud (I need a quiet workstation), have significantly lower boost clocks (I also need single-thread performance), and are either really hard or even impossible to actually buy in my country through normal retail channels.
If you’re using Rayon for anything you expect to benefit from a lot of cores, my experiences have been very negative in the past. Rayon is often extremely inefficient once you allow it to use more than a handful of cores.[0]
If your unit of work is absolutely massive, you might not be as affected by Rayon’s scheduler, but it makes me sad that this issue still hasn’t been resolved. My comment on that thread (documenting just one instance where I ran into this issue) was almost 4 years ago! And the issue itself is almost a year older than that. Sure, I could try to contribute a general solution, but it’s just easier to stop using Rayon.
I no longer consider Rayon to be an advantage for Rust. There are plenty of good ways to wire up a parallel work pool, but they do require more than a one line change.
It depends very much what you're doing of course (compiling linux kernel repeatedly perhaps) but if you have a little you can get a heck of a lot out of it with experience, although you surely will hit a wall sometime. But 32 cores not enough is just unimaginable for me.
But your case, ok, understood.
BTW The biggest slowdown IME is not from lack of cores but lack of RAM. Odd that no-one's mentioned that. Or using a server with larger numbers of memory channels to keep their cores fed. I/O > cycles more often than not.
So much so that I’m now trying to figure out whether we can somehow cheaply move to a dual Epyc 7742 setup.
At the risk of ill manners but i must ask, have you looked at all this because I've got in some cases orders of magnitude speedup by popping the hood and instrumenting.
Random examples:
Trawling through 70 GB of web server log files going back years with ad-hoc analytics. I tried ingesting this into a cloud service but it was taking forever, and cost a non-trivial amount of money. It was also dog slow compared to just doing it locally with SQL Server Clustered Columnstore.
Compiling Chromium. I'm not even a browser developer or anything, but there was a "global" change that I had to preview ahead of time, so I made a custom Chrome build that had the change embedded in it. I also made a matching build without the change to enable A/B comparison testing.
Server base image task sequence development. You would think a server is ideal for this, but no! It's very I/O heavy, and doing it locally in Hyper-V using an NVMe drive can make this 4-5x as productive as any shared platform with throttled disks. There are zero cloud services that can even approach a Samsung Pro 980 SSD for reasonable costs.
Multi-server lab tests. Compared to cloud services, it's faster and easier to build a local pool of machines for testing how they interact. With 64 GB of memory, I can (and have) easily run 20+ machines. Compared to cloud platforms, I can do things like thin-cloning to build (and then delete and rebuild) these nearly instantly -- about 5 seconds or less in some cases. Even the relatively heavyweight Windows Server 2022 VMs boots in just 1.4 seconds! Some build jobs require nearly 100 GB of content to be deployed, and the upload to the cloud is a problem in some customer environments. E.g.: due to security restrictions. Doing it locally is easier and faster.
Photography and video editing. I do both as a hobby. Lightroom will cheerfully use whatever number of CPUs (and GPUs!) you have to process images. My recent hardware upgrade made a very noticeable difference to the workflow. Similarly, photogrammetry and other advanced 3D modelling workflows can use up virtually any amount of compute.
Password cracking. I occasionally need to reverse a password hash for work, typically to fix lost access to some ancient piece of network gear. Sometimes it's for verifying password strength where the plain-text password is not known. I could distribute this across a pool of cloud VMs, but my lab subscriptions have hard limits and this is sensitive data.
Automated searches through code for a breaking change. E.g.: scripting the checkout, build, test, etc... in a loop. (Git bisect doesn't solve every problem)
VS Code now has ripgrep integrated, so "find in files" will use every CPU. I have about 100 GB of files on my laptop from various old projects, so this is a very helpful feature for digging up some code snippet that I only vaguely remember writing. It can find a match for a regular expression at 700 MB/s using 16 threads at 100%.
Defragmenting moderately large (50-500 GB) databases. I have some customers that have built up horrendously fragmented databases over decades, but can't afford the IOPS and/or downtime to fully rebuild all of the tables and indexes. I recently utilised the aforementioned Samsung SSD to defragment an 80 GB database in just minutes, that would have taken days or weeks on the production server. Again, transferring this out to a cloud is a no-go, and the problem was that typical server storage is much slower than a typical laptop. Locally I could get 200K IOPS sustained. SQL Server Developer Edition is the same as Enterprise, and will utilise all cores to rebuild indexes in parallel.
Etc, etc...
If you know what you are doing, you can make the CPUs go "brr...".
NB. "Trawling through 70 GB of web server log files" unless you partition your search you're going to be doing a lot of IO due to having only 64-GB of RAM. Chucking in extra mem to keep it all ram-resident will beat cores any day.
The query engine in Microsoft SQL Server is absurdly advanced, and simply beats the pants off just about everything else for complex, general-case queries. There are other engines that are better tuned for special cases, but if you've got "stuff" that's within the capacity of a single box, it's really hard to beat MSSQL.
Let's expand on my example above: Clustered ColumnStore.[1]
It's basically PowerBI or Vertica embedded within the traditional MSSQL engine. This extension enables the ability to have a per-column (vertical) on-disk storage format instead of the traditional row-store format most databases have.
You can have it as a secondary acceleration index, or as a clustered (primary) index. In the latter case, it'll compress your raw table data. For my log files it was 33-to-1 compression (3% of the original file size).
Second, even if the data doesn't fit into memory, it'll utilise the columnar storage to simply ignore (skip) columns it doesn't need for the query. I was computing performance statistics, so it was just reading in some numbers instead of the text. A further 100-to-1 optimisation can be gained here.
It then parallelises this across all cores/hyperthreads, so I get 100% utilisation on 16 cores.
All of this I/O is fully asynchronous with a huge queue depth. Expect to see 200K IOPS on a laptop, easily.
Once in memory, it uses Intel AVX SIMD instructions to chew through the data 8 rows per clock tick per core. With 8 cores going at once, this is 64 rows per clock, or up to 234 billion rows processed per second.
What's neat about it is that it's fully general: Columnstore can be mixed -- in a single query -- with in-memory tables, on-disk tables, traditional row-store, and external tables streamed in over the network. The engine will just... "figure this out". It'll build indexes on the fly if it has to, ideally in memory, but failing that... it'll stream disk-to-disk as required. I've seen "sort spills" reach 600K IOPS on my laptop.
If I have to scan through years of logs and be able to join it against additional tables for "enriching" it, then it's hard to beat. Sure, there's a scale where the are dedicated log analytics databases, but they have their quirks, limitations, and costs.
I take your incredulity that others can fully utilise their computers as a sign that general knowledge of just what is possible is poor. People have pointed out that some simple shell scripts can outperform Hadoop clusters up to a surprising scale. Similarly, knowledge of the capabilities of traditional database engines like MSSQL is oddly lacking, even amongst developers. Not to mention more esoteric ways of making your computer do your bidding, such as Mathematica, Julia, GPU codes, or whatever.
Your computer is a power tool for your squishy, manual-labour meat brain! A lever for the mind. Learn to utilise it better and you'll be better at thinking.
[1] https://docs.microsoft.com/en-us/sql/relational-databases/in...
> It then parallelises this across all cores/hyperthreads, so I get 100% utilisation on 16 cores.
that suggests your working set is indeed smaller than ram. Good.
However be aware that the parallelise feature of mssql is much better that it was but is still moderately crap. Disable it and see if you get a 16X slowdown. My guess is you won't, which means when enabled you won't be getting a 16X speedup (also hyperthreading can actually slow things down - disbale it and see what happens). Edit: I'd not be surprised if you were only getting 4X to 5X speedup (I may be wrong but I do have experience in this).
> With 8 cores going at once, this is 64 rows per clock, or up to 234 billion rows processed per second.
(giggle) Oh, I don't think so!
> I've seen "sort spills" reach 600K IOPS on my laptop.
Then you're hitting IO and yer cores are useless. It spills to disk because it's overflowed ram - you may have a problem.
> Learn to utilise it better and you'll be better at thinking.
... agreed, but the reverse is true also.
edit 2: Depending on what you're looking for, and if you do it often, check if MSSQL's free-text indexing can index it, if so you may get a far bigger speedup from that than anything else. Maybe. Depending.
So keep in mind that the SSD I'm referring to can read at 7 GB/s. Not seven gigabits... 7 gigabytes per second. I've benchmarked some not-so-old servers (the entire box, not a VM!) as capable of only 35 GB/s streaming reads from main memory! So these days "disk" is more like "5x slower memory". It used to be 100-1000x slower.
I told one customer to get a 16x PCIe gen 4 adapter card with 4 NVMe M.2 ports on it. This would -- in theory -- be able to pump out 28 GB/s of reads, equivalent to a 200 Gbps NIC. Total storage would be 32 TB with CoTS 8TB drives.
That's a mind-blowing amount of IO in a single box!
The hardware has the capability, it's the software and inability to utilise that is the bottleneck now..
> I've benchmarked some not-so-old servers (the entire box, not a VM!) as capable of only 35 GB/s streaming reads from main memory!
Then you have either a low-end server or something is very wrong (I think you benchmarked wrong, see below). You should be getting maybe 15 GB/sec per channel, so 4 channel = 60GB/sec, for a decent server you may well have 6 or 8 channels. (figures may be a bit off but 35 is puzzling me).
You may have measured on a single core, but each core has its own mem reading stuff so it should multiply up if your benchmark uses extra cores.
Anyway, I have to get back to work on my dual-core haswell. Good luck!
(Edit: 15 * 4 = 60, not 80, corrected)
We could sit there and optimize the approach, but the time that would take and the complexity added, it would be cheaper to just buy more powerful computers.
For example, on reasonably new 8 core 64gb ram, 7gbs HDD machine, I optimized a calculation for one of the features down from 2-3 days to about 15 minutes. Throwing cores or ram or disk at this algo before would’ve got nowhere. Now htop glows like a Christmas tree when it’s running. This was costly to do but it was the only way.
We could talk about python being slow and blah blah blah use C but once again, whenever we do tricky stuff with code that then gets shared amongst a bunch of python devs I need to think about the holistic cost including maintenance, ease of changes, bug fixing, resource that actually know c, client appetite, what the production load is going to look like etc
And finally, at some point a client will just come along with another million rows of data
M1 builds a lot faster than Intel CPUs even with fewer cores because of the memory bandwidth.
A CPU with so many general purpose high-perf cores that it manages to look like a GPU in terms of execution units count is a beautiful thing.
But maybe that's just because it's a first generation technology for Intel? I hope things improve in the future because I sure would love to have more cores available (currently rocking a 32 core Threadripper) and AMD is sure as hell not interested in actually pushing HEDT forward since they became the top dog. (I have a few grand burning a hole in my pocket waiting for a product they don't want to release.)