News for hackers. Stuff that matters.
Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by Aardwolf on Wednesday February 15, 2023 @05:17PM (#34811538)
Thanks Intel for publishing something that's useful on AMD consumer CPU's but not on Intel ones.
Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by RandomTisk on Wednesday February 15, 2023 @05:34PM (#34811788)
AFAIK Consumer Zen4 supports 12 of the 15 AVX-512 extensions, do we know for certain this doesn't target one of the ones AMD is missing?
Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by aseipp on Wednesday February 15, 2023 @05:39PM (#34811842)
The newest extension it needs is -VBMI2, which is supported by Zen 4. -DQ and -BW are all quite old and very common amongst all implementations by this point.
Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by paulmd on Thursday February 16, 2023 @01:51AM (#34815833)
zen4 supports basically everything except the xeon phi SMT4 intrinsics (4VMMW or whatever). As did alder lake before its removal.

The support story for AVX extensions is not as complex as people make it out to be anyway. Server is a monotonic sequence, consumer is a monotonic sequence, both of them are converging apart from 1 or 2 that are unique to one or the other. Xeon Phi has the SMT4 intrinsics that are completely its own thing due to the SMT4 there, but you'll know if you're targeting xeon phi.

https://i.imgur.com/idAjB1X.png

So as you can see, consumer supports everything except BFloat16 for neural net training. Consumer doesn't do that so it's not a problem. And it doesn't support the Xeon Phi stuff because Xeon Phi is its own crazy thing.

No uarch family in that chart has ever abandoned an extension once it was adopted. So unless you are taking a consumer application and running it in the server, it's literally not even a problem. And server gets bfloat. Like that's it, that's literally the only two things you have to know.

but letting AMD fanboys draw le funni venn diagram is obviously way catchier than a properly organized chart representing the actual family trees involved... SSE would look bizarre if you represented it that way too, like all AMD's weird one-off SSE4 extension sets.

--

Really the thing that has been a problem is that server has been stalled out forever... first 10nm problems and now sapphire rapid has more than a dozen known steppings. They can't get the newer architectures out, so consumer has been moving ahead without them... up until alder lake nuked the whole thing. If server had been able to get newer uarchs out, there would be a lot more green bars in server too.

supposedly the fab teams are actually ready to go now, and the problem is the design teams aren't used to operating in an environment where they can't go down the hall and have the fab teams fix their shit. Intel put the foot down and aren't letting them do that anymore, since the fab teams need to sell the resulting process/cell libraries to external foundry customers, and the design teams need to be able to make their shit work on external foundries. You can't do this hyper-tuned shit where the process is tweaked to make your bullshit cell designs work. But some of the teams are not mature enough to work in a portable environment where design rules actually have to be obeyed because Intel historically never had to.

When you hear the infinite steppings of Sapphire Rapids and the network chip team's continued inability to put out a 2.5gbe chipset that works (I think we are on public release number 6 now?), it's pretty obvious who the worst culprits were. Meteor Lake may also be having packaging/integration problems (although this is supposition by me based on what products are delayed - coincidentally it is a lot of chiplet/tile stuff and intel obviously lacks experience in advanced packaging) but the products that have infinite steppings obviously can't get their own shit together even on their own tiles let alone talking to other people's tiles.

But Intel supposedly are not kidding that Intel 4 is ready to go and they've just got nothing to run on it yet. Hence looking for outside partners. Supposedly they've got at least one definite order signed for Intel 3 in 2024, and I think there will be a lot of people happy to diversify and derisk away from the TSMC monoculture that has emerged... if TSMC stumbles, right now there is no alternative.

https://www.tomshardware.com/news/intel-ifs-lands-3nm-to-mak...

Samsung has all the same conflict-of-interest problems as Intel and also a track record of really mediocre fab execution. Supposedly they are ahead on GAAFET but like... we'll see, it's Samsung, who knows. They've stumbled just as much as Intel, just not on 7nm tier - I remember the iphone "is it TSMC or Samsung" games too. Samsung has put out a lot of garbage nodes and a lot of poorly-yielding nodes of their own.

Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by paulmd on Thursday February 16, 2023 @01:40PM (#34823490)
edit since I can't edit: "And server gets bfloat" meaning "if you were to bring a ML training server application over to consumer it might not work".

Basically what I'm saying is, the only 2 situations that would be a problem is going consumer->server (which I don't see happening often) or going server ML training -> consumer if it doesn't have a non-BFloat16 fallback. And everyone does ML training on GPUs anyway.

Otherwise everything supports everything. Going backwards within a family might be a problem, but, that's always a problem, it's not a support matrix problem where there's a mixture of capability, it's just backwards compatibility to older hardware with less features.

The real problem, as I said, is that "Cooper Lake" there is Ice Lake-SP which was stalled for years, and by the time it was adopted Milan was already in the market and Cooper Lake was dead on arrival. So nobody actually has Cooper Lake, if you have AVX-512 server it's 99% chance it's either Skylake-SP or Cascade Lake-SP.

Sapphire Rapids is Golden Cove cores, so it should have the same support bars as Alder Lake there, ie basically everything, including server bfloat as well.

https://www.phoronix.com/image-viewer.php?id=intel-sapphirer...

(and of course the other problem being Intel has no idea what the fuck they're doing with big.LITTLE on the consumer platform... the support matrix for everything consumer-family going forward is apparently "nothing".)

Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by jeffbee on Wednesday February 15, 2023 @05:36PM (#34811805)
Numpy is something you could expect to find running on a workstation and Intel's workstation CPU line has had AVX-512 continuously since 2017.
Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by mumumu on Wednesday February 15, 2023 @05:52PM (#34812065)
Alder Lake and Raptor Lake workstation (W680 Chipset) doesn't have AVX512 enabled.
Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by sliken on Wednesday February 15, 2023 @06:34PM (#34812572)
Sad, big win for the Ryzen or future Zen 4 threadripper.
Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by jeffbee on Wednesday February 15, 2023 @06:00PM (#34812150)
Fair. The "entry workstation" thing from Intel is baffling. I was thinking Xeon W, but then of course there was the Xeon W-12xx that lacked AVX-512.

In short, I was wrong. It would have been more correct to say that Intel has offered a workstation part with AVX-512 continuously since Skylake.

Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by bXVsbGVy on Friday February 17, 2023 @02:42PM (#34839384)
Intel released a couple days ago new Xeon-W CPUs with AVX-512.

The wx-24xx uses the same p-cores as alder lake, but with avx-521 enabled. Same for wx-34xx but with raptor lake's p-core.

The entry levels xeon are believed to be identical to Core ix, with the e-cores disabled and avx-512 enabled. The extra I/O and ECC support is done by the chipset.

In summary, they just went to support avx-512 only on xeon w.

This makes sense, since must be hard to schedule between cores with and w/o avx (the e-cores doesn't have avx).

Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by cbsmith on Wednesday February 15, 2023 @06:09PM (#34812285)
There was a longstanding issue where AVX-512 would trigger frequency throttling on a number of Intel CPUs, resulting in a net performance penalty for mixed workloads.
Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by saagarjha on Wednesday February 15, 2023 @06:11PM (#34812309)
Ok, but that hasn’t been a problem in a while.
Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by cbsmith on Wednesday February 15, 2023 @06:38PM (#34812608)
Yeah, but the mystique still lingers on.
Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by Rimintil on Wednesday February 15, 2023 @06:31PM (#34812518)
Hasn't been a problem because software corrected for that, or hasn't been a problem because Intel resolved the underlying throttling issues with AVX-512?
Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by mhh__ on Wednesday February 15, 2023 @06:36PM (#34812582)
Both
Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by jackmott42 on Wednesday February 15, 2023 @08:05PM (#34813520)
It was never really a problem.
Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by rurban on Thursday February 16, 2023 @01:29AM (#34815747)
It was only useful in a couple of corner cases, but not in general library functions.

Intel could never fix its thermal issues, AMD did.

Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by jackmott42 on Wednesday February 15, 2023 @08:05PM (#34813516)
No, there was a longstanding issue of people worrying that it would result in a net performance penalty for mixed workloads. Meanwhile people actually using AVX-512 dealt with that by making sure mixed workloads were batched appropriately and it is usually a big net win even if you don't worry about it.
Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by stephencanon on Thursday February 16, 2023 @10:35AM (#34820002)
“Longstanding” here means “really only in a single CPU generation (SKX), and even then only if you were dumb and used like 50 AVX-512 instructions in isolation for no reason at all.”
Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by sliken on Wednesday February 15, 2023 @06:33PM (#34812550)
Assuming Intel didn't add code that's:

  if (AMD_CPU) { go_slow() }
.... again
Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by mepian on Wednesday February 15, 2023 @06:38PM (#34812607)
It's open-source, as the article says.
Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by optimalsolver on Wednesday February 15, 2023 @08:02PM (#34813486)
I still don't get why people think Intel is obligated to optimize AMD performance. From what I recall, it wasn't a case of slowing down AMD devices, they just didn't apply code optimizations to machines not using Intel.
Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by sliken on Wednesday February 15, 2023 @08:15PM (#34813595)
The code literally checked for CPU = AMD, and ignore the CPU feature bits what show which accelerations are available.

So sure Intel shouldn't tune for AMD, but if the AVX2 is listed as available a compiler should use it. This was proven when via some shared library trickery you could lie about the CPU name, and suddenly the AMD CPU was faster.

I stumbled across this on a "Why is matlab much slower than I'd expect" thread. Lying about the CPU greatly improved performance and still showed the correct answers.

Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by jeroenhd on Thursday February 16, 2023 @04:12AM (#34816607)
With all the errata compilers need to correct for, I wouldn't blame the compiler for not optimising for foreign chips. Matlab chose to use a library that only works well for Intel (a simple benchmark would've shown that), I don't think Intel's compiler team should be forced to write code for AMD chips. I very much doubt AMD's driver team will optimise their OpenCL tools for Nvidia hardware either.

Blame Matlab and friends for slowing down their software on your computer.

Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by stephencanon on Thursday February 16, 2023 @10:45AM (#34820160)
If Matlab performance is important for competitive reasons, AMD should hire a few SW engineers to build tuned linear-algebra libraries like Intel does. It’s not rocket science. They could be competitive or faster with a low-millions-per-year investment (and only slightly behind with a much smaller investment in grad students and open source).

I worked on these sorts of libraries (not at Intel) for a decade, it’s very, very common to dispatch on CPU family rather than feature flags, because performance details are tied to the microarchitecture, not availability of the feature. Even though Skylake and Haswell both support AVX2 and FMA, there are huge differences in implementation that effect what the best implementation will be, so you can’t just say “oh, FMA is available, use version 37.” Instead you do “if we’re on Skylake, use the Skylake version, if we’re on Haswell, use the Haswell version, otherwise fall back on the generic runs-anywhere version.” Nothing underhanded about it.

Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by Dylan16807 on Wednesday February 15, 2023 @09:02PM (#34813989)
> it wasn't a case of slowing down AMD devices, they just didn't apply code optimizations to machines not using Intel

What's the difference?

The thing that would be okay is "not having optimizations designed for AMD devices".

But when you already have the optimizations, and you refuse to use them because AMD, that is not okay.

Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by rf15 on Thursday February 16, 2023 @12:56AM (#34815585)
> What's the difference

Let's be naïve: "not doing something" is indeed, on a literal level, different from "doing something to slow it down". It's a bit like the nuances of a lie by omission vs a proper lie. That being said... both are still considered lies in a more abstract sense, and so is this deliberate slowdown. I assume the author just took it all a little more literal than most people (probably?) deem necessary.

Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by Dylan16807 on Thursday February 16, 2023 @03:45AM (#34816475)
I would say "not doing something" is an incorrect description of what it did, though. They had to write extra code to make the behavior on Intel and AMD differ.
Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by rektide on Wednesday February 15, 2023 @09:46PM (#34814331)
For those not in the know here, Intel's actually had some fairly ok AVX-512 implementations on consumer chips in the past, even if they do cause the whole chip to downclock significantly.

But for the new Alder Lake cpu, which has P/E Performance/Efficiency cores, the efficiency cores don't have AVX-512, so code would have to find a way to switch modes it runs in as it is shuffled between cores. So generally, AVX-512 is regarded as not-actually-available on Alder Lake.

Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by raisin_churn on Thursday February 16, 2023 @12:28AM (#34815410)
> so code would have to find a way to switch modes it runs in as it is shuffled between core

It's even worse than that. Initially you could disable E-cores in BIOS to get the system to report AVX-512 being available, but Intel released a microcode update to remove this workaround[0]. Intel also stated that they started fusing off the AVX-512 in silicon on later production Alder Lake chips[1]. Also compare the Ark entries for the Rocket Lake[2], Alder Lake[3], and Raptor Lake[4] flagships. Only the 11900k lists AVX-512 as an available Instruction Set Extension. So it's reasonable to say that AVX-512 on consumer Intel lines is dead for now, whereas AMD has just introduced it in the Ryzen 7000 series.

[0] https://www.tomshardware.com/news/intel-reportedly-kills-avx...

[1] https://www.intel.com/content/www/us/en/support/articles/000...

[2] https://ark.intel.com/content/www/us/en/ark/products/212325/...

[3] https://ark.intel.com/content/www/us/en/ark/products/134599/...

[4] https://ark.intel.com/content/www/us/en/ark/products/230496/...

Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by herpderperator on Thursday February 16, 2023 @01:08AM (#34815648)
Does anyone know why they would do this? If AVX-512 works fine on P-cores, and if certain people disable E-cores because they want to use AVX-512, why would they stop those who want to from being able to use it? Why would they go to such extreme lengths to disable something?
Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by paulmd on Thursday February 16, 2023 @01:46AM (#34815803)
Nobody really knows for sure.

The immediate problem is that CPUID is not deterministic for naive software, if you don't set affinity-masks you don't know whether you will be scheduled onto p-cores or e-cores, and so the result you get will vary.

More generally, software doesn't know what configurations of threads to launch... you want to launch as many AVX-512 threads as you have logical cores, but not more, because they won't run on e-cores.

Software could potentially run a cpuid instruction affine to each logical core though, and collate the results... all you need know is "16 logical cores with AVX-512 and 4 without".

And software that isn't AVX-512 aware doesn't need to worry about it at all, since it doesn't know AVX-512 instructions. I guess the long tail of support is the stuff written for Skylake-SP in the meantime, but how much adoption really is there? It's that narrow gap between "regular stuff that never adopted AVX-512 because it wasn't on consumer platform" and "stuff that isn't HPC enough to be really custom" but also "stuff that won't receive an update". How much software can that really describe, especially with the reaction against Skylake-SP's clockdowns in mixed AVX-512+non-AVX workloads?

And also, that software can just launch AVX-512 threads and if they end up on the e-cores you trap the instruction and affine them to the p-cores. Linux already has support for this because Linux doesn't save AVX registers if there have never been AVX instructions used, so, it just would become another type of interrupt for that first AVX-512 instruction. Linus has commented that this is perfectly feasible and he's puzzled why they're not doing it too.

Nobody knows what the fuck is going on and there has been no plan expressed to anyone outside the company as to what the exact problem is and whether they're looking at anything to fix it going forward. It's a complete mystery, nobody even knows if it's something critical or everything is just too on-fire to care about that right now.

(and if it wasn't on fire before, it probably is now, nobody you want to retain is hanging around after a 20% pay cut off the top and truly insulting retention bonuses... ranging as high as $200 for a senior principal (no, that is not missing a "K"). Oh and we paid $4b in dividends, and you need to move to Ohio if you want to keep your job, yes the ohio with the cancer cloud. Intel is fucked.)

Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by eyegor on Thursday February 16, 2023 @02:18AM (#34815967)
Perhaps market segmentation, perhaps they heard of a vulnerability in their implementation that they couldn't patch (hence the microcode update). Intel loves market segmentation (server specific avx extensions, bfloat, ecc, overclocking), and I wouldn't be shocked to see them sell avx512 support as a "dlc" microcode update down the road.
Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by formerly_proven on Thursday February 16, 2023 @02:47AM (#34816116)
"The glibc problem"

You can schedule among heterogeneous cores, that's not really a problem. You simply have another bit for "task used AVX512" and let the task run without AVX512 so it faults the first time it tries to use it. The same stuff is done (or used to be done) for AVX, because if you know a task doesn't use AVX, you don't need to preserve all those registers.

The issue is that eventually someone will find that memcpy* is 4.79 % faster on average with AVX-512 and will put that into glibc and approximately five minutes later all processes end up hitting AVX-512 instructions and zero processes can be scheduled on the E cores, making them completely pointless.

* It doesn't have to be memcpy or glibc, it's sufficient if some reasonably commonly used library ends up adopting AVX-512 when available.

Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by Aardwolf on Thursday February 16, 2023 @05:31AM (#34817030)
> and zero processes can be scheduled on the E cores, making them completely pointless.

So because AVX-512 is fast, but E cores are slow, we should keep everything slow and prevent adoption of fast AVX-512 to prevent those E cores becoming pointless?

Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by thfuran on Thursday February 16, 2023 @08:21AM (#34818272)
Well, Intel is in the business of selling e cores.
Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by kristianp on Thursday February 16, 2023 @12:48AM (#34815533)
I wonder what Intel's plan is for the future here. Will a future efficiency core support AVX-512? Or will intel just abandon it on consumer in favor of a variable-length simd instruction set?
Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by paulmd on Thursday February 16, 2023 @01:36AM (#34815765)
Crestmont is the next e-core after gracemont and appears to still not have AVX-512.

The next p-core will continue to have AVX-512 but it will continue to be fused off.

This obviously is completely insane, like, even if you validated Raptor Cove already and you can't just take AVX-512 out, you're just going to keep including it in all your future designs too? It's not an insignificant amount of area, even on consumer it's probably at least 10-20% extra area, and it just looks crazy to introduce and then abandon it while your competitor adopts and supports it.

Only thing I can think of is that maybe they have some other instructions which utilize microcode intrinsics implemented on the AVX-512 engines... like how Turing implements its Rapid Packed Math (dual-rate FP16) support using the tensor engines.

It would be highly desirable for e-cores to implement microcoded AVX-512 support to break the heterogeneous-ISA problem, if nothing else.

Note that this also includes mobile going forward since mobile will have big.LITTLE designs too...

Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by kristianp on Thursday February 16, 2023 @02:53AM (#34816144)
Perhaps there wasn't enough lead time to get it into Crestmont after the decision to go with P+E for Alder Lake.
Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by rektide on Thursday February 16, 2023 @11:29AM (#34820861)
It's going to be somewhat hilarious if Intel makes a AMD Bulldozer alike architecture for E-cores where there are AVX-512 units shared among 2 or 4 e-cores.
Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by paulmd on Thursday February 16, 2023 @02:51PM (#34824781)
To go back to a past tech-screed: CMT as implemented by Bulldozer is just SMT with inefficiently-allocated resources. If the frontend (cache, fetch, decode, scoreboarding) and the FPU and retirement are all shared, what exactly did Bulldozer have that was unique to each 'core'? It was an integer ALU dedicated to each thread, that's it. And that's functionally identical to SMT but with a dedicated ALU for each thread. And if one of the threads has enough ILP to occupy two units and the other one isn't being used... why not let the thread use them both and get more work done?

https://news.ycombinator.com/item?id=34494484

So sure, let's do Bulldozer, a bunch of weaker but space-efficient threads (you know, e-cores) but put four threads on a single module sharing an AVX-512 unit, but let's also make it SMT so they can steal unoccupied execution units from other threads in their module. We could call it... Xeon Phi. ;)

https://en.wikipedia.org/wiki/Xeon_Phi#Knights_Landing

And sure maybe Bulldozer was "ahead of its time" but I think that probably undersells just how weak they are especially when threads in a module start contending for shared resources. Both Bulldozer and Xeon Phi get incredibly weak when multiple threads are on a module, the higher threadcount is offset by a reduction in IPC too. And while that is still probably a net perf-per-area gain, your application really has to like threads for it to be worth it.

I'm gonna say it: if you think Bulldozer was "ahead of its time" then so was Larrabee and Xeon Phi. Bulldozer was a first stab at this Xeon Phi idea for AMD. And in both cases I'm not 100% sure it was worth it. The market doesn't seem to have thought so.

Now again: the devil is kinda in the details. It all depends what is shared. If you can make it so the performance hit is really small except for the shared FPU, that's one thing - there's nothing inherently wrong with this idea, that's what the Sun Niagara series does too. But Sun Niagara is also noted for comparatively weak FP performance (it's an integer-monster for database work). It all depends on just how much duplication per-thread and how much shared resource and how much area benefit it gets you.

But like, take a Niagara core, and let's say we have a couple threads with an opportunity for a bunch of ILP and a bunch of unused execution units sitting there. Why should you not launch onto them? That's the CMT vs SMT question to me. And it's fine if the answer is "scheduling complexity" but you need to think about that question before just blindly pinning resources to specific threads.

And again none of this is to dump on e-cores specifically. Intel's P-cores are too big, they are like triple the transistor count of AMD for a smidge more performance. I think the long-term future lies in the e-cores for Intel, they will replace Coves with Monts eventually. Sierra Forest is the most interesting Intel product in a long time imo. AMD is in less need of e-cores, a Zen3 core only has about 2x the transistor count of a Gracemont core and to me that's fine, it's an SMT core with higher per-thread performance too, that's a reasonable sidegrade. AMD's strategy of pursing "compact" cores makes sense to me, they don't need a whole separate e-core, their P-cores are already area-efficient. They just are going to squeeze the last 10-20% out of it for area-optimized applications and call it a day.

But I'm just not sure the Xeon Phi/Bulldozer/Niagara concept has really worked all that well in practice.

Anyway it's also possible that instead of sharing one unit among four cores, they put a unit in each core but it executes over multiple cycles, like AMD did with Zen1/Zen+ and 256b vectors. Or you have two 256b units that fuse to become a 512b unit. That seems to have been the design trend recently, that's how AMD does their AVX-512 on Zen4.

But those kinds of changes are what I mean when I say "if there were changes in the vector units it would probably show up on the die shots". Crestmont die shots seem to show a pretty unchanged AVX unit from Gracemont - it seems unlikely they changed it too significantly.

https://www.semianalysis.com/p/meteor-lake-die-shot-and-arch...

https://twitter.com/Locuza_/status/1524441315441786881

Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by rektide on Thursday February 16, 2023 @08:58PM (#34829693)
Delightfully fun write up. I made a pretty superficial jab, but you've really painted a great picture of microprocessor design/tradeoffs as they've happened. Nice links. Just getting started on the Mike Clark of AMD interview & the background story alone has been delightful to hear, excited for the rest!

Happy 1 year-since-reveal to Sierra Forest's, announced February 17th 2022 at the Intel Investors Meeting. Definitely have a deep love of the "communication processor" grade gear, many-thread cloud systems, & this really can set the tone on what to expect from new Intel & massively-many-core systems. Wikipedia says it's Gracemont and Intel 4 (Intel 4 being due really soon, Gracemont from 2021), and Intel's recently said SF ought ship in 2024; I hope the plans for this chip have some room to evolve, or that we see more follow-up parts in good time. As you say, P-cores are just too damned big; Intel's been iterating on one big-core design for too long & it's gotten too big. Figuring out what we can do with smaller-core is much closer to the sweet spot for nearly all cloud louds: lots of processes of all sorts running; seems like turf for SMT (and you're 99.98% likely to be right about CMT but who knows, especially if we have a limited number of very big vector executors).

Two random mentions. I did really like Lakefield, which tried to be a ultraportable capable 1P 4E alike system, extremely well integrated. Also, I was incredibly fascinated by the semianalysis post mentioning a rumors, that the Intel Meteor Lake SOC die might have it's own integrated E-core island? That seems insanely bright; just turn off the core complex, a lot. Extremely smart for consumer computers. I think that task of actually understanding when real P-cores really should be brought up is an interest challenge facing consumer computing today; a place where Lakefield probably had good hardware but not enough software tuning to make the power-efficiency trade-offs that would have let it truly shine as exceptional.

Ok, so meanwhile AMD is going to be trying Zen 4c (starting with Bergamo, due about now-ish), with SMT2 (alike Zen4) & a huge number of somewhat cut down Zen cores (less cachesd, power, clocks). AVX512 is supposedly still included, but at an even more reduced rate than Zen4's reduced rate; awesome, sounds great. Not as relevant to the discussion so far, but just gonna mention: rumor-mill this week is that Genoa, the large Zen4 epyc chips, which were expected real-soon-now, are alleged to be facing some significant delays.

I'd love to see some SMT3/SMT4 show up again. Calling out Knights Landing as an SMT4 chip with a big vector unit is extremely on target, extremely interesting. That question of how bad the scheduling complexity really is is a compelling question, one that is much less visible than many of the knobs & dials that core design more visually alters (cache sizes & bus widths often being measurable via a random die shot at a trade show, for example). I suspect it probably is not really that huge a barrier. Still, we seem lament to explore much beyond SMT2, for the time being, but maybe that's fine.

Thanks again for the very fun posts paulmd. You have a ton of other delightful chip-design scuttlebutt in comments elsewhere too; this is a treasure to read.

Re:Intel Publishes Fast AVX-512 Sorting Library, 10~17x Fast... by paulmd on Friday February 17, 2023 @12:05AM (#34831009)
> Delightfully fun write up. I made a pretty superficial jab, but you've really painted a great picture of microprocessor design/tradeoffs as they've happened.

PaulGPT aims to deliver. Not actually an AI, just frequently accused of being one because I read some shit and it triggers Opinions and Tangents. And I don’t stake any position lightly, I have More Opinions why I’m right lol. It leads to Controversy. But I’m perfectly willing to defend my opinions against counterarguments and ultimately I’d rather mald and then admit I’m wrong.

> Definitely have a deep love of the "communication processor" grade gear,

I really wish Denverton had been more available. I can't even bite on surplus enterprise gear because there's barely any out there. Same with xeon-D, sick on paper but way too expensive. Plz make the intel accelerator thingy just onboard everything and also useful for zfs checksumming, that would be a gamechanger for ZFS on NVMe :\

> I did really like Lakefield, which tried to be a ultraportable capable 1P 4E alike system

Yes as I have commented, I really have been a fan of Kabini (Athlon 5350), Airmont (N2808), and Goldmont Plus (J5005) and recently I landed a pair of Skylake NUC7i7 for $125 a pop as well. They still are compelling for certain "microserver" applications given their extreme low price - a $50 CPU+mobo or a $125 booksize changes the expectations. 10 years ago it was $50 for a 5350 and mobo, 8 years ago it was $125 for a 2GB/32GB ECS Liva X, 3 years ago it was $125 for a J5005 NUC, recently $125 for a barebones with thunderbolt support? Yes, I like cheap machines even if their power is limited, $150 for a barebones machine that offers a faster capability at low TDP or some other unique capability is fine with me.

I am looking to use a RPi4 for a local NTP stratum-1 server with GPS and maybe use some of the nucs or other minipcs for freeIPA or a wireguard bastion or similar. With sufficient RAM a J5005 NUC actually made a really nice thin client during COVID WFH - swapping completely tanks performance and 16GB+ makes it perfectly fine even with lots of tabs/etc.

I have my eye on the Atlas Canyon NUCs too, which are finally available in quantity. The only things I don't like are the reduction to single-channel (Which seriously impacts performance vs the expected scaling, especially in iGPU) and the continued lack of Thunderbolt/USB4 - I like that it finally has M.2 NVMe and some other niceties but it really needs USB4 so you can plug it into more powerful stuff if desired. External expansion is going to be very baseline once USB4 reaches saturation and it’s not going to be that many years.

> rumors, that the Intel Meteor Lake SOC die might have it's own integrated E-core island? That seems insanely bright; just turn off the core complex,

Yeah that would be a cool workaround to the data-movement power penalties of chiplets/tiles. I mentioned elsewhere but having to have IF links powered up just to have cores idling along is an obviously dumb thing and yeah that’s a good solution for it, turn off the IF links and just run shit on the IO die.

Data movement and general idle-power is obviously the penalty of MCM and the more data you move across the more (smaller) chiplets the higher it is. It’s all just some new asymptotic limit of scalability (which direct bonding like cu-cu will change again, at the cost of thermals).

Intel claims EMIB is less than an interposer… I’m not really clear how the fiberglass-style interposer vs silicon interposer vs EMIB all stack up in practice. EMIB is 900% crucial to Intel’s future though, remember that TSMC offers advanced packaging solutions and (just like sapphire rapids getting their shit together so Intel has a viable cell library+process to sell to custom foundry) getting their shit together on EMIB is going to be a mandatory requirement for custom foundry’s success. The idea that a couple major intel chiplet/tile based products are seeing a lot of delays is generally concerning.

> I think that task of actually understanding when real P-cores really should be brought up is an interest challenge facing consumer computing today

I think phones have pretty good solutions for this but consumer and server both may have a different optimum for poweriness and boostiness. I think an optimum solution probably is not computable for the same reasons most “sufficiently advanced compiler-magic” doesn’t work - you’ll only really know at runtime. Still I am highly in favor of whatever hinting schemes we can come up with - good dev behavior can make a lot of difference.

Apple’s cores are really interesting in this area. Blizzard is extremely fast and small (like 1/3 of a Gracemont core transistors for similar-ish performance? I’ve never seen exact numbers but broadly that’s how it stacks up) and honestly Avalanche is sick, especially for any sort of JVM task. It’s just generally good at JIT, it’s not just x86 it really just crushes JIT compared to other architectures and JVM falls into that too. Cinebench is underselling the perf/w at load (because of less frontend load on x86) and I think the idle power stuff is undersold too. When it’s really on Linux I think the numbers are going to be impressive.

What’s a real easy answer to “when should I run p-cores”? Just have a real fucking fast e-core and if the e-core complex is getting overrun on a prolonged basis, pull out the really big guns.

> Not as relevant to the discussion so far, but just gonna mention: rumor-mill this week is that Genoa, the large Zen4 epyc chips, which were expected real-soon-now, are alleged to be facing some significant delays.

I did not know this, welp. Intel catches a bit of a lucky break. I think their mindshare is damaged though, even if Genoa were 6 months after SPR-SP nobody would really care tbh, intel’s been a lot later and AMD shows signs of better execution these days.

Sapphire Rapids is a good chip though. It’s not “lol throw your AMD shit in the dumpster” tier good, but, the game is back on, it’s good enough Intel can sell it, especially if they continue to be willing to cut deals. They need cash, they’re desperate, and this is obvious even now, if you’re paying more than half list price for intel you’re a complete fucking chump and quarter or less is more typical. Same for the 10-series during pandemic and the 12/13th gen prices, Intel is willing to deal to keep the fabs busy and keep the cash coming.

https://www.youtube.com/watch?v=_2yjjHzifL8

> Calling out Knights Landing as an SMT4 chip with a big vector unit is extremely on target, extremely interesting.

That’s never occurred to me either but the framing of “lol what if bulldozer but with AVX-512” rubbed both those nerves at the same time. No fuck you what if that already exists and everyone hates it!?!? ;)

(And then the transformer model takes over, beep boop paulGPT online, that's an interesting one for the following reasons ;)

> That question of how bad the scheduling complexity really is is a compelling question

Yes I agree, that is the money question. How much area did AMD save by having some fixed portions of the pipeline that don’t have to be scheduled between threads? How much does Sun save on Niagara? Or Intel on Core-SMT2 or Phi-SMT4? This would be an extremely interesting three-way from chips+cheese or similar, how well did all those sets of tradeoffs work and why were respective decisions made for those designs/use-cases? They all made different decisions around their frontend, I should take a peek at agner fog’s microarchitecture on those uarchs sometime.

> I suspect it probably is not really that huge a barrier.

That’s my guess as well. Alternating threads on the frontend may be the worst of it. If you have fixed decode/fetch units per-thread ala HyperThreading (or the option of a split of 5/0, 3/2, 2/3, etc) maybe that’s most of the squeeze. IDK though.

> Still, we seem lament to explore much beyond SMT2, for the time being, but maybe that's fine.

No… there is another ;)

https://en.wikipedia.org/wiki/POWER9

TBH I really really want a TALOS II setup, that is going to be one of those things that I snipe in 10-15 years when they’re cheap, it’s a neat piece of hardware. I have an AMD Quadfather and a KNL pcie coprocessor (we have a discord, folks!) and some other assorted nostalgia-tech too.

https://discord.gg/2qJXMTmE (expiring link to control spam, feel free to ping me on another tech thread later if anyone needs)