News for hackers. Stuff that matters.
Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by _a_a_a_ on Sunday September 08, 2024 @04:28PM (#41482938)
Seems very complex therefore very expensive (and possibly slow where it matters, at L2). Or it might just work.
Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by jauntywundrkind on Sunday September 08, 2024 @05:20PM (#41483315)
On the contrary!

Yes there's a lot of cache. But rather than try to have a bunch of cores reading each cache (sharing 96MB L3 for AMD's consumer cores), now there's a lot of separate 36MB L2 caches.

(And yes, then again, some fancy protocols to create a virtual L3 cache from these L2 caches. But less cache heirarchy & more like networking. It still seems beautifully simpler in many ways to me!)

Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by buildbot on Monday September 09, 2024 @03:48AM (#41486326)
L3 caches are already basically distributed and networked through a ring bus or other NoC on many x86 chips, for example sapphire rapids has 1.875MB of L3 per core, which is pooled into a single coherent L3. Fun fact, this is smaller than each cores L2 (2MB).

From https://chipsandcheese.com/2023/03/12/a-peek-at-sapphire-rap... “ the chip appears to be set up to expose all four chiplets as a monolithic entity, with a single large L3 instance. Interconnect optimization gets harder when you have to connect more nodes, and SPR is a showcase of this. Intel’s mesh has to connect 56 cores with 56 L3 slices.”

Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by jmclnx on Sunday September 08, 2024 @04:39PM (#41483021)
To bad the mainframe business will not be spun off from IBM. Then you may see innovation, but IBM see it as a cash cow.
Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by wmf on Sunday September 08, 2024 @05:40PM (#41483457)
Do the customers want innovation?
Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by kev009 on Sunday September 08, 2024 @08:22PM (#41484417)
The CPUs has been a tour de force from the S/360, they have never relented, so empirically yes the customers care a lot or they wouldn't keep doing this.

The software side seems to be more a tale of dichotomy. The MVS lineage is technically impressive but undoubtedly bizarre and old feeling. The TPF lineage seems like eventually somewhere the cloud movement will dip for certain cases so it is ahead of time. Linux is neither stale nor avant-garde, I guess that is their strategy to remain "contemporary". VM was always the most delightful one but internally forever the odd one out.

Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by rbanffy on Monday September 09, 2024 @06:16AM (#41487025)
I fail to understand where you think IBM has been lacking in innovation - Telum and Telum II (as is POWER10) are very impressive designs the likes of which you won't see anytime soon on x86 or ARM space. They target a relatively small segment of the market where people will pay whatever it takes to reach 99.99999% uptime or the most transactions per second.

If mainframes were not competitive at that, they would have ceased to exist a long time ago.

Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by nxobject on Monday September 09, 2024 @07:33AM (#41487516)
Or, at the very least, IBM would have sold a product that required far less investment in unique technology, e.g. software emulation on commodity hardware.
Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by rbanffy on Monday September 09, 2024 @09:12AM (#41488210)
Everything from the ground up is designed for business transaction performance. That includes the OS, which is somewhat limited for a lot of other uses.
Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by mikewarot on Sunday September 08, 2024 @06:47PM (#41483899)
That's a huge amount of effort to let most of the transistors in a computer (in the RAM) sit idle most of the time. Surely there are viable non-Von Neuman alternatives that could be spun out into general purpose computing.
Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by MichaelZuo on Sunday September 08, 2024 @07:24PM (#41484127)
The vast majority of transistors in any modern CPU are ‘idle’ at any given moment.
Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by kev009 on Sunday September 08, 2024 @11:53PM (#41485303)
With the cascade of different clock domains on a core and package, the control loops can spend that thermal budget effectively elsewhere; idling is one of the benefits of CMOS.
Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by rbanffy on Monday September 09, 2024 @06:21AM (#41487054)
I believe GP was counting the transistors in DRAM, not only those on the CPU.

If someone is really into high performance, it's ideal to never have to wait for DRAM, either with predictive fetches or explicit cache warming. For that, the more cache you have, the better.

Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by yencabulator on Monday September 09, 2024 @01:07PM (#41490727)
Perhaps most weirdly, we've reached the point where that is actually desirable. Power+heat is the limit now, and slapping on some extra circuitry that is only used for some operations makes the chip perform better.

https://en.wikipedia.org/wiki/Dark_silicon

Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by creer on Monday September 09, 2024 @04:00PM (#41492902)
There have been quite a number of attempts at computing in memory kinds of chip design through computing architectures. So far with very limited success. It's not impossible but in search of good applications.
Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by tedunangst on Sunday September 08, 2024 @06:50PM (#41483912)
I wonder what workloads would benefit from having an L4 victim cache on another CPU, but that other CPU doesn't need its own L2 cache.
Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by kev009 on Sunday September 08, 2024 @08:05PM (#41484340)
The claimed latency for it seems not far off from some other vendor's L3 caches which may be by virtue of rethinking where to share and therefore paying interconnection coherency taxes.

The innovation here seems to be adaptive sizing so if by whatever algorithm/metric a remote core is idle, it can volunteer cache to L4.

Presumably the interconnect is much richer than contemporary processors in typical IBM fashion and they can do all the control at a very low level (hw state machines&microcoding) so it is fast and transparent. It will be interesting to hear how it works in practice and if POWER12 gets a similar feature since it shares a lot of R&D.

Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by _a_a_a_ on Sunday September 08, 2024 @07:25PM (#41484128)
At a guess, a single thread which benefits from as much cache as it can get.
Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by tedunangst on Sunday September 08, 2024 @08:29PM (#41484453)
Sure, but having to buy entire CPUs filled with idle cores to scale up cache seems very expensive.
Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by kev009 on Sunday September 08, 2024 @08:43PM (#41484521)
These cores are typically licensed with class/restrictions so in absolute terms yes but in the financial engineering of how the system is delivered with excess and restricted hardware no (see core types on the prior/shipping generation here https://www.ibm.com/downloads/cas/6NW3RPQV)

There are probably design reuse and RAS considerations that make it not currently worthwhile to i.e. have a distinct physical design for SAP or whatever cores.

Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by sillywalk on Monday September 09, 2024 @02:25PM (#41491727)
I don't know if it's still the case, but in terms of RAS, the Z/Series CPUs from ~2004 had duplicated/compared instruction-fetch/decode and execution units.

https://pages.cs.wisc.edu/~remzi/Classes/838/Fall2001/Papers...

Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by nxobject on Sunday September 08, 2024 @10:32PM (#41484986)
I wouldn't be surprised as well if there was some binning that occurred – the dies are huge, so why not overprovision in the design? (Although erring on the side of slighlty more surprise in the case of some binning, since IBM mainframes seem to exist beyond the laws of commodity economics, and it looks like they're using a 5nm node.)
Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by _a_a_a_ on Sunday September 08, 2024 @08:33PM (#41484480)
I said this is where it would help (AFAICS), not that it was the best solution.
Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by nullc on Monday September 09, 2024 @12:52AM (#41485547)
the CPU's only appear to use about 1/3rd of the die area. Most of the space is cache.
Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by twic on Monday September 09, 2024 @04:31AM (#41486539)
People buy whole machines to run memcached!
Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by unnah on Monday September 09, 2024 @03:06AM (#41486120)
At a basic level, anything with a working set on the order of 360 MB should benefit from 360 MB of combined L3 with a worst-case latency of 11.5 ns, regardless of which parts end up in which L2 slice (and the cache allocation heuristics described in the article look pretty smart to me). Similarly, if you have a total working set of a couple of GB then the 2.8 GB combined L4 at 48.5 ns latency should be great. Is there any other hardware on the market that can offer so much memory at such a low latency?
Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by lostmsu on Monday September 09, 2024 @02:53PM (#41492116)
/Uneducated/ these latency numbers seem large to me. DDR5 memory sticks I browsed yesterday for a home PC listed 10ns first word latency.
Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by unnah on Tuesday September 10, 2024 @04:07AM (#41498390)
If the data is not in cache, it takes quite a while longer from the time the CPU core issues a load instruction for the results to get back to the next instruction. The CPU core has to first try L1 and L2, do a TLB lookup to convert a virtual address to a physical address, send a request to L3 over an on-chip connection, then after L3 lookup fails the memory controller has to transfer a 64-byte cache line from the main memory, and the results are then sent back to the core...

Have a look at the section "Cache setup" at https://chipsandcheese.com/2024/08/14/amds-ryzen-9950x-zen-5... for some real-world latency values. Once we're talking about a 100+ MB working set (i.e. DDR5 instead of cache), a top-of-the-line Ryzen 9950X has an access latency of about 100 ns. There is also some older data for a wider variety of CPUs at https://chipsandcheese.com/memory-latency-data/ - and there the older IBM z15 is in a class of its own.

Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by virgulino on Sunday September 08, 2024 @08:17PM (#41484393)
"Why Do Mainframes Still Exist? What's Inside One? 40TB, 200+ Cores, AI, and more! - Dave explores the IBM z16 mainframe from design to assembly and testing. What's inside a modern IBM z16 mainframe that makes it relevant today?" - by Dave Plummer.

This is an amazing 23-minute video by the Microsoft programmer who developed the Windows NT Task Manager, among other things. He visits IBM and talks to engineers about the Telum chip architecture (Hot Chips 2023), used in the z16 mainframe. Special attention is paid to the cache.

https://youtu.be/ouAG4vXFORc

Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by pclmulqdq on Monday September 09, 2024 @01:10AM (#41485631)
Dave Plummer seems to be a bit careless with facts in his videos, and I wouldn't generally trust him as a source of information.

In an episode on hard drives, he talked about how drivers for hard drives still report a constant number of sectors per track, so they must have a physical layout that matches that. Hard drive manufacturers are open about the actual layout of their drives and that they virtualize the hard drive for the OS so that it behaves well.

A few Microsoft engineers also dispute a lot of the facts of his stories about the development of the start menu.

Caveat emptor.

Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by froh on Monday September 09, 2024 @02:59AM (#41486093)
do you happen to have factual.insights into false reporting on the Mainframe in this specific video?

afaict (and I've worked with mainframes for a couple of years) this is spot on. poor signal/noise ratio but the facts are right.

Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by nxobject on Sunday September 08, 2024 @10:39PM (#41485004)
Another good article by Chips and Cheese on Telum II @ Hot Chips – an interview with IBM directors of development at the processor and system levels. [1]

[1] https://chipsandcheese.com/2024/09/05/an-interview-with-susa...

Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by chihwei on Monday September 09, 2024 @04:38AM (#41486574)
Mainframe sounds like a good idea to solve many of today's problems. Why don't people start thinking about making a RISC-V or x86 Mainframe?
Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by rbanffy on Monday September 09, 2024 @06:12AM (#41487003)
A mainframe is just a very large server, with lots of reliability features (RAID-like memory, fault detection and mitigation, redundant components, etc) and lots of intelligent peripherals that offload work from the CPU so that it can spend as much time as possible running application code (and don't waste time handling interrupts, assembling network packets, dealing with IO, etc). A lot of these offload functions are baked into the ISA, making it a VERY CISC machine.

I believe Unisys still makes x86-based mainframes running MCP.

Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by neverartful on Monday September 09, 2024 @09:41AM (#41488461)
And loads of IP that IBM will defend vigorously with any infringement.
Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by rbanffy on Monday September 09, 2024 @10:30AM (#41488896)
They invest a colossal amount of money creating those patents. There are lots of bullshit patents in this space, but IBM is not playing that kind of game.
Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by beAbU on Monday September 09, 2024 @10:26AM (#41488860)
Modern cloud environments are basically virtualised infinitely scalable mainframes.
Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by yencabulator on Monday September 09, 2024 @01:16PM (#41490858)
Modern cloud environments tend to be aimed at running multiple independent workloads well on a huge server. Mainframes are generally aimed at running a smaller number of large workloads well on a huge server. Sort of analogous to multithreaded vs singlethreaded performance in CPU benchmarks.

My personal take:

The typical x86[1] is a sports car. Gets going fast, reaches most destinations fast, not great for driving for several hours, and not great at moving lots of cargo.

A mainframe is a freight train. Somewhat slow to get going, but can haul large amounts of cargo without breaks for a long time.

Mainframes weren't built for an interactive, highly variable, query-response workload; they were built for the classic overnight/monthly batch job that streams through a large amount of data.

[1]: It's not about the CPU, it's about the architecture around it, like this article talks about cache, expanded to I/O etc concerns.

Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by MobiusHorizons on Wednesday September 11, 2024 @02:27AM (#41508561)
That’s how I had always thought about mainframes before, but the focus on low latency here seems to suggest a different purpose (more sports car like than any x86 server cpu) is this a different kind of mainframe?
Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by yencabulator on Wednesday September 11, 2024 @10:09AM (#41511557)
The freight train part of my thinking is admittedly dated .

This is an impressively fast design, but you'll get much more x86 for the same money -> x86 wins when you can scale out.

Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by nxobject on Monday September 09, 2024 @06:06AM (#41486977)
Random question to anyone who might know anything about this – is the uarch internally POWER, like System i (AS/400)?
Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by rbanffy on Monday September 09, 2024 @06:18AM (#41487034)
No. It's completely different. Unlike System i, it's also fully documented (in the rather impenetrable "Principles of Operation" red book).
Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by nxobject on Monday September 09, 2024 @07:28AM (#41487483)
Ah, thank you! I would have never thought to search under that name. It makes sense for such a non-mainstream architecture, but I wish there were (even reverse-engineered) resources as well on "this is what the execution engine looks like". There are instructions there that clearly scream "very extensive microcoding is going on here", e.g. vintage EBCDIC/BCD conversion, string/stream instructions, control channel supervision, etc.
Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by ruslan on Monday September 09, 2024 @10:50AM (#41489053)
It seems to me the mentioned "Principles of Operation" document describes a virtal machine compatible with good old System/370 what's now known as IBM z/Architecture. But the Telum CPU itself runs on Power10 cores or alike RISCs.
Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by rbanffy on Monday September 09, 2024 @12:46PM (#41490409)
No. The cores are not POWER10 at all. While there is a ton of microcode and interception magic happening (and nothing but the hypervisor, PR/SM IIRC, runs on metal - the hypervisor exposes partitions to the zVM environment) the cores are still very different from POWER and run their own s390x ISA.

You might be confusing it with the AS/400 CISC ISA, which exists as an emulation layer on top of POWER, since all IBMi machines are almost identical to their POWER counterparts.

Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by sillywalk on Monday September 09, 2024 @02:06PM (#41491488)
There isn't an emulation layer for AS/400 though, it's native POWER code.

The AS/400 / 'i' are descendants of the System/38 and implement a "Technology Independent Machine Interface". Applications target this high-level interface, rather than the underlying hardware. Before first run (or when they're installed?) applications get compiled from abstract Machine Interface code to native code.

Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by Pet_Ant on Monday September 09, 2024 @12:41PM (#41490347)
So looked into this in Wikipedia and it seems like it is "z/Architecture" which is basically a 64-bit extension of the s/390 instruction set which was the evolution of the s/360 which was in a sense the first instruction set (it was the first instruction set intended to be implemented by multiple CPUs). It looks like software for the s/360 should still be able to run unchanged on the modern CPUs but there was some mention that not operating systems.
Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by MobiusHorizons on Wednesday September 11, 2024 @02:23AM (#41508542)
These mainframe COUs are really cool. But I still don’t feel like I understand where they make sense. I’ve heard they are used in finance, but what does that mean? High frequency trading? Processing credit card payments? Managing bank balances? I’ve always thought of mainframes as batch / offline systems, but this sounds much more online and low latency.
Re:Telum II at Hot Chips 2024: Mainframe with a Unique Cachi... by sillywalk on Wednesday September 11, 2024 @10:47PM (#41517101)
Transaction processing was basically invented on mainframes. High-throughput, high-volume, low-latency. I'm not sure about high-frequency trading, but in article nxobject linked above they say ~70% of financial transactions are processed on mainframes. So credit card transactions, bank transactions. Historically, things like airline reservations were also done on mainframes.

https://en.wikipedia.org/wiki/Transaction_Processing_Facilit...