"feature-full" is also quite a stretch when it comes to software when most modern software needs to be ported or run in a Linux LPAR.
In addition I'd question that not getting access easily is the main reason for the lack of adoption. It might be part of the problem but the other problem is that the thing is a niche product and IBM doesn't seem to be utterly successful in either explaining what the niche is nor how to extend it.
Consider that partitions are completely isolated from each other. Not some pesky soft isolation either, it's all done in hardware. In practice, every partition in a mainframe is a different logical yet isolated mainframe.
Mainframes are built for different kinds of workloads. They are not cloud machines. They are batch machines with 6 or more nines of uptime.
In my current job, a mainframe would be useless. However, fore mission critical core services which needs predictable latencies (bank transaction engines, central big databases etc.) I'll take a mainframe all day, every day.
"Completely isolated in hardware" as in isolated by a software hypervisor (PR/SM) that doesn't have a whole lot of hardware support?
> Mainframes are built for different kinds of workloads. They are not cloud machines. They are batch machines with 6 or more nines of uptime.
That's kind of the point. What exactly is the niche, i.e. which new customer with exactly what software and latency requirements would switch from their current system to a z? IBM won't tell you apart from buzzword bingo.
I'm guessing that many of the terms in this thread are buzzwords for many of us reading it :), since relatively few people work on mainframes.
I had not heard of the term buzzword bingo before, so I googled it:
https://en.m.wikipedia.org/wiki/Buzzword_bingo
Interesting. It aligns with my corporate experience in a few cases.
I'm nominating this video for the next Oscars, Grammies, or whatever the heck the movie awards are called (I don't keep track of such stuff). Bet it'll win gold in the under-1-minute category ;)
Are you saying the GP's statement is outright false?
Moreover the norm for zLinux system is to isolate guests with zVM, not LPARs. LPARs are more likely to be boundaries for chunkier workload definitions - that advice may have changed since I last cracked open a redbook; zVM offers similar isolation to what you'd see with KVM, VMWare, or other hardware-assisted hypervisors.
They’ve spent the last 60 years developing and refining everything for extreme levels of uptime and high availability. It’s truly unique to these mainframes because IBM controls every aspect of it. The closest equivalent of that kind of vertical integration is your Macbook Pro.
I highly doubt there are any brand new mainframe customers these days. But many of the biggest companies you know and use every single day have tons of workloads that will never move off of it.
But this doesn't say anything about "unbelievably powerful" or "feature-full" as in the OP?
The niche customer claim probably is "will never move off of it because nobody ever wants to touch millions of lines of COBOL" then that's fine. It's likely sane from a business perspective to continue using them as long as maintenance burden is manageable. Luckily managers in those more conservative companies consider full rewrites dangerous, rightly so. But in order to claim otherwise (i.e. unbelievably powerful and thus for everybody) we need to see numbers.
I pointed out that Redbook about HA features since it’s a major differentiator of mainframes even today, but there are also these that document other features:
https://www.redbooks.ibm.com/redbooks/pdfs/sg248950.pdf#page...
https://www.redbooks.ibm.com/redbooks/pdfs/sg246366.pdf#page...
You can find 1 or more books that go into a deep dive of every chapter.
IBM has taken input from the biggest companies in the world over many decades to run as many of their workloads as possible. They’ve honed everything in their software and hardware to do so, down to developing specific cpu instructions to support specific use cases. If all of this doesn’t convey an extremely feature rich system, I’d like to hear why you think otherwise.
For example, you can say "this operation will take at most 3ms" in an RTOS, and it'll never exceed that number. If you're running on a fixed-frequency system, you can even say that I expect an answer in 2.8ms all the time, every time.
In a mainframe this is a bit relaxed. You can say that latency is <3ms 99% of the time, <3.1ms in 99.999% of the time and <3.5ms in 100% of the time.
IOW, you never think/say "somebody is running a heavy load on the host, and I also slowed down because of them".
Some RTOS even doesn't consider late answers as acceptable.
I am not sure if that continues off-package, but if it does, a drawer with 4 chips of 16 cores each will have 2 GB of off-chip cache and a full-sized Z with 5 drawers would have 10 GB of off-drawer cache (at this level it's probably not that much faster than same-drawer memory).
As for the RAIM, I think it's safe to assume it has a very wide path (n-1 modules) to the sockets and that aggregate bandwidth will not leave the 4 sockets starved (even if latencies suffer because of the redundancy, but you can replace a defective memory module with a running computer without it having to pause.
You buy a DGX A100, or a cluster of them, for training and running large deep learning models (or for doing "traditional" HPC).
IBM's solution is more a small inference engine that is part of the CPU, so you don't need to move you data off-chip when doing a little bit of inferencing as part of some other workflow. I don't work with mainframes so I could be talking out of my behind, but maybe something like DL-assisted fraud detection as part of processing bank transactions?
Yes.
2 chips make a dual-chip module with 512MB cache, 4 dual-chip modules make a drawer with 2GB cache, and a 4 drawer system with 32 chips makes 8GB cache.
"The L3 and L4 virtual design across the fabric provides 50% more cache per core, with improved latencies. The idea is that software and microcode still see two separate caches. These caches are shared and distributed across all eight cores via a 320 GB/s ring and across the integrated fabric. Horizontal cache persistence should further reduce cache misses as well. Specifically, when a cache line is ejected, the system looks for available cache capacity on other caches, first on the chip and then even across the 32-chip fabric."[0]
"The accelerator itself delivers an aggregate of over 6 TFLOPS of 16-bit floating-point throughput per chip to scale up to roughly 200 TFLOPS per system. 1024 processor tiles in a systolic array make up the matrix array, and 256 fp16/32 tiles make up the accelerator for computing activations and include built-in functions for RELU, tanH, and log. The platform also provides enterprise-class availability and security, as one should expect in a Z, with virtualization, error checking/recovery, and memory protection mechanisms. While 6 TFLOPS does not sound impressive, keep in mind that this accelerator is optimized for transaction processing. Most data are in floating-point and are highly structured, unlike in voice or image processing. Consequently, we believe this accelerator will deliver adequate performance and is undoubtedly much faster than offloading to another GPU-equipped server or running the inference on a Z core. The latency of off-platform inference can cause transactions to time out, and inference does not complete"[0]
"Intelligent Prefetcher and Write-Back – 120+ GB/s read bandwidth to internal scratchpad – 80+ GB/s store bandwidth – Multi-zone scratchpad for concurrent data load, execution and write-back Intelligent Data Mover and Formatter – 600+ GB/s bandwidth – Format and prepare data on the fly for compute and write-back"[1]
[0] https://cambrian-ai.com/wp-content/uploads/edd/2021/08/IBM-T...
[1] https://hc33.hotchips.org/assets/program/conference/day1/HC2...
In any case, I have played with their LinuxONE Community Cloud service (running on the previous-gen z15) and it's very fast. The impression I get is that it doesn't need to wait for IO. There is a ton of very clever engineering on those machines and the z16 is a technological wonder.
https://www.anandtech.com/show/16924/did-ibm-just-preview-th...
It does a reasonably good job at comparing mainframes to a regular server setup. It's not about SPEC or GFLOPs etc.
Maybe things have changed in the last 10 or 15 years, which is the last time I had a legit mainframe at a day job, but back then none of those things were true if you looked at it more than 20 minutes.
I seem to remember nothing being included in the base mainframe… when you started to add things like DR and data duplication and virtualization, it became extremely expensive. Like on the DB side, effing Oracle was much cheaper.
And software licenses are one of the reasons why LinuxONE machines exist - they don't run z/OS, so you don't pay those licenses. You can even start a dozen VMs under an LPAR and run your Kubernetes cluster as if it were running on more common hardware that just never, ever fails. IIRC, you can run a special version of z/VM to manage your Linux VMs if you don't want to run Linux on the LPAR and use KVM for your VMs.
I would argue Kubernetes talent is cheaper than mainframe talent these days because it's ubiquitous.
Also, Kubernetes talent is far from ubiquitous. That sounds more to me like you're counting anyone who has successfully deployed one time.
But in regards to K8s talent, lots of the people who think they know Kubernetes in production but have never had to actually upgrade the cluster, or go through the process of having to update manifests, deployments, and CSIs, and having to actually deal with api removals.
I do not want to make it see that LPAR isolation is just waiting to be compromised, but security-by-unavailability also plays a part :)
How many security researchers are even familiar enough with the concept of a mainframe to consider looking for an LPAR breakout, let alone have access to the necessary hardware?
Both of these were anticipated in security papers dating back to the 1980s. It wasn't practical to use those types of exploits on 10mhz VAXen.
For context, there was about 1 admin for every 400 servers at the tech company, and that was for the entire tech stack (LAMP).
Mainframes require a lot of care and feeding. In my experience, having worked at 3 different companies who relied on them (education, aero and finance), their capex is higher, their opex is higher and their uptime / reliability is entirely dependent on the facilities / staffing.
This whole time I just thought mainframe was an older style word for a large rack based server or server room. Like 'cloud' storage for someone's computer running miniserve.
What is the real difference between a mainframe and, e.g. a rack full of H100s, or rack full of 100GBps networking stuff, or some nice stack of 12x blades with 8x 256 core CPUs?
Why or how does a "mainframe" have more power than that?
It’s designed to work as a single high-availability machine with a focus on I/O speeds, rather than a PC cluster that implements high availability at the application level and communicates over IP. You can hot swap faulty components without turning it off. I haven’t ever used one, but I believe you write application code without worrying about such details - the hardware and the libraries you link against will just “make it work”.
Perhaps my knowledge is slightly off though.
Essentially the bunch of boxes model(k8s being the new kid on the block) has been trying (and mostly failing) to provide what mainframes have been providing for 60+ years.
Which is being able to treat your workloads as just a random virtual job you can push wherever and let it run while also giving you ridiculous uptime.
Mainframes are basically the hardware infused uptime deliver machines. They can and will offer 5 9's without any trouble. AWS, Azure, Google's cloud, none of them can deliver that amount of uptime, they ALL have failed repeatedly, so much so, that they purposely try to obfuscate their downtime records. Many don't make any historical data available.
k8s and the like have been trying and failing at reliable uptimes. Sure we've arguably been making some progress, but your average self-hosted k8s team has full-time dedicated teams of people that do nothing but babysit k8s. How many staff do your average mainframe org dedicate to keeping the mainframe alive? Usually 1 person, maybe two. Of course the price you pay IBM or whoever you choose as a mainframe provider will help offset the staff savings from your k8s team :)
It's not about raw compute power. It's about keeping a workload alive for as long as you can deliver power to the mainframe. i.e. the Mainframe promises to deliver uptime for as long as you can keep power to the machine. As parts fail, seriously any part: memory, CPU's, disks, backplanes, it doesn't matter. Mainframes can route around the failed part and you can replace it without turning anything off or affecting your workload. This means your mainframe is sized larger than your workload of course. It's not like the fundamentals of compute change in that regard.
The question then becomes, is the juice worth the squeeze? If your entire business model requires uptime, then you best really, really care about uptime. There is a reason the Visa and Mastercard networks have basically never, ever been down. It's because they know their business only exists as long as their network works. When you want uptime at all costs, you don't run k8s(or whatever the latest craze is tomorrow), you run a mainframe.
Most of us get more uptime than we need with insert favourite cloud provider here. Uptime isn't something they actually sell when you read the contracts you sign. Uptime is just marketing spam.
AFAIK VAX's are still around, though good luck trying to buy a brand new one. ;)
I should point out, I was talking about mainframes in general, to include Vaxen, Fujitsu, Unisys, etc. IBM isn't the only mainframe game in town.
VAX -> Alpha -> Itanium -> x86.
I worked on VMS at the tail end of the VAX days and then through the Alpha days. Overall I thought VMS was pretty neat. Built in file versioning, the cluster stuff was awesome, arguably better than what k8s and the like can even provide now. They had uptimes in decades, that's not something k8s or any of the other new clustering stuff can even dream of achieving right now.
However, their website and staff members are so brain damaged that I just couldn't bring myself to waste time with it.
Specifically, their website can't even take apostrophe's (nor many other non alphanumeric symbols) in passwords.
Reported that to them, and their staff not only didn't give a shit, they were adamant that only "$#@!%*&" are all the non alphanumeric characters anyone might ever need (in passwords). :(
Thus my complete lack of interest in VMS from that point onwards. ;)
With k8s on a major cloud provider you get 99.99% for what? 150$/month? And there is no maintenance what so ever.
Full team managing k8s is a lie, even on prem, what exactly there is to manage on a daily basis?
Gross understatement of costs don't you think? Last time I checked it was a base fee of 150$/month for managed k8s + whatever computing resources you end up using.
k8s is brittle, all the other competing tools are also fairly brittle, so it's not like k8s is really alone here. This is why we keep replacing the newest mess of virtualization every once in a while, someone eventually gets fed up with whatever the current crap is and writes a new one. It becomes popular, breaks in new and unique ways, rinse and repeat.
k8s is not even a decade old at this point. Mainframes have been around with 5+ 9's of reliability for 6+ decades.
I'm not overselling what mainframes provide. I think you just have selective memory.
99.99% is not what Mainframes provide, they provide many more 9's than that think 7+.
Most of us don't even need 99.99% uptime, so most of us probably shouldn't be buying mainframes. If you DO need severe uptimes, then you either have a huge oversubscription and dedicated teams of people around the clock babysitting things or you buy into mainframes. You absolutely don't buy AWS or Google Cloud or Azure and say, that's good enough, because their uptimes are just marketing speak, not reality.
https://www.ibm.com/docs/en/zos-basic-skills?topic=concepts-...
A decade ago there was an Eclipse version of this..
This guy is showing all three ways:
[... for new users]
For expert users? I'd pit it against any interface you want.
No-mouse + hands-on-keyboard + tab is hard to argue with.
Sadly, the number of people who remember when GUIs could be driven from memory by keyboard-only has dwindled.
Watch a 10-year veteran insurance employee use a green screen.
Or as a counter-example, try to tab through your modern app du jour. (Modern developers often don't even bother to set indexes correctly)
Yes! I knew an Insurance customer service op who self-described as having no computer skills, but when I saw her using Reflection it was like watching a speedcuber savant.
I would explain to them to stop after each screen or keystroke, so I could document it.
Despite being smart people and understanding the ask, almost every SME I worked with failed at some point, blazing through multiple screens without even realizing it.
That's some strong muscle memory there!
Linux CLI with a good modern shell and Python versus 3270 block mode and Rexx would be an interesting match up.
Similarly Emacs vs XEDIT.
This is so very true, and it's tragic.
Watch an experienced keyboard user, such as many blind users, navigating Windows and it's a wonder of efficiency and speed. It makes the Vim prowess of many Linux folks look very sad, because Vim is just a single editor, while with Windows' keyboard UI, the entire OS and all its apps are operable at this speed. Desktop, file manager, all bundled apps, all 3rd party apps... everything has one global unified keyboard UI.
The idea of "Unix as your IDE" is a deep, deep cut into Unix lore.
The creators of JCL admitted later that the original sin of JCL was that it explicitly tried to NOT BE a programming language. Everything "programmable" evolved out of real-world needs, creating the mess that it has become.
You can see it today with "programming in YAML", whether it is Ansible, Kustomize, Kyverno (K8s manifests get a pass because they are just serialization of non-programmable thing), or any other weird "we started to add functions to JSON/YAML dumps"
From my own musings about making such a tool :)
I was an intern at IBM UK in the late 80s and I had a 3279 on my desk connected to a 3090 mainframe. My recollection is that it was fine for editing and running code: sharp, bright text and snappy response. The clicky keyboard was nice too - probably the best I've ever used in the decades since - although the height of the enclosure would probably hurt my wrists now. The key caps had APL symbols on them too, which was interesting and prompted me to learn about that language a bit. Mostly I was writing APL and REXX code for image processing.
Some of the f/t employees I worked with had worked on the development of the 3297 at Hursley Park. They had some interesting stories.
I really doubt they are any more powerful than the best x86 rack servers. Their featurefulness is super dated. They are definitely not cost effective (good luck buying it from IBM without a huge contract).
I pity the company that gets into mainframes today. Does that even exist?
It's a proprietary nightmare.
I do agree it is debatable whether it is beter than x86 most of the time cost savings projected when using a LinuxOne it is based on licensing. In the past oracle licenses where per core and we did a project where 8 ifl’s replaced 140 x86 cores
It seemed rather pointless to me. The DB2 server was serving queries just fine and when we were working on setting up a microservice that would allow viewing claims history, the storage and server costs for being able to provide queries against three years’s claims were prohibitive.
But they were still persisting down that road when I left.
The problem with access is not access as such, but the fact that the community hardware that you can access for free tends to be heavily oversubscribed (or maybe connected to a mismatched storage system).
Turned out that s390 couldn't fuse cmp+jmp when the operand was a 8-bit value (_Bool). (That was a while ago, maybe current generation mainframe HW can do it?)
Mainframes are really powerful for some specific domain which most people on HN don't work in.
Another take is that the most valuable company in the world ( tech one ) don't run their core businesses on mainframes.
Yeah, but the cases where mainframes run the show are the ones where stuff really matters, where actual human lives and existences are at stake: banks, insurances, governments, airlines, logistics. If Google has a day long outage, not much of value will be lost... but a large US bank, stock exchange or MC/Visa suddenly failing? That can be enough to trigger an actual bank run with all its consequences.
(Of course, the argument can and should be made that no company should grow large enough to even be able to be such a threat, but that one will have to wait until at least 2028 to get solved)
To be clear, this only applies to regular mainframes running z/OS. LinuxONE machines, as the name implies, only run Linux where you'll have all the usual devops/SRE responsibilities.
Surely there should be some specifications behind this? Benchmarks?
https://www.anandtech.com/show/16924/did-ibm-just-preview-th...
You can open a terminal and come back in a month and it will still be there. Unlikely kunernetes where containers regularly go down.
You can obviously achieve high reliability with Kunernetes, but you'll need queues and retry logic, that you don't need with SystemZ.
I assume the achievement part is that any CPU can be replaced transparently to the terminal running?
Not a big deal today, but in the 80s hard drives died more frequently and a bank losing just 5 seconds of transactions could cost them millions.
It wasn't cheap, but it made a typical "modern" high availability setup look like a crude homemade toy.
But as impressive as it was, there are just very few places where that impressiveness provides enough value to justify the cost. And having to deal with IBM.
You mixing apples and oranges there, by comparing kubernetes workloads to mainframes. Kubernetes isn't really designed to serve long-persistent workloads of that fashion. Although tbf I've had VM's that last for years and real hardware (albeit Sun) that's had over a decade of uptime, so I'm not sure what all the fluff is about.
It's just not compatible with consumer CPUs (well, ddr5 is, but ddr4 and lower wasn't) because Intel was voluntarily segmenting the market to upcharge for server CPUs.
I've heard it said a couple of times that working in cloud is closer to working on a mainframe than most want to admit.
As an example, in many main frames, you can configure them with a spare set of CPUs, and if one of your CPUs fails, the replacement is brought online automatically and transparently, the code you write doesn't have to know about anything related to the failure.
It's definitely intriguing if it were.. I don't know, available to tinker with?
From way back in 2004[1], Commercial Fault Tolerance: A Tale of Two Systems compares the reliability / philosophy of IBM mainframes with Tandem Servers.
[0] https://arstechnica.com/information-technology/2023/07/the-i...
Could you say where and what "not very much money" is? Like, are we talking $5 DO droplet equivalent or only a few hundred bucks a month? (but to be clear, this is a serious question; I'm a sucker for weird options and would be interested in running a s390x Linux box if it doesn't have too bad price+caveats)
For fun, then try picking z/OS instead of Linux and see what that software license costs ($/hour). Take a guess first how much you think it'll be and then see if you're right.
however, cost-effective? only if you're sitting on piles of cash.
IBM mainframes are unique. They have 75 years of incredible research (IBM is often the #1 patent inventor in the world, year after year) into a platform that they completely control. This has produced an absolutely bulletproof product. But you gotta pay for that.
Your other point that it would take a team of engineers to not-even-compete; yes, you're right about that.
I'd love to own one, but I'd rather own my house first.
Having run IBM's own software (WAS and WAS derivatives) on both x86 and IFL, and found the same applications with the same workloads having a ratio of 1.5-2 x86:1 IFL, while the IFL itself is 1-2 orders of magnitude more expensive than an x86 core, that zVM is an order of magnitude more expensive than VMWare's hypervisors, that memory is two orders of magnitude more expensive than x86... well. I think you're long on the evangelism and short on the reality.
Once upon a time, we made a product that was selling to Fortune 500s, and IBM loaned one of the smaller zSeries to us to make sure the product was running well in a Linux VM on it. The thing only fit in the elevator after we took it out of the wood crate...
A lot of the porting effort happened on Hercules, an emulator: http://www.hercules-390.org/
The reason you don’t is because it is basically impossible to get access to a mainframe you can play with and learn on. And IBM’s internal incentives and financial metrics ensure that can never happen.
* My views are my own and not those of my former employer.