That aside, samepage sharing has been a thing in virtualization for more than a decade. If you have 800 VMs running RHEL8 for POWER, a significant amount of the memory load is kept in identical pages, which lowers the burden of oversubscription. The recommended workload is unlikely to be 1000 unicorn/pet VMs, and more likely to be a large number of VMs running the same base OS, and some of the same workloads.
Synchronous interrupt locking is also a solved problem.
0: https://www.kernel.org/doc/html/latest/admin-guide/mm/ksm.ht...
The CPU cycles are a non-issue for the configuration on this mainframe, and it's fair to assume that, since IBM developed the silicon and IBM engineers wrote the hypervisor layer, that most of the detrimental effects of "we need this to work across multiple CPU vendors and multiple generations of the architecture" are covered.
Even x86 CPUs get improvements to cache coherency in memory dedupe scenarios with generational improvement, and intelligent NUMA topology layout helps a lot. It's also worth noting that it's a mainframe, so essentially every configuration has a large number of other processors with a couple of instructions disabled so they don't "count" as CPUs for licensing, but they still perform work. I would not be surprised in the least if there were 1+ co-processors dedicated to offloading operations exactly like "LPAR || z/VM memory dedupe"
http://www.redbooks.ibm.com/redpapers/pdfs/redp4827.pdf
[old]
It is called active memory sharing in the POWER world
https://www.ibm.com/docs/en/power9/9080-M9S?topic=sharing-ma...
Slide 17:
http://www.vm.ibm.com/library/presentations/syslimit.pdf
>Synchronous interrupt locking is also a solved problem.
No, it really isn't. I have not seen a production implementation of virtualization with > 4:1 CPU oversubscription that doesn't eventually or immediately have significant performance issues with database workloads.
This discussion is about the configuration of the systems, not this specific scenario. While this may have been the scenario outlined for you, it wasn't apparent from your comment.
Production workloads on busy servers have very different requirements than consolidation, VDI, resiliency/redundancy, hardware abstraction, etc. Every workload needs a different evaluation.
All of these VMs running production Ora/OAM/OEM/Websphere workloads? No, don't oversubscribe. Some of them running similar workloads? More oversubscribe is ok. Few of them? Lots of oversubscribe.
Similar for interrupt locking. The "old" synchronous interrupt lock I was speaking about was "I have a bunch of VMs with CPU oversubscribe, and even if they're doing nothing, 30% or more of your CPU time goes to interrupt scheduling so every vCPU for a given VM can schedule simultaneously". This is solved.
"I'm running a CPU-intensive workload on a massively oversubscribed server" is not, and we really shouldn't expect it to be.
I am having a generalized discussion about virtualization oversubscribe. You are having a specific discussion about CPU-heavy DB workloads. Apples cannot be compared to oranges.
>All of these VMs running production Ora/OAM/OEM/Websphere workloads? No, don't oversubscribe. Some of them running similar workloads? More oversubscribe is ok. Few of them? Lots of oversubscribe.
All of the VMs are running Oracle/Websphere/Apache, per the link. That is what the article is recommending, which is where my skepticism came and is coming from.
>I am having a generalized discussion about virtualization oversubscribe. You are having a specific discussion about CPU-heavy DB workloads. Apples cannot be compared to oranges.
Everyone in this thread is talking about the article, I guess everyone but you.
That aside, comments on articles are sometimes more general. "How do mainframes compare in 2021?" is a conversation which is not intrinsically linked to "should you oversubscribe VMs 66:1 for this specific workload?", and I interpreted your comment at the former. I wasn't talking about the article at all.
There is a giant table in the middle of the article that is impossible to miss if you read it. It literally says # of VMs. As well as calling out VMware, and z/VM as tye hypervisors in use. The oversubscription is a simple math problem, and that's assuming best case scenario.
We just paid just under $50k per server for on-prem physical servers that have 144 cores per server and 1.5 TB of RAM each.
For 12 of them it would be less than $600k and you'd get a total of 1,728 cores and 18 TB of RAM to compare against the mainframe's 30 cores and 2 TB of RAM for over 4x the cost!
Apple’s Macintosh made waves with its cheeky hello world demo: “Never trust a computer you can’t lift” (…and defenestrate).
Similarly, with an IBM big iron box, if something goes wrong, I sure hope the 6-figure support contract gets it fixed - probably within a few hours if you’re in Manhattan or the SF Bay, but what about elsewhere? Whereas with commodity HP/Dell/Supermicro pizzaboxes in a HA configuration I don’t need to worry about needing a same-day fix for hardware issues - I could put things off for months, even - and probably do it myself and spend the support contract money on a Plaid Model S and have money left-over for a Mac Pro with XDR display (with the cool stand!).
I know IBM’s srsbsns machines have their own redundancies - including being able to hot-swap literally everything, it still feels like open alternatives are overall better in the long-run.
Yes, the kernels need to be "snowflakes", but that's not necessarily an issue if this is functionality you really need.
While an LPAR could technically be HVM, the nature of the changes required to an OS make it PV-like.
Most applications don't need gigabytes of memory though? I don't know what one uses a mainframe for but web servers, file shares, remote login software, proxies, traffic scrubbers... lots of common applications use up to a few dozen megabytes of RAM; throughput is much more important there and memory to operate on is just for some state and buffers. Even a database with millions of rows and a dozen indexed columns, the index of that (I just checked a database of my own) is in the hundreds of megabytes, far from even one gigabyte.
It seems a bit odd to assume each server always needs multiple gigabytes of RAM.
They’ve stopped being an innovation business.
I also struggle to believe that oversubscribing the z CPUs 66:1 isn't going to result in performance issues. The documents on limits aren't clear but I foresee that mainframe spending a LOT of time on locks even with all the work IBM has done to help manage them.
This whole thing just really smells like a bad advertisement for IBM, which aligns with them announcing their latest gen z processor last week.