If you’re doing transaction processing on the mainframe I could imagine wanting to score something (eg fraud) mid transaction, which might call for some additional platform capabilities. But it’s also plausible someone in marketing demanded more AI.

I can see that, but mainframes are perfectly capable of calling out to external services, and that's not new. I guess I really don't know what they are saying in regards to AI and mainframes.

I do not know if IBM will do this with the AI work.

but in many other similar circumstances they give the option of buying Application Assist Processors. That takes the work away from the main processor.

It is possible that IBM will offer some custom zAAP optimized for running the AI workloads.

IBM also open up running AI in their cloud.

> but in many other similar circumstances they give the option of buying Application Assist Processors. That takes the work away from the main processor.

zAAPs, zIIPs and IFLs are exactly the same hardware as main processors. The difference between them is purely at the licensing level. The firmware sets a flag saying "don't run classic z/OS workloads". z/OS sees that flag set and then will only schedule "new" workloads (such as Java and XML) to that processor. Actually there is an undocumented API you can use to schedule any (enclave SRB mode) workload to the zAAP – IBM keeps it under NDA, and ISVs have to agree to obey IBM's rules about what workloads are allowed to run as part of that NDA. (One ISV, Neon, started selling a tool to run an arbitrary workload on the zAAP, called zPrime, IBM immediately sued them, and Neon settled the lawsuit in 2011 and agreed to drop the tool). Other operating systems, such as z/Linux, ignore that flag completely and to them a zAAP is just the same as a normal main processor (CP).

> It is possible that IBM will offer some custom zAAP optimized for running the AI workloads.

Yes but that wouldn't be a zAAP. They do have hardware accelerators for various functions (crypto, compression, sorting) but those aren't zAAPs (or zIIPs or IFLs). Crypto acceleration is CPACF (extra CPU instructions on main CPUs) and Crypto Express (external accelerator card). Compression hardware acceleration is zEDC. Sorting hardware acceleration is the Integrated Accelerator for Z Sort.

AI inference on existing model instances need not be very compute-intensive at all. The "training" (i.e. model fitting) phase is what requires a lot of compute resources.

They are talking about customers spinning up Linux instances on their existing (mostly not Linux) mainframes, and special software to allow SIMD calls from Tensorflow to work, etc. I think that making a remote call out to a commodity box is probably easier in most cases.