why wouldn't they build a relatively small cluster for training tasks using Nvidia hardware? It's simply the industry standard, every researcher is familiar with it and writing a custom back-end for pytorch that scales to hundreds of nodes is no small task.

I doubt Apple cares about spending a few hundred million dollars on A100s as long as they make sure the resulting models run on billions of apple silicone chips.