We did a similar analysis for GCP. Preemptibles/spot were the way to go with inference. CPU performance was also faster for our scaled inference workloads.

Times change though, we’re about to conduct the same analysis over again, with latest models better architected for accelerators.