Except the point of these larger transformer models is they generalize well over a wide range of domains or only require a small amount of transfer learning for really specific domains.

I'd say they're perfect candidates for the API as a service model.