We have a small team of 7, so after a lot of bureaucratic navigation I got us custom built desktops for local experimentation with 2080 Ti's and for the rest we use your typical cloud providers. Personally, I'm partial to GCP, because of the TPU support and BigQuery, but AWS is fine too.