Except they (the gender non-specific singular) is probably running kubernetes and has multiple clusters of 10 or so gpu hosts. Not that I disagree, but spinning that up locally and orchestrating it will take time and money. And explaining why training is paused because you keep blowing breakers in the office will cost political capital.