They spent $500k doing it (in research credits for compute). Not sure if this is the best way to spend so much money, considering they made no novel contribution to NLP field, and OpenAI would have most likely released it soon anyway.