>There's what, like, 10 places in the world where you'll have enough compute to be able to train cutting-edge neural networks

You can just use a smaller amount of compute with transfer learning style stuff.

I can probably fine-tune a transformer model with my pocket money and beat any NLP solution from two years ago.