Because high end research uses a fleet of them, not just one.

4090 (or 3090, 1080Ti and so on) is a high-end consumer GPU, but at the same time it is an entry level GPU for AI researchers. Don't forget that workstation cards (RTX 8000) let alone server-grade GPUs such as A100 are an order of magnitude more expensive.

I was doing ML stuff on a GTX 1060 a few years ago. As with everything, it depends on what you’re doing.

Chances of you publishing something in ML improve proportionally to the amount of hardware you have access to. Or to put it another way, the less hardware you have, the smarter you have to be to publish something in ML.

That does not sound like "entry level" research to me.

IDK, I want to train up some AI and have the choice of using google colab or buying a GPU that can do it within a reasonable timeframe.

Not going to be spending $10k like the tortoise-tts guy (was looking into that project last night) but $2k might be doable for a hobby project. Plus I’d have a computer at the end.

Because of VRAM. Even a simple model for language can easily max out a 4090.

Also, ROC-M is a bit of a mess to setup. With Nvidia i just need to install cuda, cudnn and then pip install tensorflow/pytorch.