It's not only VRAM. more cores, more bandwith, better power delivery, workstation features...

Which card do you mean that has more cores/computing performance than 4090 and costs 4-5k?

NVIDIA RTX 6000 Ada Generation

Although they retail for more than $4k.

The previous generation RTX A6000 is probably what they're referring to above.

It is $7.5k, and slower than 4090 compute-wise. Two 4090s (which are cheaper than A6000) are strictly superior to it.

Indeed two 4090s provide more CUDA/RT/Tensor cores, but the A6000 gives you 48GB of RAM which can end up making the card faster at certain tasks that would require swapping out data to system RAM on a 24GB card.

Two 4090s have 48G too. For most (if not all) LLM-related tasks, there is no difference how many cards are used for the same amount of VRAM.