The 24 cores/48 threads EPYCs that come with 128mb L3 cache have a TDP that's between 155W and 180W. If you want to go with a lower core count (because you have a more modern architecture), you could even go with the 16c/32t EPYC 7282 (64MB L3) that sits at 120W. All in all, you're looking at almost half the power draw (AMD measures it differently than Intel, so aboutish the same) for the same performance.

I specifically mention the L3 cache size because, while not being substantial, a large cache can get you 10 to 20% improved performance due less CPU stalling from cache misses. For comparison, the Xeon in question has 25MB L3, so we'd be looking at 50MB cache, split in two dies (so it doesn't quite work as a whole block of 50mb cache).

greatjack613 also mentioned the fact that an AMD Zen 3900x matches the multicore performance (and has better single core perf) for a fraction of the TDP, see https://news.ycombinator.com/item?id=20687932.

All in all that was the general idea: yes, you can get a used server for a good price, but we shouldn't forget the efficiency aspect of it, compared to new hardware.