That's assuming it's sufficiently parallizable. If you only have a single thread, an ordinary Intel desktop CPU is still faster than any Xeon: https://www.cpubenchmark.net/singleThread.html

I thought about 4-core vs newer 8-core desktop CPUs when upgrading, but decided that I more often do work on less than 4 cores (with 1 you get the "Turbo" capacity). The only multi-threaded program is PyCharm which can use up all those cores when recalculating its static type checking of Python code (which it seems to do quite excessively).