I am assuming that is how the result was verified much faster - >1,000x the computer power used. Without looking deeper (like actually reading the article!) I couldn't say way that would be in this particular case, but such a disparity between power used to make a finding and power used to verify it by repetition is often the case when the initial result is discovered by a recreational researcher or small group, not a better funded group.
Not unlike the human scaling seen when one individual or a small group publishes a proof that could be significant, and many others poor over it to see if they can find a flaw before it is generally accepted as a genuine new finding (an order of magnitude or more of human brains to check, than to initially find).
Imagine factoring a non-prime number. It would take you quite long to factor 695587 with nothing but a cheap calculator, but you could very quickly verify my factorization if I give you the information that it is divisible by 71 and 101.
I'll have to read up on the certification/verification process to comment any further though.
See https://en.wikipedia.org/wiki/Certificate_%28complexity%29 if you're interested in learning more.
You could probably design a supercomputer specifically for prime searches, but it would be a weird one.
GPU __shared__ memory is 64kB. A 1024-wide workgroup would be able to store 4096 bytes per register, if the whole workgroup was just working on one number. So ~6 registers x 1024 shader workgroup to store the full 20kB number.
__shared__ memory would allow for very quick shader-to-shader communications.
I think... it can work. But its difficult for me to visualize the full program. Its going to be complex, but probably doable. Its also not really a design pattern that a lot of GPU code (ex: Thrust) works in very well, you'd have to use harder to use libraries like CUB instead.
That's why GIMPS is the way it is: with volunteers offering computers that are separated and do a siloed calculation.
But the fact that this was done on a Threadripper 3990x, a high-core count (but slow interconnect / core-to-core communication) kind of computer, makes me believe that there's a degree of parallelism in this problem.
Otherwise, another computer (ie: higher GHz / less core count) could have been used instead.
---------
I really don't know this specific algorithm: ECPP. It comes down to how that works... especially its parallelization.
I'm guessing there just isn't a good CUDA implementation out there.
EDIT: reading a wiki page, this sounds like recreational mathematics, not actual research math, which is a shame really