The effect GP is pointing out is how denoisers damage detail, which is true. This detail can persist over multiple frames, which is why many denoisers include a temporal comparison component to mitigate the damage, but you still lose detail.
That's exactly the point of GP though. Even though each individual frame might be almost indistinguishable from random noise you can still extract patterns over time. This is also the case if you don't average the frames in software but let the viewer's brain do it. If you just remove all "noise" from each frame and then add random noise back those patterns will be lost.
In practice you won't have static frames but also movement so recovering the signal from the noise becomes a lot more complicated.
However, the issue is that lossy compression removes various types of minute detail, smoothing surfaces to reduce the amount of data that has to be stored, be it noise "grain" or skin pores, according to compression settings. Storing the original noise as it was would basically make any compression impossible.
If you have a few static frames and average them, you improve SNR by retaining the unchanged signal and having the purely random noise cancel itself out. Retaining noise itself is not useful.
I suspect the effect you might be seeing is either just an aesthetic preference for the original grain behavior, or that you are comparing low bandwidth content with heavy compression artifacts like smoothing/low pass filtering (not storing fine detail saves significant bandwidth) to high bandwidth versions that maintain full detail, entirely unrelated to the grain overlaid on top.