If someone is building this device to protect one's private keys from the three-letter agencies, one may want to completely shield the Geiger counter and its radiation source with thick lead. Otherwise, the NSA may decide to remotely influence the behavior of the random number generator by beaming lethal level of gamma radiation toward the generator, and they would not care whether you are killed in this process. /joke

Most modern systems contain hardware RNGs that get noise from thermal effects, ultimately quantum too.

Thermal effects do not rely on quantum behaviour. They arise from an extremely large but classically deterministic state space.

Does this mean determinism is moot?

In my mind, Schrödinger's cat was an important thought experiment that pointed strongly in this direction. Whether it is just a limitation stemming from our current knowledge of physics or something else is still debatable.

In quantum mechanics, yes it is. See Bell's theorem.

And folks like Einstein felt quantum mechanics is an incomplete theory because of this. It is not an settled debate.