You want geology that is expected to remain stable for eons where the material will be secure from both intentional and unintentional excavation, for eons. And, more immediately, concerns over ground water contamination. Even cities like LA get some of their water supply from wells. And, more rural areas are often served by 100% ground water.

Finding sites to safely and permanently dispose of nuclear waste has been hard. Both because a lot of areas are poorly suited, and also nobody wants a permanent nuclear waste dump in their back yard.