If your ground heat system also provides cooling that's no longer a problem. I have that for my house. It's pretty great. You're basically time-shifting temperatures across the year. Cool temperatures get shifter to summer and warm to winter. The hole itself won't freeze, because you're not significantly pulling or pushing energy down it.