A/C or heat pump efficiency is directly related to the temperature difference over which you're trying to pump. If the temperature difference is twice as large, you need to spend twice as much energy to pump the same amount of heat.
Furthermore, the days where you need a heat pump (or A/C) the most are the peak cold/heat days where the temperature difference between your house and the air is much larger than usual. Because of this, a ground source heat pump can do the same thing by spending much less power than an air source heat pump, so it is "greener" - though the installation is more cumbersome and expensive.
As for the "pump heat from here to there" ... while we are cooling we are warming the ground below. And as at our place, the ground's consistency keeps the warmth rather local, we reuse it later when we warm our water during the night.
We do live in a densely populated area and about every fifth hoiuse built here during the last 10 years uses a ground-based heat pump. As for electricity use: each house has two hours of "blockage" where the pump is not allowed to run, so that the power lines aren't stressed. That's no problem at all, because all houses are highly insulated and thus buffer rather good.
Good point. The source and destination of heat can be air, water, or "ground source". But it's still all just the same concept as A/C units. Pump heat to one place, using less energy than what it would take to heat it using resistance heating. There is nothing special about ground source heat pumps, except that they're a pain to install in densely populated areas.
All types of heat pumps, unfortunately, often leak. Depending on the type of refrigerant they use, this can be a serious climate concern as well.