For completeness, it should also be pointed out that, in the future, the supply of usable (for growing food) dirt maybe be outstripped by demand, making it become more valuable than gold.
Actually, if the price started trending that way, I imagine other factors would have a bigger effect on the economy before it reached that stage.
If you don't have enough dirt, then you can't grow food and everyone dies.
But why is dirt not valuable? Well, that's because the supply of dirt vastly outstrips demand.
The same could happen for gold.