At which point people would buy less potatoes, or buy more locally produced goods.

Which is the entire point. To incentivize people to change their behavior.