it's not so much the cost of providing services so much as what is driving the property value. in highly desirable locales, the value of a property is affected much more by what people are doing around you than any improvements you could make to a private dwelling.

or alternatively, you could consider it a tax on the opportunity cost of you having exclusive use of a piece of land. your neighbor with the bigger lot is not consuming (much) more services, but the opportunity cost of not building more stuff on that lot is proportionately higher.