Agricultural-zoned land would be cheaper.

I would rip up urban zoning with an LVT to ensure density, but keep the agricultural zoning because we can't eat skyscrapers.

There is no actual tradeoff between availability of food and availability of skyscrapers. Even if we actively tried, we couldn’t run out of agricultural land in the US by building on it.

The only reasonable way to account for zoning would be to require the zoning board to actually buy the property and pay the associated taxes. Then they can lease out their own property with whatever restrictions they want. Without the benefit of subsidies or eminent domain, naturally—if they want to expand their reach they'll need to raise that money through the revenues from leases (less income tax and other overhead), with restrictive zoning rules limiting their revenues.