Sure. But historic districts are not the majority of zoned land in any city. I mean, the imperial palace gardens remain in the middle of super-dense super-expensive central Tokyo, as do many other historic landmarks and districts in dense cities (and non-historical but significant landmarks too: nobody is removing central park from NYC either).

Actually, an even stronger argument is that, well, cities need more than housing. You need city planning and zoning because huge buildings need commensurate transportation, sanitation and water/power supply infrastructure. But neither this, nor preserving hundred years old historically important districts, are what people complain when they complain about zoning. You can plan for more public infrastructure at the same time you allow lots of new construction. Hell, you can tax the developers of the new buildings to front most of the public infrastructure upgrade costs, then distribute the maintenance over the now expanded population. People complain when a city fails to do this for years even in the face of strong demand pressure, because the lower the supply the more the real state investments of existing owners appreciate in the short term.