If the land of Joe's neighbours has no restrictions at all, it will be valuable. But if Joe's land has a weird zoning that says 'can only be used by Joe' that land would be nearly worthless on the market, but good for Joe.

Less extreme, if you have a neighbourhood with plots that allow unlimited building and a few plots that only allow single family homes, the latter would be a lot cheaper.

Exactly what I am saying.

And what is your take on when Minneapolis voted to allow multi-family units[0] and rents dropped[1] relative to other cities? Shouldn't those new units have brought higher land values, and with it higher rents?

[0]: https://archive.ph/7oCO6

[1]: https://archive.ph/lO8xY

rents =/= land value.

you can put a duplex or quad-plex on a parcel and have much lower rent, but higher parcel land value.

If you are a homeowner who wants lower land value tax, you should fight tooth and nail to prevent higher density zoning,

You would want higher density zoning in all the other neighbourhoods across town, I guess?