Yes, the political feasibility is one big problem. (The other challenge is to figure out how much revenue you could raise---there are some good theoretical arguments that elimination a tax on something else like labour income should increase the local land value and thus land value tax by enough to offset that loss, but I don't know whether that works out in practice.)

There is some hope for the political feasibility: some countries already have land value taxes with various imperfections like Estonia, and some Australian states, and some counties in the US.