This is a well-known canard. If someone can't pay their tax on a primary residence, you put a lien on the property that becomes enforceable when the ownership changes hands, whether by sale or inheritance. No one gets "thrown out" of their homes.
But that's a minor issue compared with the second part of my post, which is that if a proposed tax reform reduces the tax burden on almost every corporation and high net worth individual (bar a few land speculators and domestic oil well owners), ordinary people have to pay more to the government or get less from the government. LVT might be perfectly viable as a tax, but as a Single Tax replacing the existing set of taxes (which arent exactly optimal from the point of progressitivity to start off with) it's a wealth transfer on an unprecedented scale from people whose wealth is mainly the property they bought (including the land it sits on) to people whose wealth is mainly the shares they own. The latter category is much richer than the former.
Efficiency arguments that the average middle class homeowner would still be better off materially even with a much higher personal tax bill because the Zuckerbergs and Dorseys of this world would retain more untaxed profits and dividends to use their superior resource allocation skills to invest in land-free Bigger Adtech might appeal to the armchair neoclassical economist but for obvious reasons the public are more sceptical.
If we plug in something approaching real data, the UK government statistics agency assesses that the UK has £6tr in land value (more than the value of the assets that sit on it!) of which two thirds is owned by householders. And there's an annual government budget of 1tr so those land value taxes are going to be a really high proportion of the total value of the land (and property that sits on it). Good luck convincing the average homeowner who takes 30 years to pay for their home from their salary and typically retires not long afterwards that the tax incidence of a Single Tax will be favourable to them
But that's exactly the point - these people are glorified renters, they don't really structurally benefit when land values go up. The fact that a Georgist tax has to be introduced gradually in order not to be punitive towards those who happen to be holding land at any given time is well known. But we already have property taxes for a start, many countries have large one-time fees on real estate transactions etc. These are taxes that could quite easily be replaced by a shift to LVT.