If you don't pay your taxes you go to prison, they don't take your house away.
Bad comparison. That both failing to pay off a loan or pay property taxes can lead to foreclosure is a superficial similarity. Property tax isn't paying off a debt you owe a bank for a loan used to buy the house. It's merely a tax collected to cover municipal expenses that's calculated based on the value of your house.
Now as to whether the bank owns the house you bought using a loan, morally there is a certain practical truthfulness to that statement, perhaps, and for practical purposes is may be somewhat pedantic making distinction that determine "true" ownership. But strictly speaking, as with borrowing in general, if a loan is money you've borrowed that you then use to buy an asset like a house, then what you owe the lender isn't the house, but the value of the loan. If I lend you $20, and you buy a book with that money, you don't owe me the book, you owe me $20.
That the state putatively spends that money on common goods has no bearing on the landlord relationship.
I suppose there are rare cases of allodial title. Nevada used to let you buy an allodial title that couldn't be inherited. It was useful for doing things like getting a second mortgage from a bank that wasn't aware of the law and then immediately defaulting and telling them to get bent because they can't legally foreclose on an allod. Needless to say lobbyists successfully put a stop to that.