Honestly in my building near harvard square this hasn't seemed to prevent those from moving out from renting (rents have gone a little nuts here too). Its a decent idea, as it incentives people to sell when they move. We have had some people cashing out, because values are high.
"The City of Cambridge offers a residential exemption to property that serves as a property owner’s principal residence. To be eligible for FY17, the owner must own and occupy their property as their principal residence on January 1, 2016 shown on 2015 Massachusetts Income Tax."
If you stick a massive tax on that, that deters that kind of behavior, and forces them to either rent out the property, or sell to someone who will either live in it or rent it out. This will increase the housing supply, which would then decrease the cost of housing overall by providing more supply and more competition. This wouldn't be a tax that people would actually pay, unless they're stupid; they would take steps to avoid paying the tax altogether.
Taxes don't make stuff more expensive: they deter behavior. The simple rule is: if you want less of something, tax it. Now, this isn't always true, but to an extent it usually is, especially for consumption. (It's not really true of income taxes usually; everyone usually wants more income, and needs some amount of income, so a tax on that income doesn't discourage that, unless the tax is so high or so wacky that it discourages working for additional income.) Here, we want people to stop leaving properties unoccupied, so we slap a giant tax on that so they'll stop doing it.
Raising the cost to landlords in an sellers market like SF is just going to drive up rents - and Landlords will certainly be able to pass through their higher costs even to rent controlled tenants.
You don't solve a supply problem by adding taxes. If anything, that will reduce the supply even more.