Most owner-occupants own SFZ lots. Every owner of a multi-family dwelling is a landlord.
I'd prefer that an owner-occupant be able to write off the interest on N/N units on a parcel, just the same as a landlord is able to write off the interest on N/N units.
That's true for N=1 (SFR), N=2 (duplex, half-occupied by the owner), N=3 (triple-decker with owner living in one), or N=50.
How zoning should work is an orthogonal question in my view.