The population being taxed via tariffs instead of income/capital gains is a net gain for wealthy people always, because they spend a much smaller fraction of their income, and a significant portion of the "tariff-base" is fixed-cost like.

Consider an iphone or a washing machine: Basically everyone is gonna pay those tariffs once, it does not matter if you work a construction job or own a whole city block.