People absolutely expect to pay for fast shipping; people just expect baseline slow shipping to be free.
> What if they're in another country?
Depends on the industry you're in, but it seems a lot less reasonable to expect free international shipping, unless you're buying something that costs high-three-figures or four-figures.
> If they only order one small item, the shipping cost is a very large percentage of the total cost to the seller.
USPS and hybrid shipping options can be relatively cheap, for domestic shipping.
> you've now priced all your items with the assumption that they're only buying that one item
You could always offer multiple-item discounts at that point.
Actually, they really aren't, not for private shippers. If your item is tiny, you can ship it for around $3 by 1st class; if it's over 1 pound, it's not going to be cheap. For something that only costs $5-10, that's very significant.
>You could always offer multiple-item discounts at that point.
People look at initial price first, and won't see your discount until later. The competitor with a cheaper per-item price will get the sale.