For simplicity, assume that the company buys back stock from all shareholders equally, and then the shareholders trade amongst themselves.

Dividends and buybacks are simply a way to get money from the company to the shareholder. The total number of outstanding shares is of no import---as you can see in stock splits.