Even in the days of MS-DOS games, I could just buy a CD for $2 from a contact (he'd ride his bike to me and deliver them), it would be full of games that you could just extract and play, no installer involved.

It became slightly harder for a period of time when CD sized games became common, and Internet speeds have not yet caught up.