You can fit 4 people in a self driving car pretty easily. Most people currently drive by themselves because it's too difficult to arrange on-demand carpools on your way to/from work.