1 - http://www.pewsocialtrends.org/2015/05/07/family-size-among-...
And sure, technically you can fit five people in a sedan, but it ain't comfortable in the slightest, especially when you have carseats. An SUV or minivan is much more practical for this 40% chunk of US families.
And as mentioned elsewhere, it's not always just the mom/dad and kids. You might have the family dog, band instruments, friends, non-core-family relatives, a month's worth of groceries, etc.
Plus, even as one person, I've gotten a lot of good use out of my Highlander because I can fold down the seats and haul relatively-large things (like furniture) and keep the seats up and drive people around without having to own two separate vehicles to do it. As a bonus, AWD and all-terrain tires means that I can do these things during snowstorms and/or on unpaved mountain roads with relative safety and ease compared to your run-of-the-mill FWD (or - worse - RWD) sedan (the latter activity is niche, for sure, but the former is typically routine for vast majority of car-owning Americans living north of the Mason-Dixon Line).
And if you want to be able to transport anyone else (kids have a friend coming over, give someone a ride, etc, etc) you need a 7 passenger vehicle?
How is that "quite small"?
You measure average and no one is average.