SF doesn't sell that well either. But I think it's not generally helpful to compare the genres as a whole with best sellers. The top literary fiction - and certainly winners of prestigious prizes - does sell well - and I'd assume that the very top significantly outsells SF for the same reason the very top non-fiction sells crazy amounts: Most people don't read much, and what little they read (or at least buy) is seriously concentrated things the mainstream deems culturally significant.

But I get the impression that the genres that sell well overall does so in large part because of binge readers that reads broadly. A client of mine runs a small press - I help them with tech stuff; small enough that they can "see" in the stats how often people buy an entire 20+ volume set of everything a given author has published at the same time. That doesn't happen in literary fiction, not least because few writers in literary fiction write enough significant books for it to happen.