One thing I really like about the van (Honda Odyssey) is that you step slightly up to enter and step down to exit. Very comfortable. And this is on a car that's so low to the ground (good for little kids and old dogs) that it's like a snow plow if there's more than a few inches of snow.
I want something lifted more than a sedan so I step up to enter and also has more ground clearance for snow, potholes, rough roads, etc. But that doesn't mean 44" tires with a lift kit.
Now, it's true that someone dropping all of the extra cash for one is less price sensitive but we're still talking a pretty big difference going from mileage in the teens to the 40+mpg a decent car gets. High gas prices would get at least some people to reconsider, especially since you have to pay a fair premium as well.
None of those physics issues are specific to SUVs. Even full size SUVs end up on shared platforms with full size sedans, with "reasonably close" weights (essentially take a well optioned sedan and it weighs around what the base SUV version does)
With the focus on all of these automakers on modular platforms, we've reached the point where even the largest SUVs could meet the exact same standards their sedan counterparts do and not really change at all.
99% of the "SUVs" you see today are CUVs, which in turn are barely more than a lifted sedan.
So the difference is gas milage is miniscule, to the point gas prices will no longer affect the sales of the most successful SUVs