The manufacturer can set the invoice price at whatever they want already. Dealerships are only negotiating their piece of the pie, they aren't actually able to alter the core price of the vehicle. The market/competition determines what a manufacturer charges, the dealerships just obfuscate the exact figure.

So I agree with the other commentor, they exist to extract local money, to make a few locals rich, the manufacturer's piece of the pie is whatever they think they can charge either way. If you cut out dealerships you make prices less obscure and force manufacturers to compete more closely with one another, and also save the middleman rent-seeking.

Sales margins are low. Service is the big money maker, and if you buy a new car you don’t want to void the warranty.

Making money on the back through financing or upsells is normal.

Sorta. Most of the big names already make more with financing than they do from the car.

Cars are odd. Shipping is not cheap on them. And will never be cheap. If there is not some contractual obligation to ship inventory closer to consumers, I see little reason to think prices won't go up in a market that is dominated by direct sales.

Well, we've recently seen dealers marking up cars insane prices, so maybe direct sales at MSRP do help keep money local.