They do it because it works, and the reason that it works is that passengers (and flight-booking web sites) tend to sort the list of available flights by base price. Only.
It would be nice if the travel sites let you enter preferences: "How important is a decent meal to you?" "How important is a free checked bag to you?" and so on, but I haven't seen any sites like that. There's an opportunity for someone. :-)
It's only when I go directly to carriers like Air France or Lufthansa that they specifically flag - for example - flights on an A380, which I am absolutely willing to pay a bit more for.
For example, I just booked ORD-SFO on UA in business, which is a completely different experience on the upper deck of a 747 (UA1213) than the usual 737 :)
Charging for checked bags didn't start until 2008, the better part of a decade after travel sites began. There is zero correlation between charging for checked bags and travel websites.
Prior to this awful and cheap practice the assumption was that you bought a ticket that covered the cost of getting you and a bag to a destination.
Advertise a low base fare and add extra fees/cut add-on services to make up the difference.
The checked bag fee is just one of the more recent ploys in a process that's been going on for decades.
It makes sense to the airlines. For them, it probably doesn't matter that much if you leave a bit late. The cost of the flight doesn't change with how much time it takes to board, so they're making more money by charging for bags (even if not that many people check bags). There might be 2nd order effects, like shifts being affected because flights take a bit longer, but I can't imagine that overwhelms the additional income from charging for bags.
The cost of a plane is v. expensive. Sitting on the tarmac for 1/3rd longer is VERY expensive, just in deprecation/interest/lease payments.
However, it probably makes sense because I imagine some/all airports charge luggage handling fees per item/kg, that are probably quite expensive.
This has not been my experience. Flying AA on the east coast regularly, the folks at the door generally seem to be super on top of eyeballing bags and giving out the red checked-at-gate tags liberally.
It never ceases to amaze me that the airlines refuse to abandon this absurd policy of charging for a piece of checked luggage. When you look at when this practice was introduced of charging for checked luggage, the time it takes to board and un-board a plane has increased by about 20 minutes due to everyone trying to shove as much as possible into overhead bins. Perhaps this time axis doens't carry much weight for carrier? Although that would be hard to imagine. It seems like false economy to say charging $25 per luggage checked make sense when you have now added 40 minutes to each flight. I can count the number of times on one hand that my flight has left at its scheduled time even though it boarded on time.