University administrators, grant-writing bodies, and others pressure professors and graduate students to publish with specific publishers. It is not because those publishers are more trustworthy; it is simply inertia, institutional tradition, and credentialism. It is the same attitude that leads some companies to turn away candidates who never completed a bachelor's degree. To give an example of just how bad this situation is, we sometimes hear complaints about people citing the IACR eprint version of a paper rather than the "officially published" version -- not because they are any different, nor because there is something better about the official version (in fact the eprint version typically includes details that are absent from the official copy), but simply because European universities use citation counts to judge professors and only consider citations of articles published by specific publishing companies.
Academic publishers have raised their fees even as their costs have greatly declined. The number of journals that are actually printed and distributed on paper has been shrinking, and the cost of distributing over the Internet is almost a rounding error. Yet in that same period of time the publishers have increased subscription fees to the point where some university systems could not justify paying for the subscription. These companies have outlived their usefulness and they know it -- now they are trying to extract as much money as they can before the business model completely fails.
>But they don't, and I think it's because they realize that . . .
They don't realize any such thing: they're just doing what benefits them personally (e.g., surviving as a professor or apprentice professor) at the expense of the broader ecosystem.
It is a problem of coordination: if the professors, students and the governments and foundations that fund research acted in unison (and were sufficiently informed and clear-thinking), then the entire academic literature would probably be available without restriction (like, e.g., Wikipedia is) on the internet by now.
Such an arrangement would satisfy the values of the professor, students, etc, better than the current system does. But it is hard to get there because it is hard for 10s of 1000s to act in unison.
Sure, it's not the publishers themselves that are de facto forcing researchers to publish with them, but they do make use of the fact that they're being put in a position of being a rent-seeker, and they lobby a lot to keep it that way.
Of course, the question is not "who's to blame?", but "how to change this"? And indeed, the publishers aren't too relevant to that question - it's the incentive structures that should change.
(Disclosure: I contribute to https://plaudit.pub, which aims to change these incentive structures.)
Some young researchers often claim that they feel pressured by the system to choose the so-called prestige journals. This pressure is coming from older researchers who are making a decision from their experience. They don't need to reward journal writers but they do. Indeed, the open access professors could announce that they will penalize the tenure reviews for those who publish in copyrighted journals. But they don't, and I think it's because they realize that, for all of the costs, the copyrighted journals do a good job curating the information.