> This thinking is slowly beginning to change

it has been "beginning to change" for 20 years now. The fundamentals of how prestige/significance is allocated have not changed however, apart from the fact that there are a few reputable open access publishers like elife.

> We want our work to be disseminated as widely as possible!

Of course! So few people in the world read these papers already, it is absurd to deny access to interested minds. Publicly funded research needs to come with open access strings.

Perhaps a blunt question, but what took scientists so long to figure this out?

Its only been a viable option to self publish for about 30 years, and more like 10 if you didn't want to invest large amounts of personal resources.

Open access review and channelization is still an open problem. The for profit journals actually do provide a service of worth and note.

There's also sort of an issue from a bunch of open access publishers acting like con-artists - charging researchers made up fees to publish in their made up journals. A certain librarian made a rather useful list of "predatory" open access publishers: https://beallslist.weebly.com/

Well for one, open access requires the Internet. The closest thing to "open access" prior to the Internet was the willingness of universities to allow unaffiliated members of the public to walk into libraries and read the journals stored there. It took the Internet for there to even be something to figure out.

Beyond that, it is just the general time needed for society to adjust to a new way of thinking; people, scientists included, do not start questioning the status quo overnight.

> Perhaps a blunt question, but what took scientists so long to figure this out?

The other science disciplines (mostly the life sciences), saw the AI/ML field greatly advance with their open research (e.g. arxiv, Github) and now want to copy it.

And yes, it was specifically AI/ML. Look at IEEE -- they're holding back the entire EE field. The ACM holds back the other areas of CS.

CS may be the faster growing field in the arXiv over the last decade but it was relatively late to the party. https://arxiv.org/help/stats/2017_by_area/index

From an outside perspective it appeared to me that the leaders in this were actually the physics and mathematics community, rather than ML.

Physics and other disciplines were broadly using arxiv before ML did it. When I switched from physics to ML (CS) I was surprised that arxiv isn't used more.

The publication infrastructure should be implemented and funded by the governments, preferably, as part of some international agreement to share the costs. The only valuable service that traditional publishers do provide is acting as a trusted authority that verifies publications. The rest (distribution etc) is already provided almost at no cost by the internet. Apparently, when there's such an oligopoly of few for-profit organizations, there's no good for anyone except them. Since a big part of research is paid by the taxpayers' money, they have the right to replace the dysfunctional market with a niche communism and run their own government publication agency.

Well, work cannot be cited, if it cannot be accessed. I would be surprised if any researcher would not support open access.

What's stops people from publishing in both?

You're only allowed to publish in one. When you submit an article, you're normally specifically asked to confirm that the work hasn't been accepted / isn't under review elsewhere.

That seems like it would necessarily lower the quality of the journal, not to mention your own paper.

Sounds a lot like traditional fiction publishing. They want exclusivity, sometimes indefinitely.