Unfortunately the "universally accessible" part seems to be incompatible with current copyright law, so amazing resources like the Google Library Project are reduced to a glorified card catalog/search index rather than an actual online library.
Unlike Google, Sci-hub has so far managed to evade copyright law, so it can fulfill more of the "universal access" part.
The Internet Archive and Open Library ... to a much greater degree ... have not. And much of the material there is the result of Google scans.
I still prefer unencumbered sources (LibGen, ZLibrary, Sci-Hub), but at least having the option of access through the Archive is helpful.
Reading books on my own device with local optimsed software remains superior to Archive.org's Bookreader, though the latter is pretty darned good.
Not to diminish the disappointment over Google Books being curtailed (Google actually went to court for it!), or DMCA takedowns on Youtube or Google Search results. But we also have to admit that today Youtube is exercising editorial control to ban opinions they don't like, and that Google Cloud is as quick to drop "the wrong kinds of customers" as the payment processors everyone was complaining about ten years ago.
Maybe "getting ahead of the censors" minimises the damage, maybe it encourages the censors. I don't know. I guess the company is run by smart people who want to make as much money as possible in the long term, so it's likely good for consumers on net. That said, it's a shame we don't see them fighting for those principles today.
Are Google scans really in the Open Library? If so, that's very interesting.
The Open Library does seem to be operating under the shadow of copyright law, since it deployed DRM that attempts to eliminate some of the innate properties (unlimited readers, instant free copying and distribution, etc.) of ebooks, presumably to avoid copyright claims - and then got in trouble for reducing those restrictions briefly during a global pandemic.
They're qualitatively not all that different from SciHub.
News snippets seem analogous to thumbnails in image search.
I imagine a fair use defense might be affected by 1) how much of the content is used (e.g. small abstracts vs. full articles) 2) whether the use is transformative (e.g. for an internet search index vs. in a virtual newspaper), and 3) whether the use is commercial (e.g. whether it's by Google or archive.org, and whether there are advertisements shown next to the snippets.)
I agree that Google News snippet things are exactly what US "fair use" exemptions were written for, and are clearly a net public benefit. Some people don't; French law for instance definitively falls on the "deliberate infringement on a massive scale" side,
https://news.ycombinator.com/item?id=27818667 ("France fines Google €500M over copyright row")
Thesis: mainstream thinking overstates the moral difference between these two -- and understates the morality-distorting effect of money, lawyers, and lobbying.
Academics generally value their work being made accessible.
Paywalls that obstruct human progress are contemptible, and I suspect that most academics at least dislike them.
That's one of the few good things you can say about Google today.
To me, the whole point of contention here is just an artifact of a serious problem with the web as a whole, a problem to which Google contributed thanks to their role in Internet advertising. The problem is: "content creators" have too much control over the form of what they put out there. That Google is taking some of that control, by republishing the data in a different form, is just a different flavor of the issue; whether it's the "content creators" or Google that control the form, it's not the end users.
Right now, the fight is mostly between people peddling ad-infused, SEO-ridden garbage sites and Google rehosting the little real content their have in a half-assed, often wrong or improperly attributed, box on the results page. Both sides have strong, user-hostile reasons to be in this fight. Neither has "making knowledge accessible" in mind.
To truly make knowledge accessible, it needs to be published in a machine-readable format and accessible as if through a database. Just like when building a database-backed system, you don't put React components into your database rows, and you don't render your frontend from stored procedures, accessing the "information superhighway" shouldn't involve navigating countless of web pages that wrap the information you need in tons of garbage (nor it should involve hoping that the magic Google box will pick up the correct bit to show you). You should be able to make queries, get the data without all the formatting fluff, and browse it through whatever client software you want - and then issue further queries.
I mean, note how many successful startups out there exist only because publicly available information is not machine-readable, and all they do is scrapping, mixing, and republishing the data. E.g., in my country, there's a popular tool that tells you how to get from point A to point B using public transportation - but it only exists because the official time tables are published as web pages. It would not need to exist if you could just do an equivalent of arbitrary SQL query on the official data. But you can't, so you have to use that other tool whose vendor did the necessary scrapping work - and you can't get at the data itself in that tool either, because they have to lock it down to justify their existence.
Sorry for the rant. I could go on and on. I might write a long blog post about it eventually. But in this context the point is that both "content creators" and Google know they're not acting in the best interest of the people. They're just fighting over the money printing machine.
Why didn't it catch on?
There are I reckon two main causes: 1. No one cares. 2. Data has value only when kept behind a paywall. Data is not the new oil as oil is naturally scarce. Data is infinitely copyable.
1. Majority of people are just ignorant. They get their data magically and pay for it with watching ads. The information world starts and ends there for 99% of the populace.
2. The other 1% know very well the value of data, but only when it is kept hidden. If all data were openly available, just a federated SPARQL query away, there would be zero economic sense in founding almost any modern IT company as no profit can be made from data.
There is no way to make non-copyable data. Sure one can encrypt and provide decryption keys for access. And then the receiver has the data and open sources it in semantic web and the seller can't make any more profit from the data. Law and IP rights are the only deterrent here. They don't apply globally. Even locally one needs $$$ to pay lawyers and enforce the IP.
Issue number 1 is solvable with more education of the masses, so that they care. Masses later yield government officials and good stuff happens.
Issue number 2 is much more nuanced. Economic incentives must be placed very carefully so as not to lead to societal collapse. I am not sure what the best way to go is but for starting out we, the people, should pressure the government (see issue 1) so that they enforce that all publicly available data, like the timetables example or all academic work funded by taxpayers, is also available behind a SPARQL endpoint. 99% won't care, some will and have for example Ubuntu Touch timetable app without HTML scrapping.
For other data the question remains if open access is a good idea. I dislike all data mining by Google and Facebook but would the current Silicon Valley and USA wealth be possible if Google/Facebook open sourced their data, even if getting paid money to do so? Would a resell market form even tho it is freely available to anyone knowing SPARQL? What about private data protected by GDPR?
There is also the interesting discussion about technical feasibility of SPARQL endpoints everywhere. Anyone being able to run queries everywhere is a serious security nightmare and potentially waste of compute resources. Among other things.