News for hackers. Stuff that matters.
Re:Only Google is really allowed to crawl the web by magicalist on Friday March 26, 2021 @12:41PM (#26594158)
> You see this recently with Wikipedia. Google's widgets have been reducing traffic to Wikipedia pretty dramatically.

Wikipedia visitors, edits, and revenue are all increasing, and the rate that they're increasing is increasing, at least in the last few years. Is this a claim about the third derivative?

> Enough so that Wikipedia is now pushing back with a product that the Googles of the world will have to pay for.

The Wikimedia Enterprise thing seems like it has nothing to do with missing visitors and that companies ingesting raw Wikipedia edits are an opportunity for diversifying revenue by offering paid structured APIs and service contracts. Kind of the traditional RedHat approach to revenue in open source: https://meta.m.wikimedia.org/wiki/Wikimedia_Enterprise

Re:Only Google is really allowed to crawl the web by tyingq on Friday March 26, 2021 @12:52PM (#26594273)
See https://searchengineland.com/wikipedia-confirms-they-are-ste... from 2015. Google's widgets that present Wikipedia data do reduce visitors to Wikipedia.

Or see page views on English Wikipedia from 2016-current: https://stats.wikimedia.org/#/en.wikipedia.org/reading/total... Looks pretty flat, right? Does that seem normal?

As for Wikimedia Enterprise, you do have to read between the lines a bit. "The focus is on organizations that want to repurpose Wikimedia content in other contexts, providing data services at a large scale".

Re:Only Google is really allowed to crawl the web by SamBam on Friday March 26, 2021 @01:06PM (#26594450)
The first link doesn't seem quite conclusive (see the part at the bottom), and also doesn't give evidence that Google's widgets are to blame.

The flattening of users could also be due to a general internet-wide reduction in long-form (or even medium-form) non-fiction reading. How are page views for The New York Times?

Seems like it should be simple to A/B test, though. Obviously Google could do it themselves by randomly taking away the widget, but would could also see whether referrals from non-Google search engines (though they are themselves a tiny percentage) continue to increase while Google remains flat.

Re:Only Google is really allowed to crawl the web by tyingq on Friday March 26, 2021 @01:21PM (#26594623)
Edit: Removed bad "simple english graph", thanks. Though the regular english wikipedia traffic is flat from 2016-present.

As for NYT, is there a better proxy to compare to? There's no public pageview stats and they have a paywall.

Re:Only Google is really allowed to crawl the web by magicalist on Friday March 26, 2021 @01:31PM (#26594738)
That first graph is Simple English, not English, and is in millions, not billions. They also explicitly call out the methodology change in 2015...