News for hackers. Stuff that matters.
Re:Only Google is really allowed to crawl the web by onlyrealcuzzo on Friday March 26, 2021 @12:28PM (#26593993)
Wikipedia isn't monetized. Doesn't it benefit them if Google is serving their content for free and people are finding the information they want without having to hit Wikipedia??

And also, isn't Google the largest sponsor for Wikipedia already? In 2019 - Google donated $2M [1]. In 2010, Google also donated $2m [2].

[1] https://techcrunch.com/2019/01/22/google-org-donates-2-milli...

[2] https://en.wikipedia.org/wiki/Wikimedia_Foundation

Re:Only Google is really allowed to crawl the web by kelnos on Friday March 26, 2021 @03:12PM (#26595771)
> Wikipedia isn't monetized.

No, but they often ask for donations when you visit the site, which people won't see if they just see the in-line blurb from Wikipedia on the Google results page.

> In 2019 - Google donated $2M [1]. In 2010, Google also donated $2m [2].

$2M is a pittance compared to what I expect Google believes is the value of their Wikipedia blurbs. If Wikipedia could charge for use of this data (which another commenter claims they are working on doing), they could easily make orders of magnitude more money from Google.

Of course, my expectation is that Google would rather drop the Wikipedia blurbs entirely, or source the data elsewhere, than pay significantly more.

Re:Only Google is really allowed to crawl the web by tylerhou on Friday March 26, 2021 @03:36PM (#26596040)
Unlikely that Wikipedia will be able to charge for content, seeing as all of their content is CC-BY-SA licensed. https://en.wikipedia.org/wiki/Wikipedia:Licensing_update

They may be able to charge for bandwidth (if you want to use a Wikipedia image, you can use Wikipedia's enterprise CDN instead of their own), but their licensing allows me to rehost content as long as I follow the attribution & sublicensing terms.

Google has no problem operating their own CDNs, so I find it unlikely that Wikipedia will be able to monetize Google search results in such a manner as you described.

Disclaimer: I work for Google; opinions are my own.

Re:Only Google is really allowed to crawl the web by aabhay on Friday March 26, 2021 @08:20PM (#26598559)
Fine, the content is free. But if your crawlers want access to the content, then pay! Simple as that.
Re:Only Google is really allowed to crawl the web by rezonant on Friday March 26, 2021 @10:24PM (#26599304)
Will it be a flat fee, so that I, a lowly one-man crawler developer will not be able to afford it? Will it be that only Google can afford it, thus making their monopoly position even stronger?

Is there a Wikipedia crawling "welfare" program if I'm not a trillion dollar mega company?

Re:Only Google is really allowed to crawl the web by dannyw on Friday March 26, 2021 @11:08PM (#26599565)
Actually, yes! The current API is free. The new Enterprise API is paid.
Re:Only Google is really allowed to crawl the web by aabhay on Saturday March 27, 2021 @12:07PM (#26603290)
Sure! Apply to become a crawler. And if you meet certain criteria and your crawlers don’t exceed a quota then have at it. The key is not to make it technically challenging, but to erect a legal barrier.
Re:Only Google is really allowed to crawl the web by rincebrain on Friday March 26, 2021 @12:42PM (#26594163)
Wikimedia recently announced Wikimedia Enterprise for "organizations that want to repurpose Wikimedia content in other contexts, providing data services at a large scale".

So they're pretty clearly looking to monetize organizations which consume their data in a for-profit context.

Re:Only Google is really allowed to crawl the web by dathinab on Friday March 26, 2021 @02:31PM (#26595313)
monetizing != for-profit

You could e.g. just cover operational cost and/or improve the service quality from it.

Re:Only Google is really allowed to crawl the web by pmiller2 on Friday March 26, 2021 @02:48PM (#26595518)
I think they may have meant "(organizations) (which consume their data in a for-profit context)."
Re:Only Google is really allowed to crawl the web by dmitriid on Friday March 26, 2021 @12:41PM (#26594150)
Google was/is also the largest sponsor of Mozilla. This doesn't stop Google from sabotaging Mozilla.

2 mln is probably Google's hourly profit. For that they get one of the biggest knowledge bases in the world. It's basically free as far as Google is concerned.

The instant Google becomes confident they can supplant Wikipedia, they will.

Re:Only Google is really allowed to crawl the web by jedberg on Friday March 26, 2021 @01:51PM (#26594937)
> 2 mln is probably Google's hourly profit.

You don't have to guess, their numbers are public. In 2020 they made $40B in profit, so it takes them about 27 minutes to make $2M in profit.

Re:Only Google is really allowed to crawl the web by billiam on Friday March 26, 2021 @03:02PM (#26595652)
NOT a sponsor of Mozilla. Google buys web traffic (as default search engine) for ~$300M and turns it into several times that $ in ad revenue.
Re:Only Google is really allowed to crawl the web by kelnos on Friday March 26, 2021 @03:14PM (#26595787)
Not sure why you're being downvoted; I completely agree with what you're saying (modulo questionable usage of "sponsor"). If Wikipedia were to try to charge for this use of their data, Google would likely make it a priority to drop the Wikipedia blurbs, either without replacement, or with data sourced elsewhere.
Re:Only Google is really allowed to crawl the web by will4274 on Friday March 26, 2021 @06:43PM (#26597889)
> Google would likely make it a priority to drop the Wikipedia blurbs, either without replacement, or with data sourced elsewhere.

That's an odd way of phrasing things. If Wikipedia were to take away free access to their data, Google wouldn't be dropping Wikipedia, Wikipedia would be dropping Google. This line of thinking "you took this when I was giving it away for free, but now I want to charge for it, so you are expected to keep paying for it" is incorrect.

Re:Only Google is really allowed to crawl the web by zdragnar on Friday March 26, 2021 @03:59PM (#26596261)
Given the scale that google already operates at, I don't doubt that they would just take a copy of thr content and rebrand it as a google service, complete with user contribution.

Then, after two or five years, let it fester then abandon it. Nobody gets promoted for keeping well oiled machines running.

Re:Only Google is really allowed to crawl the web by dmitriid on Friday March 26, 2021 @05:44PM (#26597331)
Remember Knol? https://en.wikipedia.org/wiki/Knol?wprov=sfti1

It was actually good for writing stuff when I tried it. Never brought in enough traffic. Killed.

Re:Only Google is really allowed to crawl the web by magicalist on Friday March 26, 2021 @12:43PM (#26594179)
> Google was/is also the largest sponsor of Mozilla. This doesn't stop Google from sabotaging Mozilla.

Google isn't a sponsor of Mozilla, they're a customer. Do people think Google is "sponsoring" Apple with $1.5 billion a year too?

Re:Only Google is really allowed to crawl the web by foobarian on Friday March 26, 2021 @12:52PM (#26594271)
> they're a customer.

The cynic in me thinks the product is anti-trust insurance.

Re:Only Google is really allowed to crawl the web by saagarjha on Friday March 26, 2021 @08:59PM (#26598815)
$1.5 billion a year? You're off by an order of magnitude; the number is thought to be over $10 billion a year.
Re:Only Google is really allowed to crawl the web by dmitriid on Friday March 26, 2021 @02:37PM (#26595376)
Google being Apple's customer doesn't mean Google isn't sponsoring Mozilla.

These are two very different companies with a very different relationship with Google. And very different influences on Google.

Google wants to be on iOS. It brings customers to Google. A lot of them. iOS is possibly more profitable to Google than Android even with all the payments Apple extracts from them.

Google needs Mozilla so that Google may pretend that there's competition in browser space and that they don't own standards committees. The latter already isn't really true, and Google increasingly doesn't care about the former.

Re:Only Google is really allowed to crawl the web by tomp on Friday March 26, 2021 @02:22PM (#26595224)
Well then they can't nag users to donate to Jimmy Wales' trust fund.
Re:Only Google is really allowed to crawl the web by minikites on Friday March 26, 2021 @02:12PM (#26595132)
Couldn't you make a similar argument about for-profit uses of free/libre software? The software serves a useful purpose, who cares where it came from?
Re:Only Google is really allowed to crawl the web by onetimemanytime on Friday March 26, 2021 @03:59PM (#26596263)
>>Google donated $2M [1]. In 2010, Google also donated $2m [2].

$2 Million a year? Now I know why Googlers complained about having one less olive in their lunch salad.

How much does Google PROFIT from Wikipedia and how much does Wikipedia loses in fundraising when Google fails to send users to the info provider?

Re:Only Google is really allowed to crawl the web by lupire on Friday March 26, 2021 @05:49PM (#26597381)
Wikipedia is drowning in money so this whole line of discussion is weird.

And most of the value of wikipedia is created by its unpaid users, not Wikimedia foundation.