News for hackers. Stuff that matters.
Re:Wikipedia says traffic is falling due to AI search summar... by qingcharles on Monday October 20, 2025 @10:39PM (#45651868)
They are highly dependent on web traffic for revenue.

And their costs are even increasing because while human viewers are decreasing they are getting hugged to death by AI scrapes.

Re:Wikipedia says traffic is falling due to AI search summar... by khamidou on Monday October 20, 2025 @11:45PM (#45652263)
If you look up their latest annual report (https://wikimediafoundation.org/wp-content/uploads/2025/04/W...) you can see that they're allocating ~1.7% of their expenses towards hosting.

I doubt that they're getting "hugged to death" by AI scrapers.

Re:Wikipedia says traffic is falling due to AI search summar... by Liquix on Tuesday October 21, 2025 @04:02PM (#45660953)
this is a very eye-opening read on their financials: https://en.wikipedia.org/wiki/User:Guy_Macon/Wikipedia_has_C...
Re:Wikipedia says traffic is falling due to AI search summar... by skeaker on Tuesday October 21, 2025 @05:00PM (#45661639)
The easy counters to this article are:

1. I think their spending is a good thing. Charitable scholarships for kids and initiatives to have a more educated populous in general are things that I am happy to donate to.

2. As stated in the article, hosting is still a relatively simple expenditure compared to the rest of their operation. If Wikipedia really eats a huge loss, falling back to just hosting wouldn't be unrealistic, especially since the actual operations of Wikipedia are mostly volunteer run anyways. In the absolute worst case, their free data exports would lead to someone making a successor that can be moved to more or less seamlessly.

The only real argument in my eyes is that their donation campaigns can seem manipulative. I still think it's fine at the end of the day given that Wikipedia is a free service and donating at all is entirely optional.

Re:Wikipedia says traffic is falling due to AI search summar... by mminer237 on Tuesday October 21, 2025 @05:34PM (#45662012)
AFAIK, they don't do any scholarships or really do any educational activities. By far their biggest spending item is just $105 million for salaries, mainly for all of its leadership, which is a majority of its expenses.

The second biggest line item is grants at $25 million, primarily for users to travel to meet up.

Then $10 million for legal fees, $7 million for Wikipedia-hosted travel.

I think it's pretty unethical to say you have to donate to keep Wikipedia running when you're practically paying for C-suite raises and politically-aligned contributors' vacations.

Re:Wikipedia says traffic is falling due to AI search summar... by bawolff on Tuesday October 21, 2025 @06:04PM (#45662309)
In person meetings move things forward.

Paying the travel for a bunch of highly active volunteer contributors to meet up ocassionally and hash out complex community issues pays massive dividends. It keeps the site moving forward. Its also pretty cheap when you consider how much free labour those volunteers provide.

Whenever people criticize wikimedia finances, i think they miss the forest for the trees. I actually think there is a lot to potentially crticize, but in my opinion everyone goes for the wrong things.

Re:Wikipedia says traffic is falling due to AI search summar... by gtsop on Wednesday October 22, 2025 @02:26AM (#45665510)
What are the rights things to criticize in your opinion?

Also, asking out of ignorance, what things need to move forward? I thought wikipedia is a solved problem, the only work i would expect it to need is maintenance work, security patches etc.

Re:Wikipedia says traffic is falling due to AI search summar... by bawolff on Wednesday October 22, 2025 @03:19AM (#45665825)
> What are the rights things to criticize in your opinion?

I think criticism should be based on looking at what they were trying to accomplish by spending the money, was it a worthwhile thing to try and do and was the solution executed effectively.

Just saying they spent $X, X is a big number, it must be wasteful without considering the value that is attempting to be purchssed with that money is a bit meaningless.

> Also, asking out of ignorance, what things need to move forward? I thought wikipedia is a solved problem, the only work i would expect it to need is maintenance work, security patches etc.

I think the person who i was responding to was referring to volunteer travel not staff travel (which of course also happens but i believe would be a different budget line item). This would be mostly for people who write the articles but also for people who do moderation activity. In person meetings can help resolve intractable disputes, share best practises, figure out complex disagreements, build relationships. All the same reasons that real companies fly their staff to expensive offsites.

Software is never done, there are always going to be things that come up and things to be improved. Some of them may be worth it some not.

As an example, there are changes coming to how ip addresses are handled, especially for logged out users. Nobody is exactly saying why, but im 99% sure its GDPR compliance related. That is a big project due to some deeply held assumptions, and probably critical.

A more mid-tier example might be, last year WMF rolled out a (caching) server precense in Brazil. The goal was to reduce latency for South American users. Is that worth it? It was probably a fair bit of money. If WMF was broke it wouldn't be, but given they do have some money, it seems like a reasonable improvement to me. Reasonable minds could probably disagree of course.

And an example of stupid projects might be WMF's ill-fated attempt at making an AI summarizer. That was a pure waste of money.

I guess my point it, WMF is a pretty big entity, some of the things they do are good, some are stupid, and i think people should criticize the projects they embark on rather than the big sum of money taken out of context.

Re:Wikipedia says traffic is falling due to AI search summar... by bawolff on Tuesday October 21, 2025 @05:58PM (#45662257)
People cite this figure a lot, but its a little misleading because when you own your own servers a lot of the expenses that are typically hosting actually fall under a different category.

If you use AWS, the people hired to manage the servers is part of the price tag. When you own your own you have to actually hire those people.

Re:Wikipedia says traffic is falling due to AI search summar... by khamidou on Tuesday October 21, 2025 @09:07PM (#45663820)
I mean, it's not like you can get away with running with zero SREs if you're running in the cloud. The personnel costs for on-prem hosting are vastly exaggerated, especially if you contract out the actual annoying work to a colo.
Re:Wikipedia says traffic is falling due to AI search summar... by ryan_lane on Wednesday October 22, 2025 @02:22AM (#45665478)
Smart hands is more expensive than having dedicated datacenter staff, and the dedicated staff do a considerably better job. It's worth noting that WMF runs _very_ lean in terms of its datacenter staff.

You're also ignoring the need for infrastructure/network engineers, software engineers, fundraising engineers, product managers, community managers, managers, HR, legal, finance/accounting, fundraisers, etc.

Re:Wikipedia says traffic is falling due to AI search summar... by DeusExMachina on Wednesday October 22, 2025 @05:07AM (#45666524)
But in that case your costs don't go up because of AI scrapers, as you don't need to scale employees with traffic.
Re:Wikipedia says traffic is falling due to AI search summar... by crmd on Monday October 20, 2025 @10:55PM (#45651966)
How is their revenue traffic-dependent?
Re:Wikipedia says traffic is falling due to AI search summar... by AstroBen on Monday October 20, 2025 @11:09PM (#45652034)
Their traffic is potential donations

Something tells me a person is way less likely to donate if they're consuming the content through an LLM middleman

Re:Wikipedia says traffic is falling due to AI search summar... by crazygringo on Tuesday October 21, 2025 @08:25AM (#45654948)
I don't know -- as I said in another comment, my Wikipedia usage has gone down 90% thanks to LLMs.

But that means I'm still using it. Especially for more reference stuff like lists of episodes, filmographies, etc. As well as equations, math techniques, etc.

If you're the kind of person who donates to Wikipedia, you're probably still using it some even if less, and continue to recognize its importance. Possibly even more, as a kind of collaboratively-edited authority like Wikipedia only becomes more important as AI "slop" becomes more prevalent across blogs etc.

Re:Wikipedia says traffic is falling due to AI search summar... by arbol on Tuesday October 21, 2025 @05:10PM (#45661754)
But were you originally convinced to donate by one of those giant Jimmy Wales banners that come up once a year? People won't see them anymore if they're using AI summaries.
Re:Wikipedia says traffic is falling due to AI search summar... by crazygringo on Tuesday October 21, 2025 @05:40PM (#45662065)
My point is, using Wikipedia 10% as much is still using Wikipedia.

Does it matter if you see the banner 10 times or 100 times in a month?

Re:Wikipedia says traffic is falling due to AI search summar... by arbol on Wednesday October 22, 2025 @04:03AM (#45666101)
In terms of the effectiveness of the banner, yes it absolutely does. Multiple exposures increase your propensity to convert.
Re:Wikipedia says traffic is falling due to AI search summar... by johnnyanmac on Monday October 20, 2025 @10:52PM (#45651951)
scraping Wikipedia feels like the stupidest possible move. You can in fact download the entire encyclopedia at any time and take all the time in the world parsing offline.

For such purposes, I'd naively just setup some weekly job to download Wikipedia and then run a "scrape" on that. Even weekly may be overkill; a monthly snapshot may do more than enough.

Re:Wikipedia says traffic is falling due to AI search summar... by yorwba on Tuesday October 21, 2025 @12:46AM (#45652556)
You can download twice-monthly database dumps, but they consist of the raw wikitext, so you need to do a bunch of extra work to render templates and stuff. Meanwhile, if you write a generic scraper, it can connect to Wikipedia like it connects to any other website and get the correctly-rendered HTML. People who aren't interested in Wikipedia specifically but want to download pretty much the entire internet unsurprisingly choose the latter option.
Re:Wikipedia says traffic is falling due to AI search summar... by parpfish on Tuesday October 21, 2025 @07:48PM (#45663239)
as somebody that has wrassled with the wikipedia dumps a number of times, i don't understand why wiki doesn't release some sort of sdk that gives you the 'official' parse
Re:Wikipedia says traffic is falling due to AI search summar... by skywal_l on Wednesday October 22, 2025 @03:10AM (#45665781)
This. I tried that a few years ago and fell off my chair when I started to realized how DYI the thing is. It's a bunch of unofficial scripts and half-assed out of date help pages.

At the time I though, well it's a bunch of hippies with a small budget, who can blame them? Now I learn that there is 600 of them with a budget in the hundreds of millions??

This is becoming another Mozilla foundation...

Re:Wikipedia says traffic is falling due to AI search summar... by philipkglass on Wednesday October 22, 2025 @11:29AM (#45670630)
I have wrestled with it too. I believe it's because wikitext is an ad-hoc format that evolved so that the only 100% correct parser/renderer is the MediaWiki implementation. It's like asking for an SDK that correctly parses Perl. Only Perl can do that.

There are a bunch of mainly-compatible third party parsers in various languages. The best one I've found so far is Sweble but even it mishandles a small percentage of rare cases.

Re:Wikipedia says traffic is falling due to AI search summar... by Gander5739 on Wednesday October 22, 2025 @03:07AM (#45665763)
There are also dumps of Wikipedia in html format.
Re:Wikipedia says traffic is falling due to AI search summar... by yorwba on Wednesday October 22, 2025 @03:47AM (#45666001)
Do you mean the discontinued Enterprise HTML dumps https://dumps.wikimedia.org/other/enterprise_html/ or the even older discontinued static HTML dumps https://dumps.wikimedia.org/other/static_html_dumps/current/... or is there another set of dumps I'm not aware of?
Re:Wikipedia says traffic is falling due to AI search summar... by cm2012 on Monday October 20, 2025 @10:53PM (#45651961)
Isn't it true that only around 10% of Wikipedia massive budget is used to actually run the core website? The rest goes to bloated initiatives in the Wikimedia foundations orbit.
Re:Wikipedia says traffic is falling due to AI search summar... by vrindavan1 on Tuesday October 21, 2025 @12:35PM (#45657860)
lmao
Re:Wikipedia says traffic is falling due to AI search summar... by ethmarks on Tuesday October 21, 2025 @02:59PM (#45660072)
Page 21 of their 2024 annual report[1] has expenses listed. About $3,000,000 for web hosting, about $100,000,000 for salaries and benefits out of $178,000,000 total.

[1]: https://wikimediafoundation.org/wp-content/uploads/2025/04/W...

Re:Wikipedia says traffic is falling due to AI search summar... by cm2012 on Tuesday October 21, 2025 @03:28PM (#45660502)
That includes all of the Wikimedia websites and non-profit activities though, not just the functioning of Wikipedia.org proper. That is a much lower percentage of the total.
Re:Wikipedia says traffic is falling due to AI search summar... by bawolff on Tuesday October 21, 2025 @06:06PM (#45662334)
A web server is useless if you aren't paying someone to plug it in.

There is probably a lot to criticize, but you need to go deeper than "salaries" are bad. You need some of those to actually run the website.

Re:Wikipedia says traffic is falling due to AI search summar... by ThrowawayTestr on Tuesday October 21, 2025 @11:18PM (#45664580)
Even at a quarter million a year that's 400 salaries. How many people are needed to maintain a website?
Re:Wikipedia says traffic is falling due to AI search summar... by ryan_lane on Wednesday October 22, 2025 @02:29AM (#45665531)
How many people are needed to maintain instagram, facebook, etc? Wikipedia isn't just a collection of static content. The page contents include static and dynamic content (via a lua scriptable set of templates), semantic data (wikidata), multimedia management (wikimedia commons), editing tools (WYSIWYG editing, with full support of the wiki markup and templating), global caching infrastructure, multiple datacenters for HA/DR, etc.

There's also the need to support the staff and volunteer developers, which includes wikimedia cloud services, git hosting, config management and orchestration, CI, community hosted tool/bot services, etc.

WMF has ~600 employees, and that's quite lean, for a service of their complexity.

Re:Wikipedia says traffic is falling due to AI search summar... by fart-fart-FART on Tuesday October 21, 2025 @07:41AM (#45654655)
how much money do you people think it costs to serve a few kilobytes of text? come on now - this is the orange reddit, not the regular one.
Re:Wikipedia says traffic is falling due to AI search summar... by 113 on Tuesday October 21, 2025 @09:04AM (#45655252)
They also serve images.
Re:Wikipedia says traffic is falling due to AI search summar... by fart-fart-FART on Tuesday October 21, 2025 @10:09AM (#45656018)
most crawlers are either simple HTTP clients or JS-executing headless browsers. they don't fetch multimedia indiscriminately/repeatedly. why would they? only a few FAGMANs can afford to hoard all the jeepegs on the Internet.

the outlandish claims of chaotically evil AI crawlers DDoSing web properties to death are rooted in ideology and not objective reality. here's an example of what a fairly modest modern server can handle: https://www.techempower.com/benchmarks/#section=data-r23&tes...

Re:Wikipedia says traffic is falling due to AI search summar... by _def on Wednesday October 22, 2025 @02:28AM (#45665525)
> In this test, the framework responds with the simplest of responses: a "Hello, World" message rendered as plain text.

What a ridiculous comparison. You either don't know what you are talking about, or are a troll. Maybe even both.

Re:Wikipedia says traffic is falling due to AI search summar... by fart-fart-FART on Wednesday October 22, 2025 @08:59AM (#45668401)
there are more complex tests in that benchmark. my point was, a decent server can serve tens or hundreds of thousands requests per second when it's a few kilobytes of static content from a RAM database.

as long as your web service isn't horribly mismanaged, even to a $5 VPS the bot traffic is infinitesimal background radiation.

Re:Wikipedia says traffic is falling due to AI search summar... by lxgr on Tuesday October 21, 2025 @03:34PM (#45660589)
Some of them even fairly often: https://news.ycombinator.com/item?id=26072025
Re:Wikipedia says traffic is falling due to AI search summar... by bawolff on Tuesday October 21, 2025 @06:07PM (#45662344)
Bandwidth costs aren't the dominating cost.
Re:Wikipedia says traffic is falling due to AI search summar... by Rebelgecko on Tuesday October 21, 2025 @03:12PM (#45660274)
Bandwidth is a ridiculously tiny portion of Wiki foundation's spending
Re:Wikipedia says traffic is falling due to AI search summar... by ryan_lane on Wednesday October 22, 2025 @02:31AM (#45665541)
Especially considering a considerably amount of their bandwidth is free, via peering agreements.
Re:Wikipedia says traffic is falling due to AI search summar... by WatchDog on Wednesday October 22, 2025 @12:12AM (#45664818)
Infrastructure has long been a tiny portion of wikipedias costs. I think Wikipedia even makes it easy to export all of its data, I don’t think AI scrapers would be a significant new cost
Re:Wikipedia says traffic is falling due to AI search summar... by _def on Wednesday October 22, 2025 @02:23AM (#45665489)
These are two very different cost factors. Scrapers don't use the available data dumps, that's why they scrape. They are also kind of dumb as they get lost in link structures constantly, which leads to unecessary traffic spikes.
Re:Wikipedia says traffic is falling due to AI search summar... by Aurornis on Wednesday October 22, 2025 @01:34AM (#45665218)
> they are getting hugged to death by AI scrapes.

Wikipedia is not getting hugged to death by AI scrapers.

The source letter shows a relatively small portion of traffic was reclassified as bot traffic.

They get a lot of page views globally. It’s a popular website. The bot traffic is not crushing their servers.