News for hackers. Stuff that matters.
Re:Google Is Burying the Web Alive by nicbou on Monday May 26, 2025 @01:30PM (#44099570)
I can weigh in on this. I run a small information website (https://allaboutberlin.com). My traffic has been growing steadily for 7 years, but now its down 30-ish percent year on year. From what I'm hearing, my competitors (and friends) are having it worse.

My main mission is to put new information on the internet. It's harder to do this if AI destroys the economics of it. It's also harder without an audience who provides feedback and encouragement. Having all information mediated by two companies isn't just digging into my revenue, it's killing the fun.

The unfortunate side effect is that it made me focus more on business and less on serving my readers. I'm working on health insurance stuff when I would rather work on a citizenship guide. It's also sad to see all the nuance stripped from my carefully chosen words.

It sucks, honestly. I am being replaced by AI, but I am still supposed to go experience the real world and translate it into LLM training material. In the end, there still is a thinking, feeling human being doing the essential work. He's just not getting paid anymore.

Re:Google Is Burying the Web Alive by mike_hearn on Monday May 26, 2025 @03:58PM (#44101095)
If you want readers more than impact you might want to consider something like a Substack. The paying audience would be smaller, but, it's not crawlable by AI (at least not today).

Still, it's a bit ironic that your own website advises people to use bots to get apartments (thus bypassing the work put into the adverts and apartment hunting sites), whilst also decrying bots who would bypass your site.

More generally perhaps there's a market for a Steam-like walled garden but for "old web" style sites. Crawlers would be controlled via technical means, micropayments would be built in, and use of AI would be strictly controlled. For instance maybe the garden uses AI to build link directories like DMOZ used to be, but the information itself isn't trained on so you're guaranteed to have organic readers.

Re:Google Is Burying the Web Alive by qkeast on Monday May 26, 2025 @04:17PM (#44101267)
>Still, it's a bit ironic that your own website advises people to use bots to get apartments (thus bypassing the work put into the adverts and apartment hunting sites), whilst also decrying bots who would bypass your site.

I believe these are pretty different scenarios. There's an ongoing, massive shortage of apartments in Berlin, and bots here often aren't bypassing "work put into adverts and apartment hunting sites"—they're trying to get an application in at all before the listing is flooded with hundreds of applications and disabled. The content of the application has nothing to do with it.

(Of course, this just leads to an arms race of apartment-application-bots, but this is the situation the city's found itself in. Same goes for registration appointments, residence permit applications, and so on.)

Re:Google Is Burying the Web Alive by nicbou on Monday May 26, 2025 @05:08PM (#44101694)
If you have a zero sum game, any advice you give is creating an arms race. The only solution is to have fewer Berliners or more housing.
Re:Google Is Burying the Web Alive by mike_hearn on Tuesday May 27, 2025 @03:55AM (#44104831)
Or eliminate rent control, allowing prices to float to the level people are actually willing to pay. You only see bot-driven stuff like that in cases where desirable goods are being deliberately under-priced.
Re:Google Is Burying the Web Alive by nicbou on Tuesday May 27, 2025 @04:27AM (#44104987)
This will benefit the wealthier strata of society, but why does the city belong to them more than anyone else?
Re:Google Is Burying the Web Alive by mike_hearn on Tuesday May 27, 2025 @06:04AM (#44105477)
Why should it belong to people who are good at botting? That's a much less fair way to allocate resources than wealth. In a free society wealth is used to buy things, that's the point of earning it. If you don't generate more wealth and you trade it away then eventually you end up poor again. If you're generating wealth continuously, then the things you can buy with it are the reward for doing so. It's much better for everyone that desirable apartments go to people who earned them rather than those who randomly managed to shave some milliseconds off a TCP connection.
Re:Google Is Burying the Web Alive by Larrikin on Tuesday May 27, 2025 @07:00AM (#44105723)
The wealthy can buy bots then.
Re:Google Is Burying the Web Alive by nicbou on Monday May 26, 2025 @05:04PM (#44101670)
A walled garden or paywalled website would go completely against the website's ethos. Everything is meant to be free for everyone. "Information wants to be free." Besides, those who stand to benefit the most from that information are the ones who can the least afford it.

The bots are not a solution, they're table stakes. Withholding that information would not do anyone any justice.

Re:Google Is Burying the Web Alive by mike_hearn on Tuesday May 27, 2025 @03:54AM (#44104820)
If your goal is for information to be as free as possible, then having AI read it is the best possible outcome. But it sounds like you have other goals too involving personal satisfaction, maybe community building etc. All very fair and reasonable goals to have, but they inherently conflict with "information wants to be free".
Re:Google Is Burying the Web Alive by nicbou on Tuesday May 27, 2025 @04:25AM (#44104981)
It's not. It strips me of my income, meaning I have to survive through other means. Of course I have other goals! One of them is to make rent.

For the readers, it also strips the information of a lot of nuance. I obsess over word choice for a reason. Then there are the footnotes, the citations, and the purpose-built tools embedded in the content.

As I said, I also lose access to the community and its feedback and encouragement.

That's now, before tech companies start throwing ads on top of the information it hoovered from my website. I'm glad that for a while, destroying the independent web will create value for the shareholders.

Re:Google Is Burying the Web Alive by pempem on Wednesday May 28, 2025 @12:10AM (#44112674)
If information is free, then the places it is accessed from should not be limited.

The premise behind libraries being public works rather than, say, private collections is that society as a whole is benefitting from this information being free and creating that information as well.

Here you are limited by the whims of a private actor which can be one way one day, and another way when VC money runs out or when it has a "fiduciary responsibility to shareholders" which is the inherent conflict the original poster is sharing.

Having all of the worlds information being parsed by private actors or non-profits who may become private at some time is a problem and the anthesis of "information wants to be free"

Re:Google Is Burying the Web Alive by bwb on Monday May 26, 2025 @03:08PM (#44100565)
Ya, I'm in a similar boat (https://shepherd.com). Despite sky-high engagement stats, we lost 70% of our traffic from Google. Luckily, we have a strong brand, a growing community, and get decent traffic directly (and from other search engines).

We are surviving, but we've changed our entire focus as the web we know is a dead man walking. AI bots are the new frontend for the web, and I am curious about what the web will look like in 2 to 10 years. I hope the AI companies start paying to access the data as they answer queries or construct dynamic frontends (but I doubt it).

I have many friends who ran great indie websites, and it has been sad to see that 90% of them are now frozen or closed.

Re:Google Is Burying the Web Alive by nicbou on Wednesday May 28, 2025 @04:32AM (#44113889)
I expect to lose a lot of information websites, especially when the outcome of a long research is a few bullet points (such as product/place recommendations). Once tech companies feel the need to monetise LLMs, all bets are off.
Re:Google Is Burying the Web Alive by bwb on Wednesday May 28, 2025 @05:31AM (#44114219)
Ya, I'm already seeing a bunch disappear.

I am curious what it looks like when you need a good product review in two or three years. Is YouTube the only trustworthy source by real people? For most products, the web is just going to be Astroturfed Reddit posts and Forbes AI-written junk that didn't even have the product to test.

I'm hoping paid MCP data sets might get there, then I could make my data available there, and hopefully make a $1 to $4 per thousand requests to the data set. If that replaced the loss of Google traffic, that would be amazing.

I'd love for 25% of my AI subscription to go toward that each month, i.e., I can pull "a list of all science fiction books the NYT recommended in the 1950s" and it gives NYT a few cents to go through their data...

Re:Google Is Burying the Web Alive by asdhtjkujh on Monday May 26, 2025 @02:27PM (#44100140)
I came across your website organically and it's been a great resource for me in the last few months, including times when the answers presented by Gemini have been definitively and legally incorrect. Thank you for organizing all of this information so clearly.
Re:Google Is Burying the Web Alive by boplicity on Monday May 26, 2025 @01:41PM (#44099674)
> My main mission is to put new information on the internet. It's harder to do this if AI destroys the economics of it. It's also harder without an audience who provides feedback and encouragement.

I also run an online website that provides valuable information. However, we have a very different philosophy. The mission is very not not to "put new information on the internet." Rather, it is to provide a valuable and much needed service for people in our niche. While we do greatly benefit from SEO, we also work very hard to cultivate our audience, both in terms of finding new people, and nurturing our relationship with the existing ones. "Information on the internet" has zero inherent value without an audience. If you're not working to build that audience (outside of third party platforms), then it's going to be very tough in the long run, and you'll run into situations exactly like this.

FWIW, our SEO traffic has remained shockingly steady, though the search console has reported a dramatic increase in "impressions," likely due to featuring our website in AI generated summaries. We'll see if things change over the long-term, but for now, we're not changing any strategies based on this.

Re:Google Is Burying the Web Alive by nicbou on Monday May 26, 2025 @01:49PM (#44099765)
The problem is that there is no audience without a third-party platform nowadays. At least not at a scale.

Google was the lesser evil, but I still fully expected to get killed by an algorithm update some day.

I vowed to ride this website into the ground if I have to. I believe in its value, even if I need a side job to support it.

Re:Google Is Burying the Web Alive by CobrastanJorji on Monday May 26, 2025 @02:08PM (#44099972)
There is no "nowadays." There never an era of big Internet audiences without third party platforms. The early web only had small audiences, and the reason it worked was that people were okay with that.
Re:Google Is Burying the Web Alive by tough on Monday May 26, 2025 @04:40PM (#44101462)
You could have a small audience of a few million users, not try to compete with the giants, and still, be serving millions of users (daily)

The internet was a beautiful place

Re:Google Is Burying the Web Alive by ElFitz on Monday May 26, 2025 @03:24PM (#44100721)
> He's just not getting paid anymore.

Or the credit. Or even just the feedback.

Even LLMs get at least some feedback.

Re:Google Is Burying the Web Alive by nicbou on Monday May 26, 2025 @05:10PM (#44101714)
This is a surprisingly taxing change. There's a difference between giving free eggs to your neighbours, and giving free eggs to the Holiday Inn across town.
Re:Google Is Burying the Web Alive by ElFitz on Tuesday May 27, 2025 @04:09AM (#44104907)
Exactly.
Re:Google Is Burying the Web Alive by RobRivera on Monday May 26, 2025 @03:43PM (#44100913)
We need a new information backbone
Re:Google Is Burying the Web Alive by RajT88 on Monday May 26, 2025 @08:58PM (#44103017)
That was the idea behind Internet2.

Never used it, maybe someone can explain how it has evolved.

Re:Google Is Burying the Web Alive by DataDaemon on Monday May 26, 2025 @01:44PM (#44099706)
This is a great information website, congrats, Google is happy: "thanks for feeding me", they will scrape all and display on their site in the nice table. Sorry about that.
Re:Google Is Burying the Web Alive by AfterHIA on Monday May 26, 2025 @08:38PM (#44102925)
Just a little bit of encouragement: this is a really, really nice site that reminds me of growing up on the 1990's internet. You can tell you, "actually give a shit" about what you're sharing and that alone I think gives it credence and relevance. Don't let a silly thing like economics keep you from this worthwhile human achievement pal.
Re:Google Is Burying the Web Alive by ChiMan on Monday May 26, 2025 @08:48PM (#44102973)
Major free-rider problem, this is—one, however, that property rights could protect against: You own your data. If the OpenAIs of the world want to ingest your data, they pay you for it.
Re:Google Is Burying the Web Alive by pentagrama on Monday May 26, 2025 @08:50PM (#44102979)
A good strategy can involve betting on alternative distribution channels beyond search engines. Here are some ideas:

- Newsletters. You own your audience and your list is portable and interoperable.

- PWAs with push notifications. Be careful not to overwhelm users with too many messages. Make opt-out options easy to find.

- Door flyers with QR codes. A return to the basics that can still be effective.

- Social media. Still useful, but its reach and engagement seem to be declining.

Re:Google Is Burying the Web Alive by nicbou on Monday May 26, 2025 @08:58PM (#44103013)
I am providing information for specific situations. People don’t casually read about replacing lost keys until they lose their keys. It’s not entertainment.
Re:Google Is Burying the Web Alive by p3rls on Tuesday May 27, 2025 @12:58AM (#44104028)
My niche has been dominated by indian slop for a decade now who cares about a few robots getting in on the action?
Re:Google Is Burying the Web Alive by paulette449 on Tuesday May 27, 2025 @03:54AM (#44104824)
Thanks for sharing. I've been thinking of creating a non-revenue online guide to my new city, Lyon, France, but I've been concerned that I'd only be feeding the AI machine.
Re:Google Is Burying the Web Alive by -__---____-ZXyw on Tuesday May 27, 2025 @03:10PM (#44109848)
Excuse me for veering off topic but I saw this, was intrigued (I used to live in Lyon), browsed some of your previous comments here, saw the book Carmilla, which I'd never heard of, and am very happy. I will be reading J. S. Le Fanu, and am stunned to learn of him. Cheers!
Re:Google Is Burying the Web Alive by Jommi on Tuesday May 27, 2025 @05:23AM (#44105276)
its super helpful. supporting. you should also accept crypto!
Re:Google Is Burying the Web Alive by nextaccountic on Tuesday May 27, 2025 @05:43AM (#44105383)
What I can gather from your comment is that the problem is capitalism, not AI.