For twitter at least, that would have to be done manually. It still shows a timeline for grey checkmark (government) accounts, and a "best of" type page for all other accounts.

Most sites serve a special version of the page to visitors with "googlebot" in their UA string and/or coming from an IP range google controls with more SEO'd contents too.

That would be equivalent to demonetizing the entire web. Free content would win out over paid content regardless of quality. As the old adage goes, "when you're getting something for free, you're the product being sold." Only sites making money by, shall we say, "indirect" means would be able to survive. A search engine which prioritizes free content over paid would become nothing but a propaganda engine.

I think I should at least be able to see even a subset of the content that caused the item to be returned in the search result, though. If I try to navigate away or see more content, sure, make me log in. But, if I search something, click on a Twitter/Facebook/Linkedin result, I should at least be able to see something.

The search --> visit --> immediate redirect to login results should be de-ranked.

“Free content would win out over paid content regardless of quality” this doesn’t follow unless we assume the most extreme implementation, the openness of the content is just one factor of many that should count in the contents favor. Further it assumes the only non-shady way to monetize content is put it behind a login which is not true.

A site can be a billboard for a product or service, or provide a social hub, without participating in the surveillance adtech industry. There are plenty of hobby forums, like those for craft brewing, which get supported by brewery suppliers, for example. There are luthier communities which get supported by toolmakers and professionals, and so on. The implicit community networking, reviews by community members, and other interactions reward quality and honesty, and penalize the shady shit.

It's just not scalable into the exploitative cash cows that VCs drool over.

>>> nothing but a propaganda engine

And that's different from Google, how?

A search engine which prioritizes free content, reviewed intelligently, is curation, and not Goodharted gotcha games. If you can crawl the web and index sites with human level content curation, with a reasonably performant scaffolding, you can prevent SEO style exploitation, and use natural language rules like "does this content contain text attempting to game the ranking of a site or violate policy XYZ?"

Most AIs use bing and google, so the best you can get is a curated list from the already censored and politically filtered results from those sources, funneling commercial traffic toward the highest paying adtech customers - it's just refined, ultra-pure SEO results, unless they use their own index and crawler.

I'd almost rather have a naive raw index that can be interacted with, but custom indices, like xAI and Kagi, are definitely superior to Google and Bing. Google's a dumpster fire and Bing's a tawdry knockoff, and they're both interested in gaming the surveillance data and extracting as much money as possible from their adtech customers.

Paying for a service incentivizes the quality of that service. If that service is honest curation of and effective web search with custom indices and crawlers, then the free and paid distinction don't matter - the highest quality based on the curation criteria is what gets a site surfaced. I want my search engine to return McMaster Carr over Temu or Amazon, or a local flower shop over some corporate slop. Google doesn't get paid by meeting my expectations, it gets paid by exploiting my attention and extracting fractions of profit from commercial interactions, and makes more money by pushing me into business with companies that I'd otherwise want nothing to do with.

Demonetizing the entire web - dismantling the surveillance adtech regime - sounds like an absolute utopic victory to me.

> Free content would win out over paid content regardless of quality.

Quality content has never been synonymous with paid content for as long as I can remember. The primary driver of low quality or harmful content is greed. Guess what fuels the most greed in modern society?

> A search engine which prioritizes free content over paid would become nothing but a propaganda engine.

Are you suggesting that including Twitter in search results would mitigate propaganda?

Because Google wants the web to be broken like that, they're also part of the design team of tech behemoths that made the internet shitty und no fun.

They have had ways of letting people who give Googlebot access to content that requires login for a long time. A decade?

Because people can get a login. If the best quality result is behind a login and a paywall, I still want it to be the first result. Only quality should decide ranking.

Please do tell how to get an X account? It instantly locked my account after registration and I have several friends have the same issue.

I would much prefer if Google just stopped showing inaccessible information completely.

I have no idea, I've never used X or Twitter. But apparently millions do, so it is not inaccessible.

Well I sure don't

Openness and accessibility should absolutely be factors in ranking, otherwise where does it end? I dk what twitter requires these days, maybe an email, password and a couple more fields, what if a site starts doing id verification? What if accounts require a subscription? What if all the best content on the first page of your search results is behind a paywall with 3 easy payments of $299

It ends with you paying for information. If I need some information and it is only available behind a paywall, then I'll pay for it or I didn't need it anyway.

Google is doing the correct thing in not discriminating against content which is paid or behind login walls. Some of the most important content are on social media, and most of them only serve logged in users.

If you want to decide yourself how search results are presented to you, you should try Kagi for a search engine.

You haven’t made a case for why free content should be more important than relevance.

If the search engine orders by relevance, than I can make the decision for myself of where to trade-off with paywalls.

I don’t want a search engine to make the decision for me because it cannot: what if the only answer to my question is behind a paywall?

“ You haven’t made a case for why free content should be more important than relevance.”

I didn’t make that claim, i am contesting the claim “Only relevance should decide ranking”. I am arguing ease of accessibility should be a factor.

Unless you mean as a tie-breaker in the niche case of equally-relevant results, then you're making the claim that it's better to see less relevant (or nonrelevant) results just because they have some other quality beyond relevance.