It really seems high-quality search is fundamentally in opposition to serving ads, alas. (At least once every page in existence probably serves ads via the search operator's network.)
I have a lot of complaints about, say, McDonald's, but they make their money through giving people something of value to them. Advertising legitimizes the making of money in a way unrelated to value delivery. (The same is true about a lot of finance.)
When you combine that with up-and-to-the-right numerical goals and standard executive incentives, over time you pretty much guarantee what Doctorow calls "enshittification". Delivering value becomes at best a side effect of the system.
"[W]e expect that advertising funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers"
and
"[W]e believe the issue of advertising causes enough mixed incentives that it is crucial to have a competitive search engine that is transparent and in the academic realm."
Both from The Anatomy of a Large-Scale Hypertextual Web Search Engine by Sergey Brin and Lawrence Page (1998)
... but more seriously: Yeah, their reasoning (now) isn't exactly going to be unbiased and/or unblemished by their own experience. They probably have the most extreme survivorship bias ever. (Not their fault, they just do because of whatever factors got them into the position they're in.)
There are, of course, a few relatively successful paid general purpose search engines but these serve a niche demographic if you consider the world-wide scale of google et al. Possibly specialized search (we build one) will be able to thrive in the future, but these engines also serve niche markets in the end.
Thus, the real competition to ad-based search is not high quality search and that is likely why search results don't get better.
Google should have leaned into just this small segment of tooling and done much more of a ban hammer on the bad actors. They didn't and a bunch of people have left. I use DuckDuckGo mostly and sometimes Google but never by default. I don't even like DDG that much but it's good enough.
The popular "Awesome ${whatever}" collections are very much that, link indexes for relatively narrow areas.
Curation matters. This is something computers are still bad at, or are too expensive to deploy for a mass-market service.
The challenge will be staying true to not showing ads, respecting user privacy, and not requiring a subscription. So far, the only thing that works is free daily quota + pre-paid
Now we can use ChatGPT to filter through Google's infested mess, but this double edged Sword of Damocles will be able to create infinite attempts to bury genuine content with ad spam.
Like in the back of a book, or an old subject based card catalogue ... Dewy-decimalise the web :)
I've always felt that an index by subject would be more useful than string-match based searching. Of course, the index might rank links within each sub-sub--sub-sub...category with something like the original page-rank.
Now if Yahoo (or whoever) could avoid the enshitification trap ... imagine what a fabulous resource that could be.
Now that is an awesome idea. I really do think we need to have category-based indexing as well as page search.
Do you mean self-managed?
Everything else is effectively the influencer scene. Which is increasingly deplorable as well.
Anything with wide enough reach becomes cost-effective for gaming.
So one would have to return to a highly fragmented world to make gaming the system cost prohibitive.
And that would get us to a pre-Internet world. But then again, it’s not entirely unthinkable that we’re headed towards increasing Internet fragmentation if various governments get their way.
That amount of money is probably more than Google now makes from my online presence because I adblock, block 3rd party cookies, tend to click "block" to everything including the idiotic "legitimate concern" and never ever click on ads.
wow, Search really isn't worth developing.
Edit: not beam but color
Fuck it, I'd pay $15 a year to have a Google search that puts as much effort into finding me the shit I actually want as Google does today in serving me BS ads I never pay attention to.
They announced "Google Contributor" at one point but it never went anywhere
The ads industry would not like their reach to be limited to those not paying for premium search.
Tell me exactly how much the advertiser paid for his placement, and that's a hugely important signal here.
If I'm searching for weird hobby parts, even though it's a high purchase intent query, they're probably paying pennies per click.
But if you start searching, say, financial stuff and the ad placement figures start showing multiple dollars per click, it's a warning "these people are willing to spend THIS MUCH MONEY to present a message to you, this probably means there's something sketchy involved."
I know, for example, anything pertaining to insurance and financial products is highly likely to turn into a farm of cross-selling and personal-information harvesting, because the cost per acquisition is so high and the tendency for everyone to sell the information to everyone else is so great.
Would 3 EUR or even 10 EUR / year for each customer be enough to run operations?
Of course people game the system, apply shady practices, sell courses with tips. It is a nuance topic, the name now is bastardized by marketing companies doing shady things, but on his core is just practices to create good websites and provide a good user experience
Unfortunately, there's no going back - the closest we can get is BBS-via-SSH. The entire landline phone infrastructure is crumbling around us (or in many cases, completely gone). Voice calls are packet-switched now, rather than circuit-switched as in the past. The upshot is, fancy modulation techniques that made full-duplex 33.6k possible over voice-grade connections aren't going to work, and even good old Bell 103 (300 baud) may end up being problematic.
I'm not sure I can even get new landline phone service, and if so, it's going to be expensive - and the wire plant is an unmaintained mess. When I got my folks off their landline and onto VoIP some years back, their old landline had so much hum it was nearly unusable. Once the inside wiring was disconnected from the landline and connected to an ATA, the hum was gone. It wasn't our wiring.
When gaming search engines became a profession, the end of search appeared on the horizon. Guess we're headed back to web rings and link indexes (which will be consolidated, heavily monetized, and abandoned). If we're lucky, we'll be back to dialup BBSes by the 2040s.