Which websites are shown in the search results is not influenced by whether Google has ads on them.

The only thing that influences Google search results is Google's desire to keep as many people using Search as often as possible, since nearly all of Google's money comes from showing those text ads at the top of the Search results. This is all public information, you can read it in the 10K etc.

So if Search sucks, it's not because Google has the wrong incentives but because they can't solve the problems Search faces.

Hard to proof this but anecdotally lots of energy that used to go into people's hobby or passion websites now is going into digital platforms like reddit, pinterest or Twitter, or if lucky substack or Medium. Some of those are walled gardens and Reddit as discussed here is where people now search. Much less out on the actual net

Well, since I actually run a search engine looking for this sort of stuff, I think I'm able to demonstrate that a lot of what was out on the net is still there, just exceptionally hard to find on Google.

Admittedly more in some areas than others.

Hah, I didn't catch that I was responding to you. Definitely would defer to your experience. Love your search engine btw. It brings the fun back to the web!

If anything I think I'm only scraping the surface, since a lot of my methods involve ruthlessly discarding data that doesn't live up to a fairly blunt set of criteria. I think with something like a headless browser (and a lot more processing power), I could probably use laxer standards and find even more good stuff.

Show me a site made by people with manually instead of algorithmically curated links and content.

I'm not quite sure what you are asking, but this maybe one of these?

https://simplifier.neocities.org/

https://cheapskatesguide.org/

https://indieseek.xyz/

https://www.gameboomers.com/

https://sadgrl.online/

Hello, I have never heard of cheapskatesguide.org before and I am loving it so far. Thanks for sharing, I will check out the rest as well.

Great, now compare this to all the content generated on content farms for selling ads and gaming search engine rankings.

Right, but that's pretty trivially identified. I've had great success doing that with my search engine. Here's a thousand domains that are low in farmed contents in no particular order:

https://downloads.marginalia.nu/good-domains.txt

So why do you suppose Google doesn't surface the domains and results you presented? For example, I searched for "game reviews" and gameboomers was nowhere to be found.

I have no insight in their search engine, but I do know it would hurt their ad revenue to surface results that have no ads. Lends itself to speculation. But it could just be some confluence of other factors, of course. They seem to aggressively favor recent content (I do the opposite).

Which is why I said it's an ouroboros eating its own tail. The scale it operates at and given that it's also an ad network guarantees that whatever results it finds will favor its own ad network. It doesn't even have to be intentional since all they're doing is optimizing some metrics and running ML algorithms. There is no single person that could be blamed for the deterioration of the results. There is no way around this conflict of interest and they will continue pushing the envelope to increase their own revenue at the expense of useful results for as long as possible.

If a site is hosting ads from the Google ad network and is barely above being spam then Google will prioritize it over other results in order to maintain its quarterly revenue predictions.

That's a reasonable analysis. I do think Google has grown about as it can legitimately grow, which means the only way it can maintain an appearance of growth is by cannibalizing itself. The increasingly aggressive use of search ads are probably an example of this.

When are you going to have a VC fork over $10,000,000 and start your empire??

I use your crazy search engine all the time, and it's fucking great. Last time I checked, though, you're getting like $30 a month from Patreon and that's it, haha.

You're too much of a well-adjusted normal human being, that's your problem. You need that touch of Zuckerwellian sociopathy to get to the next level

Well I'm actually up to a $100 now, which is about twice my burn rate, so you might as well say I've broken even already. To the moon, baby.

To be serious, it's notoriously difficult to actually make a profit off search. I guess we'll see if Kagi will be able to eke out a market for itself, that may change things. Right now, I think investors will look at Bing and go "yikes".

Doesn't go to say I couldn't build a fantastic search engine with more resources than my current shoestring budget, the hard question remains: Why isn't this just a waste of money? Would take one hell of a sales pitch to gloss over that concern.