Is it just me, or I feel like Google does not provide anymore good results for me.

Like every time I search something completely out of my knowledge, like "How to purchase a property in Mexico", it will give me 100+ results of some results with autogenerated content like "10 best places to buy property in Mexico". And the only way to fix that would be to add something like `site:reddit.com`

I am starting to suspect that there might be nothing to find.

I just don't think people (other then the tech oriented) are creating websites and running forums - and why would they? Reddit might be be only place you _can_ find that type of content. What should search engines do then?

With a tiny number of exceptions, it might be that people chat on reddit, read Wikipedia, ask questions on the stackexchange network/Quora, local communities use facebook groups, and businesses have a wordpress site with nothing more then a bit of fluff, a phone number and an email address.

If all websites try to optimise for SEO, they undermine the assumption that the evaluation of a search engine is the pure consequence of how well a site satisfies a query.

I think it's very possible that we have effectively raised the noise floor so high that there is no signal, but also likely that perverse incentives from trying to profit off of search engines have made them our enemies instead of our friends.

For instance, does Google favor sites that run their own tools on them? I've stopped paying attention but recall hearing mutterings to that effect. If so then running the tools is a protection racket.

For other perverse incentives: if you try to rank sites by how long someone stays on them before backing out, or searching again, then you end up favoring rabbit-hole sites, that either string you along or suck you into a tangent. "Oh, this must have answered their question about keeping bees," no, they're reading gossip about the Queen of England and have forgotten all about beekeeping.

I believe people will at least start looking for alternatives. For example, I have been collecting search engines, and whenever I encounter a page with too many commercial-laden SEO-porked results, I use a different search engine in Firefox.

I have enabled the Search Bar, I can do Alt+D, Tab, Tab, enter my query, then click a different search engine, which searches instantly, unlike the main bar, where you have to press Enter once more after clicking.

I just added this one also. See my collection: https://0bin.net/paste/ZSCRYVx1#sxD+jBIpScJismXBYwaoJPh75TH9...

I had to do a `pip install lz4` then apply and run this change: https://gist.github.com/Tblue/62ff47bef7f894e92ed5?permalink...

And it did not work. I hit a brick wall. I completely lost trust in Firefox. I want a browser created by a non-profit. Thank you Google for corrupting everything you touch.

I then found this blessed soul: https://www.jeffersonscher.com/ffu/searchjson.html

And here are the ones not built-in:

https://metager.org/meta/meta.ger3?eingabe={searchTerms}

https://wiki.archlinux.org/index.php?title=Special:Search&se...

https://www.mojeek.com/search?q={searchTerms}

https://www.openstreetmap.org/search

https://pypi.org/search/?q={searchTerms}

https://www.qwant.com/?q={searchTerms}&client=opensearch

https://searx.xyz/

https://swisscows.com/opensearch.xml

https://yandex.com/search/?text={searchTerms}&from=os&clid=1...

https://whoogle.sdf.org/search

https://lazyweb.ai/nav/

https://en.wiktionary.org/w/api.php?action=opensearch&format...

https://github.com/search?q={searchTerms}&ref=opensearch

http://www.urbandictionary.com/define.php?term={searchTerms}

http://alternativeto.net/browse/search/?q={searchTerms}

https://search.marginalia.nu/search?query={searchTerms}&ref=...

https://kagi.com/search?q={searchTerms}

https://you.com/search?q={searchTerms}

https://millionshort.com/search?keywords={searchTerms}&remov...

https://search.f-droid.org/?q={searchTerms}&opensearch=1

https://www.gnod.com/search/

http://teclis.com:1333/?topics={searchTerms}

https://wiby.me/?q={searchTerms}

https://www.alexandria.org/?q={searchTerms}

Many more installable engines are available at https://mycroftproject.com as OpenSearch XML plugins, compatible with Firefox and discoverable by Chromium.

Instances of the engine indexing locally on the machine where a user is browsing the web.

I'm regularly reminded of this concept of a distributed search index, but at the moment it doesn't seem very likely that it'll gain traction.

Speaking about myself; I cold turkey migrated to DDG ~2 months ago. So far I've had to resort to Google search 10 times or so.

One thing I miss though is Google's nice visualisation of fast changing results e.g., match scores. For example: https://imgur.com/a/Q5nZkjo

Are there search benchmarks to be found somewhere?

There must be. If you want to write a search engine, you need a way to validate the results.

Here is an example (2014) Web track paper of the 23th TREC: https://trec.nist.gov/pubs/trec23/papers/overview-web.pdf (TREC has a plentitude of difference benchmark tasks and you can submit your own: https://trec.nist.gov/pubs/trec29/trec2020.html - recent TREC 2020 papers)

For some standard corpus.

The phrase I was looking for. Thx a bunch! Gonna marginalia that now.

Just to expand, if I want the api reference, say I search for defaultdict (for some reason I like using them but always have to look at the reference), I want the python documentation. I definitely don't want a third party telling me about it.

And if I search a "make list of tensor into tensor" type question, I want SO where someone had asked the same question and got "tensor.stack" as the reply, so I can understand the answer and follow up by looking at the tensor.stack pytorch reference it I want.

Anything else is wasting my time, I think most users with similarly specific queries are not looking for tutorials, they are looking for the names of functions they hypothesize exist, or references. That's why intermediary sites that try to give an explanation are annoying, at least for me.

In other words, I'd rather see these at the bottom of the SERP than the top, but I wouldn't want to completely eliminate them.

I'd like to try them out, could you mention which?

I first discovered Alexandria in early February: https://git.sr.ht/~seirdy/seirdy.one/commit/935b55f10f9024ee...

Around the same time, I also discovered sengine.info, Artado, Entfer, and Siik. By sheer coincidence they all were mentioned to me or decided to crawl my site within the same couple weeks. So yes, from my perspective there have been more than a few new smaller engines getting active on the heels of bigger names like Neeva, Kagi, Brave Search, etc.

The biggest thing I've found is that when doing technical searches it always turns up the sources I'm actually looking for, actively filtering all the GitHub/StackOverflow copycat sites.

It also seems to up-weight official docs compared to Google. For example, "how to read a json file in python" turns up the Python docs as the second result, where in Google they're nowhere to be found.

1. free version is curated topic filters that sit on top of Google -- best on laptop or desktop at moment / iterating mobile due to ad splash; it's unclear due to TOU that we can ever make that fully server-side legally, though do so have some things testing to see if free version can be ad-free or more tracking free

2. premium version will be mix of our scraping and Bing, depending on topic - standard web search via Bing + same curation as (1), closer to real-time for us

3. have tested out most other indie indexes or sites, for full web scale, money is on ahrefs or brave giving bing / google run for it

4. our primary emphasis is on bringing back some of the Yahoo! directory or Alta Vista look & feel of drilling into topics, so balancing 1-3 (^) as best we can atm, small team, fully bootstrapped modulo tiny F&F round

5. also @DotDotJames on twitter, still iterating when add team info to site

Try comparing results with a Bing-based engine (e.g. DuckDuckGo) or a Google-based one (e.g. StartPage, GMX) to see if they differ. (Don't use Google or Bing directly, since results will be personalized based on factors like location, device, your fingerprint, etc.).

To paint with a broad brush, I look at three criteria:

1. Infoboxes ("instant answers") should focus on site previews rather than trying to intelligently answer my question. Most DuckDuckGo infoboxes are good examples of this; Bing and Google ones are too "clever".

2. Organic results should be unique; most engines are powered by a commercial Bing API or use Google Custom Search. Compare results with a Bing or Google proxy (duckduckgo, startpage, etc) to avoid personalized results. Monitor queries over time to see if SERPs change in ways that diverge from Google/Bing/Yandex.

3. "other" stuff. Common features I find appealing include area-specific search (Kagi has a "non-commercial lens" mostly powered by its Teclis index; Brave is rolling out "goggles"), displaying additional info about each result (Marginalia and Kagi highlight results with heavy JS or tracking), user-driven SERP personalization (Neeva and Kagi allow promoting/demoting domains), etc.

And always check privacy policies, TOS, GDPR/CCPA compliance, etc.

> Google is now almost a convenience. If I have a coding question, I search for "turn list of tensors into tensor" or whatever but I'm really looking for SO or the pytorch documentation, and I'll ignore the geeksforgeeks and other seo spam that finds it's way it. It's almost like google is a statistical "portal" page,

I like engines like Neeva and Kagi that allow customizing SERPs by demoting irrelevant results; I demote crap like GFG, w3schools, tutorialspoint, dev(.)to, etc. and promote official documentation. Alternatively, you can use an adblocker to block results matching a pattern: https://reddit.com/hgqi5o