I never, ever, for any reason, want to see results from the Daily Mail. I don’t even care if it’s the most relevant content: I just never want to see that site in my results. I know I can block it in hosts or whatever, but that’s not what I want. I don’t even want to see the link.
I wish DDG would give me this option.
And if you think the answer is to create and host a new, independent engine, you will be hard pressed to develop a good one, and to find the money to keep it running.
>Wikipedia is basically an information search engine without commerce or social media. The problem is that your results page is the article itself, and you need to scroll to the bottom to get to an external site.
>It's theoretically possible to change the UI or analyze Wikipedia to make a pretty solid search engine powered by millions of person-years of curation.
Using wikipedia as a data source for a "search engine" does not fulfill gp's (jacquesm) request of "One nice solution would be a search engine that throws out anything related to commerce or linking to commerce."
For example, go to the wikipedia page for the film "Groundhog Day" and look at the external links: https://en.wikipedia.org/wiki/Groundhog_Day#External_links
- one of the external links points to IMDB.com -- a commercial website owned by Amazon.
- another link points to Punxsatawney Groundhog Club website that's advertising a $30 beer tasting event (Hogtoberfest).
- another link points to a souvenir shop
To continue the example of commercial links, a lot of math articles on wikipedia point to MathWorld which has advertising for the commercial Wolfram Mathematica software package.
IMO, I don't believe a search engine based on wikipedia's list of external sites (even with ignoring the commercial links) is going to be useful for the mainstream audience. I probably do 50+ searches on google every single day and maybe 1% might be answerable from a "wikipedia search". For example, I needed a DIY answer to disassemble a Moen faucet and Wikipedia has zero articles with those instructions. In contrast, the Google (and Youtube) search results has the information I was looking for even though it has the unwanted ads and "content marketing" blogs from plumbing brands.
Rate limits.[0]
One can't really build a robust and comprehensive (server-side) search engine on top of Google's search API. In other words, Google Inc isn't going to provide an API so powerful that one can build an "alternative to Google" with it.
On the other hand, if you're talking about a "client-side" filter instead of a new search engine, (e.g. a client-side webscraper that flips through the Google results pages), you'd run into multiple problems. E.g. random CAPTCHAS screwing up the automation script, and the difficulty of creating a "rules engine" (e.g. machine learning) on your local computer that recognizes blog articles with a commercial slant.
[0] https://www.google.com/search?q=google+search+api+rate+limit