News for hackers. Stuff that matters.
Re:The internet is an SEO landfill by swayvil on Sunday June 23, 2019 @02:40PM (#20257752)
Agreed. Much like a spam-filter. Seems straightforward enough. What am I missing?
Re:The internet is an SEO landfill by Gene_Parmesan on Sunday June 23, 2019 @02:55PM (#20257866)
What you're missing is the fact that larch search engine companies have no profit motive to do this. They get the vast amount of their money from commercial ad sales -- ads for commercial results. Why would they purposefully castrate their highest paying customers?

And if you think the answer is to create and host a new, independent engine, you will be hard pressed to develop a good one, and to find the money to keep it running.

Re:The internet is an SEO landfill by swayvil on Sunday June 23, 2019 @03:27PM (#20258122)
user smt88 offered a fabulous idea :

>Wikipedia is basically an information search engine without commerce or social media. The problem is that your results page is the article itself, and you need to scroll to the bottom to get to an external site.

>It's theoretically possible to change the UI or analyze Wikipedia to make a pretty solid search engine powered by millions of person-years of curation.

Re:The internet is an SEO landfill by jasode on Sunday June 23, 2019 @04:41PM (#20258524)
>user smt88 offered a fabulous idea : >Wikipedia is basically an information search engine without commerce or social media. The problem is that your results page is the article itself, and you need to scroll to the bottom to get to an external site.

Using wikipedia as a data source for a "search engine" does not fulfill gp's (jacquesm) request of "One nice solution would be a search engine that throws out anything related to commerce or linking to commerce."

For example, go to the wikipedia page for the film "Groundhog Day" and look at the external links: https://en.wikipedia.org/wiki/Groundhog_Day#External_links

- one of the external links points to IMDB.com -- a commercial website owned by Amazon.

- another link points to Punxsatawney Groundhog Club website that's advertising a $30 beer tasting event (Hogtoberfest).

- another link points to a souvenir shop

To continue the example of commercial links, a lot of math articles on wikipedia point to MathWorld which has advertising for the commercial Wolfram Mathematica software package.

IMO, I don't believe a search engine based on wikipedia's list of external sites (even with ignoring the commercial links) is going to be useful for the mainstream audience. I probably do 50+ searches on google every single day and maybe 1% might be answerable from a "wikipedia search". For example, I needed a DIY answer to disassemble a Moen faucet and Wikipedia has zero articles with those instructions. In contrast, the Google (and Youtube) search results has the information I was looking for even though it has the unwanted ads and "content marketing" blogs from plumbing brands.

Re:The internet is an SEO landfill by swayvil on Sunday June 23, 2019 @02:59PM (#20257915)
How about a layer over Google? Take Google's search results and filter them. Any obvious barriers to that?
Re:The internet is an SEO landfill by jasode on Sunday June 23, 2019 @03:22PM (#20258087)
>Any obvious barriers to that?

Rate limits.[0]

One can't really build a robust and comprehensive (server-side) search engine on top of Google's search API. In other words, Google Inc isn't going to provide an API so powerful that one can build an "alternative to Google" with it.

On the other hand, if you're talking about a "client-side" filter instead of a new search engine, (e.g. a client-side webscraper that flips through the Google results pages), you'd run into multiple problems. E.g. random CAPTCHAS screwing up the automation script, and the difficulty of creating a "rules engine" (e.g. machine learning) on your local computer that recognizes blog articles with a commercial slant.

[0] https://www.google.com/search?q=google+search+api+rate+limit

Re:The internet is an SEO landfill by swayvil on Sunday June 23, 2019 @03:30PM (#20258135)
see user smt88's idea above.
Re:The internet is an SEO landfill by tyingq on Sunday June 23, 2019 @03:19PM (#20258064)
They are pretty aggressive about detecting that and throwing up captchas.
Re:The internet is an SEO landfill by swayvil on Sunday June 23, 2019 @03:30PM (#20258136)
see user smt88's idea above.