News for hackers. Stuff that matters.
Re:How Google uses blacklists, algorithm tweaks and contract... by smashingfiasco on Friday November 15, 2019 @10:52AM (#21545792)
I’d love to see search decentralized and have been toying with this idea for a while.

The last concept I hacked together was a custom search plugin for Grav and a command line util to use for querying.

It goes like this.

Use the command line util to search a term. The command line util run that term against the search engine _inside_ of the websites CMS itself. You essentially have a list of sites related to a topic that you chose to execute the query against.

I got this working against some sites and the proof is there. But it’s obviously highly inefficient and I haven’t figured that out yet. :-/

Re:How Google uses blacklists, algorithm tweaks and contract... by mgreenleaf on Friday November 15, 2019 @11:25AM (#21546142)
One alternative is to run a webcrawler that stores the index in a series of SQLite database files, either by topic or by site, or any other criteria. Then users could download sets of those SQLite databases and run queries on them. Not really completely distributed, but hides some information in the noise of "search sets" and mirrors, and individual queries are run on local. You could mirror the main repository and just run searches on your own server/local. You could also swap the database files with P2P, etc.
Re:How Google uses blacklists, algorithm tweaks and contract... by buboard on Friday November 15, 2019 @10:56AM (#21545828)
Could be some kind of giant computation machine running as a public blockchain. Websites could provide their own keywords and machine would be used to rank them for queries.
Re:How Google uses blacklists, algorithm tweaks and contract... by lgats on Friday November 15, 2019 @11:05AM (#21545936)
Check out YaCy