News for hackers. Stuff that matters.
Re:Ecosia – the search engine that plants trees by greglindahl on Wednesday March 06, 2019 @09:44PM (#19325179)
Pretty much every search box you see on the Internet is a wrapper around Google or bing, with Yandex as a distant 3rd and a few country-specific engines in China, Russia, Korea, etc.
Re:Ecosia – the search engine that plants trees by viraptor on Wednesday March 06, 2019 @10:28PM (#19325322)
There's also yacy which is the distributed search effort. Tiny by size, but worth mentioning.
Re:Ecosia – the search engine that plants trees by thekyle on Wednesday March 06, 2019 @10:54PM (#19325400)
I really like YaCy (I run a node), but a big problem with it that will prevent it from ever competing with Google/Bing is the lack of a verifiable link graph for ranking results. Right now it basically just uses full text search and a local link graph which is problematic.
Re:Ecosia – the search engine that plants trees by misterman0 on Thursday March 07, 2019 @01:10AM (#19325913)
>> the lack of a verifiable link graph

I hadn't heard about YaCy even though I'm in their game. I too run a search engine that doesn't use a link graph for scoring relevance. My bet is on semantic relevance over PageRank, because I think our understanding of language models have become so sophisticated we can start relying on them solely.

Also, it's harder I think to game a language model when compared to a page rank model.

What do you think is lacking from non-link graph search engines?

EDIT: maybe I misunderstood b/c it says here YaCy _has_ a link graph: https://yacy.net/en/API.html

Re:Ecosia – the search engine that plants trees by thekyle on Thursday March 07, 2019 @04:08AM (#19326601)
> maybe I misunderstood b/c it says here YaCy _has_ a link graph: https://yacy.net/en/API.html

That link graph being referenced is the local nodes one, not a global one. So it only yields links that have been discovered by the node you are sending the request to.

You could send a request to each node individually and aggregate the results, however, you have no way to verify the nodes are being honest without actually crawling those pages yourself.

If I may ask, what is the project that you are working on?

Re:Ecosia – the search engine that plants trees by misterman0 on Thursday March 07, 2019 @12:02PM (#19329694)
Ic. Thx!

I'm working on a project similar to the YaCy one. The main difference is that I do not want or even require that each node trust each other. I let the user decide who they trust.

https://github.com/kreeben/resin

Re:Ecosia – the search engine that plants trees by misterman0 on Thursday March 07, 2019 @12:15PM (#19329842)
>> it only yields links that have been discovered by the node you are sending the request to

Wait! YaCy is a little bit hard to understand. I don't mean that in any way other than in an extremely encouraging way.

What do you mean exactly? A YaCy node will interrogate the "swarm" of data and that interrogation might yield more links compared to if you interrogate another node?

What's that all about?

Re:Ecosia – the search engine that plants trees by elcomet on Thursday March 07, 2019 @04:08AM (#19326603)
What is your search engine?