News for hackers. Stuff that matters.

A comment on We can do better than DuckDuckGo. Parent comment

Re:We can do better than DuckDuckGo by bscphil on Tuesday November 17, 2020 @05:41PM (#25130897)
> If I want my blog to show up on your search engine, do I have to get it linked by one of those sites, or can I register with you? Will I be tier 1, or

I think what I'd say in defense is that we've misunderstood what search engines are useful for. They're really bad at helping us discover new things. Your blog might be awesome, but it's not going to be easy for a search engine to tell that it's awesome. It's going to have to compete with other blogs that also want views, some of whom are going to be better than yours at SEO, and so on.

What a search engine might be able to tell is that it's useful. Because what search engines are at least potentially good at is answering questions. You do that by having a list of known good sites to answer specific types of questions, and looking at the sites they link to. It's when you try to do both (index everything on the web and provide accurate answers to specific questions) that you end up failing to do either. For example this is the #2 result for "python f strings" on DDG[1]. It's total garbage, and, quoting the blog, "we can do better". (This result is also on page 1 for the same query on Google.)

What I believe ddevault is suggesting is that we make a search engine that does the only thing search engines are really good at, answering questions. You throw away the idea of indexing everything on the web, and therefore the possibility of "discovery". What that means is that in 2020 you need some other mechanism for discovering new sites, bloggers, and so on. Fortunately we do have some alternatives in that space.

To be clear, I don't know if I 100% buy this argument, but I think it's the general idea behind what's being suggested in this blog post.

[1] https://careerkarma.com/blog/python-f-string/

[ Parent | on HN ]
Re:We can do better than DuckDuckGo by mxcrossb on Tuesday November 17, 2020 @07:12PM (#25131616)
As an experiment, I searched “tech news aggregator” on both google and DDG. Neither listed Hacker News. Instead, apart from a few actual sites, most of the links were articles saying “top ten tech news sites” or links to quora q&a threads.

It definitely seems that search engines can’t find new websites for people. Now they are just aggregating Q&A.

[ Parent | on HN ]
Re:We can do better than DuckDuckGo by bscphil on Tuesday November 17, 2020 @07:42PM (#25131814)
Yeah - this is sort of my thinking. For better or worse if you enter "discovery" terms, what you will get back is not the results of a "discovery" search, but rather an answer to the question "what are some websites that will help me discover X".
[ Parent | on HN ]
Re:We can do better than DuckDuckGo by chris_f on Tuesday November 17, 2020 @07:53PM (#25131917)
> For example this is the #2 result for "python f strings" on DDG[1]. It's total garbage, and, quoting the blog, "we can do better". (This result is also on page 1 for the same query on Google.)

There are other options that may be better, but in general, very few people are looking for them.

Here's your "python f strings" query on Runnaroo:

https://www.runnaroo.com/search?term=python+f+strings

I'm the creator of Runnaroo.

[ Parent | on HN ]
Re:We can do better than DuckDuckGo by edgyquant on Tuesday November 17, 2020 @10:36PM (#25132921)
I tried this out real quick just to see for myself. First it was slow for me, if I wasn’t wanting to try it out I may have closed the tab, but I did a search for a question I recently googled (that was an example of an annoying google search.) I searched “Python arcade sprite opacity“ and while the first three results were the same as google the fourth was a github link to the project itself and brought me to the answer (which wasn’t on page 1 of google for me, although it was on page two.)

So you need to speed it up, but it does look good for searching documentation at least.

[ Parent | on HN ]
Re:We can do better than DuckDuckGo by jrochkind1 on Tuesday November 17, 2020 @06:37PM (#25131365)
> You do that by having a list of known good sites to answer specific types of questions, and looking at the sites they link to.

I mean, that's basically the core of original Google pagerank, right? A "good" site linking to another site is what makes that other site some amount of "good" too, links from better sites carry more 'juice'. "good" is of course not just binary, but a quantitative weight.

I don't know to what extent that's still at the core of their relevancy rankings. I don't know how all those annoying spammy recipe blogs or content farms get to the top of the results either. I don't think it's because Google's engineers believe they are "good" results.

Relevancy ranking on web search is clearly a hard problem, mainly because so many authors are trying to game it, it's a feedback loop.

If Google can only do as google does despite pouring a whole lot of money into it, I don't see a reason to bet that better will be what's basically an over-simplified description of how Google started out doing it (and then evolved it because it wasn't good enough).

[ Parent | on HN ]
Re:We can do better than DuckDuckGo by JProthero on Wednesday November 18, 2020 @01:17AM (#25133729)
That was exactly my thought on reading that part of the article: that the author was describing Pagerank, but substituting Pagerank's design objective of algorithmically outsourcing quality judgments to the global community of website publishers with the programmer's prejudices instead.

I hope the project of improving on DuckDuckGo is successful though, and some of the other proposals in the article sound promising to me.

In the future I would like to see an open source search engine 'paid for' using some combination of homomorphic encryption, blockchain and Tor-style technologies to trade bandwidth and processing power with its userbase in exchange for search results and other services, but I don't have the expertise to assess how feasible that might be.

[ Parent | on HN ]
Re:We can do better than DuckDuckGo by pksebben on Thursday November 19, 2020 @09:04AM (#25149537)
the money really is the crux of the issue, too. if you take as a given that consumers won't pay for a service they can get for free, then you're kind of in a bind creating an unopinionated search - if you're truly objective, and reflecting the underlying value of each link faithfully, who do you charge?
[ Parent | on HN ]