News for hackers. Stuff that matters.
Re:Adieu google by DanBlake on Sunday December 06, 2009 @12:30AM (#979216)
Am I missing something? Why using duckduckgo? The only mention I have seen of it is a paid ad they bought on reddit. Is there any reason they are better than the plethora of existing alternatives? Do they even maintain their own index, or are they a scraper?
Re:Adieu google by mlinsey on Sunday December 06, 2009 @02:46AM (#979310)
I'm pretty sure they use Yahoo BOSS, plus some in-house stuff that merges info from trusted sites like Wikipedia. Yahoo's results are pretty good, though I hope that BOSS-based sites survive whatever the fallout of the Yahoo-MS search deal ends up being.

Personally I've found that the extra features on top of Yahoo's results that DDG gives aren't usually very useful, but it's nice and fast with a simple, usable interface. For the few weeks I had it as my Firefox search box default, the only thing I felt was missing was how Google includes results from Google News and Image Search right on your search results page.

Also, the founder of DDG posts here on HN, which is one reason why you'll hear about it around here more often than other places.

Curious, what do you think the plethora of other alternatives are? I've been pretty unimpressed by the small search engines trying to use their own indexes (Cuill) and even less impressed by the "human-powered" engines (Mahalo). I haven't seen anyone else really come close to Google, Yahoo, Bing, and Ask.

Re:Adieu google by epi0Bauqu on Sunday December 06, 2009 @08:03AM (#979479)
DDG founder here. We started out solely with our own crawler, which we still use for various things, e.g. finding and removing 50 million+ parked/spam domains. But we do use Yahoo BOSS as well.

However, we do not use BOSS straight up. We re-rank, omit, edit, merge, etc. in ways (I think and hope) yield much more relevant results, especially for more specific queries.

The Yahoo/MS search deal shouldn't effect us. We already have Bing in place as a backup and could easily switch to Ask's feed as well. We treat them as somewhat of a commodity, i.e. an output of the Web link graph, with our ranking stuff on top doing the important "last mile" work.

Thx for your feedback about news & images.

Re:Adieu google by anonymousDan on Sunday December 06, 2009 @08:19AM (#979488)
Is that web link graph freely available or do you have to pay for it? I've been trying to get one for Google Scholar but can't figure out how.
Re:Adieu google by epi0Bauqu on Sunday December 06, 2009 @08:18AM (#979487)
Ultimately the reason you should use it is because it gets you the information you want faster and with less mental effort. That's the primary goal. I'd encourage you (or anyone!) to try it for a week and then let me know how it goes with this goal in mind.

We've tried to achieve that goal via UI and a number of features that may or may not appeal to you individually. We also try to concentrate on features that the major search engines won't do because of their business position. A lot of those features are listed here: http://duckduckgo.com/about.html

What's not listed there that I think appeals to this crowd in particular is the background relevancy stuff I mentioned in my other comment and answer to your other question: http://news.ycombinator.com/item?id=979479

I think what also appeals to people like me (and by extension, I assume a lot of HN types) is keyboard shortcuts, aggressive filtering of parked/spam domains, and disambiguation.

We do have our own index and use it to improve relevancy as well as to add our own Zero-click Info and links to search result pages. Search any major topic and you should see a red box on top with a quick snippet and related topics. We also use it to build our own category and disambiguation pages, e.g. http://duckduckgo.com/c/Social_bookmarking

Our most recent major feature launch was Duck It!, a search view accessible from our homepage that features more topic summaries, e.g. http://duckduckgo.com/?q=tinychat&v=d. HN feedback link here: http://news.ycombinator.com/item?id=919505

Of course, we'd welcome suggestions for additional features/improvements!

Re:Adieu google by joeyh on Sunday December 06, 2009 @12:36PM (#979854)
Hey, while you're here, one thing about Duck Duck Go that concerns me is that it can be killed on a per-IP basis by locking on "safe" search.

If I accidentially do this at home, or if I find that I have to use tor too often while in coffee shops to ensure I see all search results, (or if I use tor and some idiot has forced it on for those IPs), that could be a problem.

Also, I wonder how hard it would be to spoof packets to kill IPs one doesn't even have access too? Do you send a nonce back to the user and check it before completing the banning process?

Re:Adieu google by epi0Bauqu on Sunday December 06, 2009 @04:57PM (#980219)
The short answer is when we start running into collisions we'll change the parameters of this service, or perhaps sooner. That is, I'm fully aware of these limitations/issues and am actively monitoring them.

I wanted to try the extreme case (at least for a time) to see if a lot of schools and businesses that use one static IP would find the process super simple and just use it. After all, they could set all their computers to safe search in one click. This experiment is ongoing, but it will probably evolve into a program where we have more of a back and forth opt-in process.

I hope this addresses your concern!

Re:Adieu google by jules on Sunday December 06, 2009 @12:03PM (#979781)
Suggestion: Improve specific searches. Broad searches work very well (try "partial evaluation") but various specific searches (try "read file .net" and compare with google; their output isn't stunning but a lot better) don't.
Re:Adieu google by epi0Bauqu on Sunday December 06, 2009 @04:53PM (#980214)
Thx. Our specific searches certainly vary in quality, which is something we're actively working on. Actually, I think they are more of a curve where they are really good for broad and long (very specific) but are average on the middle range. We do semantic stuff on longer queries that make the results better. But in any case, you should be generally getting different results for specific searches. Sometimes these differences will be spot on; other times, not so much. I'll take a look at 'read file .net' now--please send other examples our way: [email protected].
Re:Adieu google by warp on Sunday December 06, 2009 @02:43AM (#979307)
I reported an issue to them via e-mail on saturday. Got a response on sunday, I added some details, and it was fixed a few hours after their first response.

they're still small, they still listen to feedback.

Re:Adieu google by stcredzero on Sunday December 06, 2009 @04:10AM (#979352)
How would one combine everything good about something like "Search Engine as a Startup" with everything good about "A Company like Google?"

Could we have an ad & donation supported open source search engine running on top of AWS or Google Apps? (Yes, the 2nd one would be ironic, if even allowed to exist by Google.) Perhaps this would be a way to combine the responsiveness of Duck Duck Go or the early Google with the infrastructure and computational horsepower of Google/Bing ca. 2009?