Show HN: Open-source search engine with 2bn-page index

Re:Show HN: Open-source search engine with 2bn-page index by laurent123456 on Tuesday September 13, 2016 @06:21AM (#12486794)

Yes I guess filtering them out would at least make the website SFW, and it would make it easier to show it to people. The issue seems to happen mainly with common words (which results also appear to be polluted with heavily SEO-ed websites).

I've also searched for less generic things like "xperia z5" and the results looked good.

[ Parent | on HN ]

Re:Show HN: Open-source search engine with 2bn-page index by deusu on Tuesday September 13, 2016 @12:47PM (#12490022)

I have the filter implemented now. It's not perfect yet, but it already filters out a lot of the NSFW stuff. Unless you explicitly search for it.

I'm gonna further improve this over the next days. Right now it's just a quick'n dirty hack. :)

[ Parent | on HN ]

Re:Show HN: Open-source search engine with 2bn-page index by rasengan on Tuesday September 13, 2016 @01:41PM (#12490532)

It would be a interesting data point to see how many of the 2b pages indexed are adult. lol.

[ Parent | on HN ]

Re:Show HN: Open-source search engine with 2bn-page index by deusu on Tuesday September 13, 2016 @01:56PM (#12490684)

I don't know. But I do know that the end-of-year statistics from search-engines about what people searched for, are complete BS. I have such a list for the German DeuSu page:

https://deusu.de/blog/2015-12-03-alle_jahre_wieder_wonach_de...

Warning! This is definitely NSFW! :)

[ Parent | on HN ]

Re:Show HN: Open-source search engine with 2bn-page index by wila on Tuesday September 13, 2016 @06:10PM (#12492685)

LOL, seems that a big part of your users is searching for adult related things.

So if you filter out all the adult stuff you might make those users unhappy. Perhaps make it configurable?

Eg. add a checkbox for NSFW results or something better.

[ Parent | on HN ]