News for hackers. Stuff that matters.
Re:How is search so bad? A case study by reaperducer on Sunday January 19, 2020 @03:37PM (#22093883)
Unfortunately, a "full" text crawl of the internets is a YUUUGE amount of data to manage

Maybe instead of a problem, there is an opportunity here.

Back before Google ate the intarwebs, there used to be niche search engines. Perhaps that is an idea whose time has come again.

For example, if I want information from a government source, I use a search engine that specializes in crawling only government web sites.

If I want information about Berlin, I use a search engine that only crawls web sites with information about Berlin, or that are located in Berlin.

If I want information about health, I use a search engine that only crawls medical web sites.

Each topic is still a wealth of information, but siloed enough that the amount of data could be manageable to a small or medium-sized company. And the market would keep the niches from getting so small that they become useful. A search engine dedicated to Hello Kitty lanyards isn't going to monetize.

Re:How is search so bad? A case study by LargoLasskhyfv on Sunday January 19, 2020 @11:22PM (#22096273)
I´d be happy with something like Searx [1,2,3]

[1] https://en.wikipedia.org/wiki/Searx [2] https://asciimoo.github.io/searx/ [3] https://stats.searx.xyz/

featuring the semantic map of [4] https://swisscows.ch/

incorporating [5] https://curlie.org/ and Wikipedia and something like Yelp/YellowPages embedded in Open Streetmaps for businesses and points of interest, with a no frills interface showing the history (via timeslide?) of edits.

Bang! Done!

Re:How is search so bad? A case study by zozbot234 on Sunday January 19, 2020 @03:46PM (#22093939)
That's the problem that web directories solve. It's not that you're wrong, it's just largely orthogonal to the problem that you'd need a large crawl of the internets for, i.e. spotting sites about X niche that you wouldn't find even from other directly-related sites, and that are too obscure, new, etc. to be linked in any web directory.
Re:How is search so bad? A case study by reaperducer on Sunday January 19, 2020 @04:05PM (#22094058)
That's the problem that web directories solve

Not really. A web directory is a directory of web sites. I can't search a web directory for content within the web sites, which is what a niche search engine would do.

Re:How is search so bad? A case study by Beldin on Sunday January 19, 2020 @06:27PM (#22094891)
On the other hand, the niche search engine depends upon having such a web directory (the list of sites to index).
Re:How is search so bad? A case study by teddyh on Sunday January 19, 2020 @06:39PM (#22094965)
Re:How is search so bad? A case study by HomeDeLaPot on Sunday January 19, 2020 @04:11PM (#22094106)
Don't forget the search engine search engine!