It's telling, that a Google search scoped to a particular web site (xyz site:website.com) almost always produces better results than that web site's actual built-in search engine. If a web site that knows in detail its own content, its own users, and its own domain, still can't make their built-in search better, how does anyone else have a chance?
Google was epic when it started but quickly ended up indexing it self in a near closed loop. Everyone else was not important, it was all about the picture in front of them that was really just a mirror. Everyone stopped making home pages, forums and weblogs because it just doesn't make sense to do it.
Shit, people looking to buy something talking among their own before doing the purchase? KILL IT WITH FIRE!
that becomes a big can of worms very quickly, and starts to put into perspective some of googles other most successful verticals: NLP, Translation, Maps ("Near me"). Of course a lot of the verticals are also a generic means to get value out of all the server resources and experts required to operate the engine and then additional related branches to those verticals (email -> drive -> docs)
You only search whitelisted sites, which removes a lot of the SEO and blogspam from the results, and makes the index is wildly smaller. Search is free, but users can also pay a cheap subscription for the ability to flag sites as spammy, submit sites they think are missing, and submit queries the site has trouble with. Someone manually goes through the top N most-repeated submissions each day (early on this is all of them) and determines whether to tweak the whitelist.
Because your initial audience has to be people who are very enthusiastic about better search results, you should identify what they're interested in and focus your efforts on initial whitelists to things they care about. That might be technical questions (MDN StackOverflow, every random framework's docs site), or maybe whatever it is journalists would want to have access to (I'm not qualified to answer).
I don't think VC-encouraged hypergrowth is possible with the above strategy, but you could probably get to ramen profitable in a reasonable time frame, with slow growth ever after.
The fate of runnaroo seems to provide evidence to the contrary.
I understand your intent with that domain-specific example but surprisingly, that reasonable intuition sometimes doesn't work...
E.g. for Amazon product search, I often deliberately use Google's engine to find the Amazon page because Amazon's own search can't do it. See example screenshots in my previous comment about that: https://news.ycombinator.com/item?id=27539199
Curious what you think these are. I prefer Bing's UI to Google's but aside from that Bing is (IMO) a dumpster fire that quite literally makes me feel negligent (towards myself) when I use it - am I missing something?
It would just take at astronomical amount of capital to challenge Google at generic Internet search, and the few organizations with that capital are pretty clear they don't think it's a good bet. For example, Amazon is quite content just being the place you want to go to first for the things you want to buy, as that is obviously the most lucrative area.
Even in niche areas, I think it's still an enormous task to challenge Google. Despite the fact that 90% of my searches for "WTF does this stacktrace mean?" end up on StackOverflow, I invariably start my search on Google because every now and then it gives me useful non-StackOverflow tidbits, and I know I can always drill in more with specific tags on StackOverflow later.