As your search engine tips in to popularity all sites on it are overrun with spammy SEO content as marketers search for the next way to get more eyeballs on their ads.
Also, I can see the value of Stack Overflow to Google but what value does Reddit add? Isn't most of the content on there disposable?
So far I've found two places with actual useful info: YouTube and reddit. YouTube has a reasonably working search engine so I just search directly there. Reddit doesn't so I end up adding site:reddit.com to all my Google searches.
Maybe there's some better forums than reddit that I haven't found yet, but this mirrors my experience from a couple of months ago when searching for new headphones, and a few months before that when researching a new laptop. All the interesting discussion is happening on reddit, and all the Google results are disguised sales pitches ("thanks for reading! Now click my affiliate link"). I'm close to automatically just adding reddit to all my Google searches when I'm researching something new.
Try an experiment. Think of a new product you might want to purchase, say, scuba diving masks . Search "best scuba diving masks" on Google. Now try adding "site: reddit.com" and check again. Which search do you think gives you more honest, useful info from people who are genuinely into scuba diving and not just trying to make a quick buck with a scuba blog?
Note: I don't know anything about diving and I haven't tested these terms, I'm just very sure it will be true for any random product you pick.
It's not that the comments are plain ads (these will be easy for the reader to discern), it's dishonest product reviews. It's an unsolved problem in e-commerce websites and even in real life.
Even if you succeed in securing an island of quality content: How big is your audience? I don't remember the exact quote but somebody said that television is the way it is because that's what people want.
Yahoo should have chosen to become a media company. Now they would have all the knowledge to mix search results in a way that is rewarded by the market. It's the academics of the early internet who want the best results. Everybody else wants to be entertained.
And that's just the first sentence!
It's also just kind of frustrating because it's the kind of response that comes from a place of refusing to engage with a hypothetical exercise. Failing to meet these exercises with the spirit of open-ended curiosity, as they loosen suspension of disbelief just enough to make it possible to canvas the space of strategic possibilities, is a misunderstanding of the exercise, and it's just the kind of sleepwalking response that turns potentially fruitful conversations into dreary dead ends.
But theoretically speaking Reddit for example can say "We don't like Google" and change their robots.txt rules in order to block Google bots.
1. You buy StackOverflow for $2 billion and Reddit for $10 billion.
2. You block Google from indexing the sites.
3. You start a new search engine that only searches StackOverflow and Reddit.
4. As the new search engine gains traction, you invite other high quality sites to join your vision and search engine. One requirement is they must block Google. You can guarantee them decent traffic because they'll only be competing against a dozen sites on your search engine, instead of millions on Google.
5. A large number of respectable sites leave Google and are only available on your search engine. Businesses start becoming eager to join your exclusive network and ask to join your mission.
6. Google is left with blog and affiliate SEO spam, and you become the hero of the search engine world.