Create 3-5 different search algorithms, with names and understandable characteristics, then let me choose which one when searching. You may have some algorithm that prioritizes recency, or rapid changes, or more interaction, and others that prioritize longevity, citations/links, others that prioritize that type of information - prioritize regional or noteworthy events over historical info, etc. Have some algorithms that penalize spam/linkfarms more heavily than others. The git/so spam farms that come up when I search for an error message are simply useless, and would love ways to switch away and try other algorithms, without necessarily having to switch entire search systems.
Have algorithms that explicitly take in to account the search/click behaviour of others. Let me choose to have the search behaviour of friends/colleagues/famous-folks explicitly included or excluded in search results ranking.
Let people easily switch between multiple algorithms, and let 1000 gardens bloom (or something like that).
Also, determining better ways to 'search' non-text stuff. If virtual worlds become a thing, finding ways to 'search' those (whatever that might mean) will be big.
Image search still has a long way to go, IME.
The way it's likely seen by many tech companies is that the more decisions you have to offload to the user, the less advanced your system is. An old car with a lot of knobs and dials and levers is not more advanced than a self-driving car with a single control, select destination. Some people will still prefer the old car, but it will be a niche market.
"You're viewing through FOOBAR lens. Click here to view results from another lens".
Doesn't have to be that complicated.
You can add three more buttons on top of the tabs that are already there, but would their value for more advanced users be worth the extra friction/confusion for average users? And how would it scale if someone wants a fourth, fifth, or tenth "lens"?
Try images.yandex.com: it's vastly better than most others. It's not too good at identifying specific people but it will find prior exact uses of an image (like TinEye) but is especially very good at abstracting what's the abstract subject, color, pose, setting etc in a photo and show you similar.
The question is if such an approach would lure a lot of people away, I could see an entry angle might be a front end to google with an app that does that.
We now have a couple digital generations of folks who've grown up with 'web', and we've generated billions of 'documents'.
TIME ACCURATE QUERIES need to be mainstream.
"Let me search for FOOBAR and get results that would have been accurate/returned for Dec 11, 2006. Now search for FOOBAR and get results from Dec 11, 2014", etc.
However as for Google, I feel like even merely surfacing a range of relevant filter options can help their search engine, since it's what informed users are already doing by structuring a query when the first one produces crappy results. E.g. These results mostly contain links to Pinterest.com, would you like to repeat the search with these omitted? (or even just being able to collapse these site-by-site.)
Additionally I think modern search could be broken down into a range of sub-tools that exist beyond google or even online. One such example is searching our web history, perhaps browsers could produce a privacy-preserving way of storing information about each webpage and its images that the user can then search later (by either terms or images.)
IMO, all startups have a sweet-spot where they've figured out what the market needs, and they can be profitable, but they haven't yet sucked that bone dry to where they're forced to extend more tentacles to grow their revenue. For Google, for me, that was about the 2007-2012 time frame.
Any other competitor would have the same financial pressures eventually. Long term, it's impossible to "not be evil".
> They stop using it if the results are poor.
These are not laws of nature. There are so many reasons why most people would still use $X even if the performance of $X was underwhelming.
At least for me, I find myself adding “site:Reddit.com” all the time I need a good answer and not just a SEO-tricked ad page. Vector Search could be a technical solution for this. Although it’s fairly new and there is still a lot of research to do.
Seems like with enough actual humans I should never have to see another geeksforgeeks page come up in my search results.
Here's another recent Show HN that demoed a new search engine:
Show HN: Goopt – Search Engine for a Procedural Simulation of the Web with GPT-3
https://news.ycombinator.com/item?id=30443738
I know this was offered as a bit of joke. But it prompted a really thought-provoking discussion and seemed to me to suggest a genuine paradigm shift in search or information organization. It is a novel way to extract a useful signal from all the noise. It brought to my mind "Dr. Know" from Spielberg's AI movie:
https://www.youtube.com/watch?v=x0QkgAuEPbk
Will the signal be a good one? It will need to get better and inevitably will. The thing that worries me: will it even matter?
For example, someone comes up with a 'Finder' thing that's barely a comprehensive directory of the internet, and that, along with 'site search' aka for FB, Amazon, does better for the use case.
Most people are not 'searching' the web, they're often typing URLs or searchers for things they want answers to.
When I type a technical thing I'm really just searching 'stack exchange'.
On mobile, we use apps, which are different than pages.
Apple could come up with something that renders Google less than necessary.
Google will not be replaced directly.
Not really, because Google can do algorithms. For a new competitor to really succeed they have to start with something Google can't do. Perhaps:
- device/platform related. Google owns chrome & android but a competitor could emerge somewhere out of their reach, like only on iPhone, only on airpods, alexa, oculus, etc.
- being worthless to 99% of the market to make it 10X better to a 1% audience (e.g. developers, nurses, or even more niche like college athletes, jewish moms, etc)
- privacy/ad related
Delicious sort of tried it but never executed well on the search side.
In a way it's a personalized PageRank algo with users linking to pages they like and to other users they follow. This always made more sense to me than a global authority ranking.
When looking for a babysitter you want recommendations from friends you trust, not some global ranking of popular babysitters.
Most long tail queries work well enough with Google and indeed most of the benefit of this approach would be in the head where ranking matters more. Furthermore nothing prohibits users from liking domains, not just individual URLs.
To me this is going to be manual ranking, but the biggest issue with manual ranking is not that you have to do it manually, but how do you know if you can trust users who rank pages. The Wikipedia system is a good start but not enough, and the second issue is how do you prevent its business model to kill its quality
Do the algorithms weight links based on visibility or relevance? A link at the top of a sorted content site HN should probably be worth more than one at the bottom, and one buried in invisible metadata somewhere should be worth 0.
But AFAIK Google no longer uses PageRank, or at least not the same version as we know it.
One thing a competitor needs is a new and innovative technical algorithm.
Back in 1998, Google's PageRank was an innovative algorithm that calculated relevance based on counting back links instead of parsing the word counts in embedded HTML text like other search engines. This created a noticeable improvement in quality of results.
Nobody seems to have The Next Big Idea for a better search engine yet. Somebody did a Shown HN of a new search engine based on whitelisted curated domains such as reddit discussions. But there are many technical problems with that (e.g. Goodhart's Law & Hawthorne Effect creates bad feedback loop of gaming the reddit discussions which then poisons the search engine.)
Another technical idea of crowdsourced decentralized search index creates a very slow query engine which is a hard sell when web surfers are used to Google results appearing in less than 1 second.
DDG's idea of "privacy" is interesting, but being (mostly) based on Microsoft Bing's search engine doesn't actually create a quantum leap in better search results.
What's the next breakthrough idea that extracts the good signal from all the noise of a trillion web pages?