Microsoft’s Bing uses Google search results—and denies it

Re:Microsoft’s Bing uses Google search results—and denies it by kmavm on Tuesday February 01, 2011 @09:11PM (#2168563)

Especially in the case of spelling correction, it is not so much that Google has "solved hard problems" to get the long-tail right, as that they have a monopoly on the relevant data. Unlike all their competitors, Google has 12 years of the entire history of queries and clicks to mine for signals about how to rewrite queries. Even if you have all of Google's algorithms, it is technologically impossible to build a better query rewriter, because you don't have their data. You can't buy this data short of buying Google, and if you believe Peter Norvig, it's an irreplaceable component of Google's quality advantage.

Microsoft, and any other would-be competitor, would essentially be committing suicide not to try to make up this data gap. If their toolbar is opt-in on the part of users, and you agree with me that my click history is mine to share with Microsoft if I so choose, this is helping consumers. Without some of this data, building a viable competitor to Google is impossible, and consumers do benefit from competition in web search.

Disclaimer: I work in Facebook search. Not the same thing as web search, and I don't really care whether Bing or Google "wins", though I'm temporarily rooting for Bing because as a user I want better, more competitive web search.

[ Parent | on HN ]

Re:Microsoft’s Bing uses Google search results—and denies it by l0nwlf on Wednesday February 02, 2011 @07:38AM (#2169985)

> Google has 12 years of the entire history of queries and clicks to mine for signals about how to rewrite queries

How about Yahoo ? They have a longer history I suppose. IMO it's all about quality and engineering.

More importantly why did Google survived and flourished for 12 years ?

[ Parent | on HN ]

Re:Microsoft’s Bing uses Google search results—and denies it by scrame on Wednesday February 02, 2011 @02:47PM (#2171780)

They have a longer history, but waaaay less traffic. Also, their early business model was a paid directory, not organic search results.

[ Parent | on HN ]

Re:Microsoft’s Bing uses Google search results—and denies it by billybob on Friday February 04, 2011 @04:46PM (#2181383)

They have waaay less traffic only since Google came onto the scene and stole their traffic by virtue of being better.

[ Parent | on HN ]

Re:Microsoft’s Bing uses Google search results—and denies it by Technocrat on Wednesday February 02, 2011 @04:16AM (#2169628)

Exactly. Peter Novig wrote a awesome spell correction algorithm.

But now Google doesnt really need ( Possibly then dont even use it) because of two set of queries where first user mispells and second where he corrects, gives them a StringMap that they can use to map mispelled queries to correct queries.

[ Parent | on HN ]

Re:Microsoft’s Bing uses Google search results—and denies it by andrewljohnson on Tuesday February 01, 2011 @11:02PM (#2168913)

Exactly... to beat Google, you need time and data. Or you can do what MS did to fill the gap for a while.

[ Parent | on HN ]

Re:Microsoft’s Bing uses Google search results—and denies it by kmavm on Wednesday February 02, 2011 @01:53AM (#2169385)

Actually, time won't do it either. Unless you somehow compete with Google today, the data will never come.

Google only gets the query volume it does because it is the quality leader. The query volume itself helps Google to retain its quality lead. Google likes to portray search quality as being algorithm-driven, and it is to some extent, but in the modern era quality is also about collaborative filtering with clicks. If you don't have the users, you don't see the clicks, and you can't have the quality. Web search is a natural winner-take-all monopoly, unless someone gets creative, which is what Microsoft seems to have done.

[ Parent | on HN ]

Re:Microsoft’s Bing uses Google search results—and denies it by metamatt on Wednesday February 02, 2011 @02:03PM (#2171561)

What volume do you really need, to get enough data to learn from? I'd think that 1% of Google traffic would still be a pretty big firehose to feed whatever learning algorithm you need to feed.

Don't Google, Facebook, et al run a lot of experiments for new projects on a subset of users/queries that's far smaller than 1% of traffic, and still yields very useful results?

[ Parent | on HN ]

Re:Microsoft’s Bing uses Google search results—and denies it by kmavm on Wednesday February 02, 2011 @05:03PM (#2172378)

In the case of spelling correction and query expansion, every little bit helps. Suppose you want to learn that people typing [mazad] mean [mazda]. (This is kind of a silly example, as dictionary- and edit-distance-based techniques can do corrections like this. So bear with me.) The event you need to catch is:

1. User mistakenly types a query [mazad], meaning [mazda]. (Probably less than 1% of total queries for Mazda, which is an infinitesimally tiny fraction of the total queries in your system.)

2. The user gets garbage results, and the user realizes their mistake and fixes it, rather than giving up in frustration. This is probably rather rare too, though

3. The user clicks through something that ranked highly for Mazda, and stays there long enough that your system thinks it is a "long click" that probably satisfied the user.

The golden datum here is literally a one in very-many-thousands-of-sessions event, and you need to catch a statistically meaningful number of them for every misspelling (or synonym, or whatever you're trying to learn from this data) you'd like to have your system learn. To have good coverage of the English language, we're talking about many billions of search sessions.

A previous commenter pointed out that Yahoo! probably has enough data; I bet they're right. I don't know if Yahoo! and Bing's technology partnership included access to such data.

[ Parent | on HN ]

Re:Microsoft’s Bing uses Google search results—and denies it by bad_user on Wednesday February 02, 2011 @11:38AM (#2170845)

      collaborative filtering with clicks ... you don't see the clicks, 
      and you can't have the quality

So why build just a "me too" product instead of finding a way to make "collaborative filtering with clicks" irrelevant?

This is so typical of Microsoft btw.

[ Parent | on HN ]

Re:Microsoft’s Bing uses Google search results—and denies it by axod on Wednesday February 02, 2011 @04:34AM (#2169654)

I'm pretty sure microsoft had a search engine before Google existed...

[ Parent | on HN ]

Re:Microsoft’s Bing uses Google search results—and denies it by metamatt on Wednesday February 02, 2011 @02:01PM (#2171552)

I mostly agree with you, but I'm not sure the next frontier in web search is spelling correction. I feel like there have to be ways of producing better search results that work better than upranking the ones people click on.

[ Parent | on HN ]