News for hackers. Stuff that matters.
Re:Matt Cutt's thoughts on Google Bing Debate by moultano on Thursday February 03, 2011 @05:58PM (#2177354)
> They should focus on trying to be innovative again, that was the Google I respected.

I work in search quality at Google. I'm busting my ass every day working on fundamental reimaginings of how results get ranked. I'm going to keep doing that regardless of what bing does, because it makes the world a better place, and it's fun.

But, suppose the stuff I'm working on works out, and tomorrow Google shows up with a wholly new set of awesome results. This is very possible, there is a ton of headroom left in search quality, I've seen the experiments myself.

Then after a few weeks of sniffing clicks, Bing comes up with the same set of revolutionary results, but they have no idea how they got there, they have no idea what the evidence is to rank them there, all they know is that people like those results on Google. Is this fair? Is this ethical? Is this even legal?

People in this whole debate have the idea that the user is creating this association between the result and the query, like the user searched through the whole web and came back saying "hey microsoft, check out this great result for this query! I found it! Isn't is awesome?" They aren't. Users click on whatever results you put in front of them, generally starting with the top and working their way down.

Ranking results is not a science with some objective optimal conclusion. Ranking results is fundamentally subjective, and while data-driven, is ultimately an opinion. The user does have some discriminating power in this whole feedback loop, but it's miniscule compared to Google figuring out how to show them that result in the first place. Bing is taking the closest proxy that they can practically acquire for Google's opinion, and using it directly in their ranking.

Re:Matt Cutt's thoughts on Google Bing Debate by DenisM on Thursday February 03, 2011 @09:35PM (#2178073)
You have computed a relationship between FOO and BAR. For as long as you keep it to yourself, this ephemeral relationship is a secret, yours to keep.

Once you tell the world that FOO and BAR are related, and the whole world looks at it and says "Yay, it IS related!", the relationship stops being ephemeral and becomes actual, reflected in actions of the users. You no loner own that relationship.

You can't claim ownership of facts, even if you discovered them first, or created them into existence.

Re:Matt Cutt's thoughts on Google Bing Debate by moultano on Thursday February 03, 2011 @09:45PM (#2178098)
>You can't claim ownership of facts, even if you discovered them first, or created them into existence.

It's not as straightforward as all that. Suppose you write a book. The existence of that book is a fact. The words that are in that book are now a fact. Now, I create a book that says, "The following book was written by DennisM, this is a list of the words that are in that book, in order" and then precede to duplicate your book. Those are the words that are in the book, it's a fact, no one can deny that it's a fact. That's not defensible, even though I'm just stating a fact.

We're in a somewhat grey area here, dealing with ethical issues that have never been clearly dealt with before. However, I feel that a general principle of the web applies: If I create some data, and don't at least implicitly give you permission to use it, it's unethical for you to use it.

This is the central idea behind copyright, behind robots.txt, behind plagiarism, and even behind privacy.

Re:Matt Cutt's thoughts on Google Bing Debate by DenisM on Friday February 04, 2011 @12:09PM (#2180148)
In your analogy there was a wholesale copyright infringement, entirely absent from bing sting. Here's a much closer analogy:

I wrote a book, in which I described some new way to dance. Before the book was published, the ideas were mine to keep secret. After the book was published, a bunch of readers picked up on the idea. They started all dancing in certain way, and someone described their behavior. That description will be de-facto copy of my book, and yet it is entirely legitimate. Because users own they behavior, not the author of the book who inspired them.

Now if someone simply copied pages from my book, that would be a copyright violation. But that's not what happened. As it is, it's an original work of art. Would that suck for me as a dance-inventor? Obviously. Do I want to live in a society where description of my behavior is owned by the person who inspired or directed it? Absolutely not.

Your idea of data ownership is contrary to tradition and contrary to the law. Facts (such as relationship between FOO and BAR) can not be owned in any way shape or form, except as a trade secret. You can not copyright a fact, or trademark a fact. In some cases you can patent application of a fact to a problem, but that's not the case here, as you can't and don't want to patent relationship between a word and target page.

Just because you put effort into something, does not mean it's yours. You probably wish it were true, but again, that's not how the law works, and not what the tradition is. You can't own facts.

Re:Matt Cutt's thoughts on Google Bing Debate by tygorius on Thursday February 03, 2011 @10:13PM (#2178184)
Actually, it is as straightforward as all that. What's alleged is that someone searches for DenisM's book on Google, and when they click on one of the results, just that piece of data is forwarded to Microsoft. Not a scraping of all of Google's search results and nothing like the contents of the book.

As to "the central idea of copyright", bear in mind that in addition to not being able to copyright ideas or data, there's also the notion of "fair use".

Re:Matt Cutt's thoughts on Google Bing Debate by moultano on Thursday February 03, 2011 @11:15PM (#2178326)
>nothing like the contents of the book.

Explain this to me. In terms of raw bytes it's probably more data than every book that has ever been written. It was certainly more expensive to write than any particular book. What is the important difference here?

To make it a little more concrete, suppose someone was doing the same thing with google's streetview data. Now, it's an empirical fact that if you go to such-and-such an address, and look around, it looks like the pictures that google took. Does that mean it's ok to use those pictures, just because their resemblance to reality is factual?

Re:Matt Cutt's thoughts on Google Bing Debate by tygorius on Friday February 04, 2011 @01:15PM (#2180452)
In terms of raw bytes it's probably more data than every book

You're referring to the aggregate, so far as I know copyright law works on specific instances. Let's say the Kindle had a button that allowed you to isolate a couple of sentences and broadcast them to your friends on Facebook, or wherever. The small portion of the text being shared would fall under fair usage and not be a copyright infringement violation, even though the aggregate of all Kindle users might be large.

What is the important difference here?

I think the biggest difference between the two sides debating this issue is that one side thinks the amount of effort Google put into creating search results is a relevant concern, while my side thinks that once those results are shared with users Google's legal/moral claims to any ownership of that data evaporate.

The streetview point highlights the problem. If Google and I both take pictures of the Church of Mxyzptlk at 14th and Main, we both own copyright to our respective pictures. The fact that I created my picture after "piggybacking" off of a Google search for Mxyzptlk is irrelevant to copyright. So far as know (IANAL) the combination of a link with search terms is a utilitarian set of data rather than a creative expression and hence not copyrightable.

Re:Matt Cutt's thoughts on Google Bing Debate by jdp23 on Thursday February 03, 2011 @06:09PM (#2177407)
I remember back when people hand-crafted directories for the web. They'd put a lot of work into updating their directory and then in just a couple of days an automated spider would look at their site and a search engine would use their link structure to point everybody directly to the best pages without having to go through the directory. The search engine had no idea of what evidence the directory used. Was it fair? Was it ethical? Was it legal?
Re:Matt Cutt's thoughts on Google Bing Debate by moultano on Thursday February 03, 2011 @06:09PM (#2177409)
robots.txt
Re:Matt Cutt's thoughts on Google Bing Debate by jdp23 on Thursday February 03, 2011 @06:34PM (#2177515)
robots.txt is a good example of something that was introduced to make the new search engine technology more ethical but I'm not sure what point you are trying to make. If you're saying that we need a new robots.txt option saying "don't use clickstream data to help people find this page", I don't disagree; I'm not sure how many sites would take advantage of it though.
Re:Matt Cutt's thoughts on Google Bing Debate by MarkMc on Thursday February 03, 2011 @07:47PM (#2177777)
The point is that the creators of the data were given a say in how it was used. Googlers have busted a gut to provide 99% of the value of the clickstream data so they should determine how that data is used.

There's a more general point here: If something is difficult to do but easy to copy, then society should prevent that copying so that the creator can be rewarded for their effort. This maximises creativity and productivity and boosts GDP. Musicians, inventors, artists, authors, drug companies and software developers all rely on this principle. If we didn't have copyright and patents and robots.txt then there would be no incentive to produce half of the things people want, and we would all be much worse off. Google (and any other search engine) should be subject to the same rules.

Re:Matt Cutt's thoughts on Google Bing Debate by moultano on Thursday February 03, 2011 @07:17PM (#2177689)
You're addressing the letter of the "law" not the spirit. Do we really need a new formal standard to indicate that this is unethical? To me it's pretty plain from the standards the web has already agreed upon.
Re:Matt Cutt's thoughts on Google Bing Debate by jdp23 on Thursday February 03, 2011 @08:51PM (#2177958)
To others it's just as plain that Microsoft's behavior is ethical. I'm still waiting to hear more before making up my mind. I don't blame you for being upset and frustrated, and can see why you think it's unethical. But a lot of people see it differently.

It would be great for Google and Microsoft to both come clean about all the different factors they use in their search but I'm not holding my breath.

Re:Matt Cutt's thoughts on Google Bing Debate by greendestiny on Thursday February 03, 2011 @10:06PM (#2178167)
Absolutely a new standard is needed. This is the first time any major player has suggested that data from a users clickstream should be covered by robots.txt or anything like it.

The comparison to a web directory is perfect - in the absence of a robots.txt Google and other crawlers think the data is absolutely fair game. There isn't yet a corresponding analogy for clickstream data so its fair game. Are you sure all of Google's tools respect robots.txt for passive analysis of user initiated actions on other websites?

Re:Matt Cutt's thoughts on Google Bing Debate by moultano on Thursday February 03, 2011 @10:50PM (#2178270)
I was using "standard" in two different ways there. Let me fix that:

>You're addressing the letter of the "law" not the spirit. Do we really need a new formal standard to indicate that this is unethical? To me it's pretty plain from the ethical standards the web has already agreed upon.

Re:Matt Cutt's thoughts on Google Bing Debate by greendestiny on Thursday February 03, 2011 @11:13PM (#2178323)
It's not plain at all. Perhaps this is a good way for the debate about clickstreams and robots.txt to start, but its certainly not settled.
Re:Matt Cutt's thoughts on Google Bing Debate by kenjackson on Thursday February 03, 2011 @06:21PM (#2177457)
Then after a few weeks of sniffing clicks, Bing comes up with the same set of revolutionary results, but they have no idea how they got there, they have no idea what the evidence is to rank them there, all they know is that people like those results on Google.

But this whole "no idea how they got there" is kind of bogus. The whole notion of link analysis is to infer from a link that a page is important. Now they're infering from a click that page is important. The relative importance of that page is a function of the search query, like the relative importance of a link is a function of the page from which it originated.

Link analysis and click throughs are both second order effects, right? If CNN.com starts linking to a page, you don't really know why that page is important, except for the fact that CNN.com now points to it.

So lets be clear MS knows how it got there as much as link analysis does. Its because the second order effect of a click implies relevance, just as a link does.

Re:Matt Cutt's thoughts on Google Bing Debate by moultano on Thursday February 03, 2011 @07:34PM (#2177737)
>Its because the second order effect of a click implies relevance, just as a link does.

There are two pieces of information in a click, one provided by google, and the other provided by the user.

Google says, "'foo.com' is a good result for [foo]."

The user says, "yup."

Which is providing more information here?

Everyone at google is happy to admit that if the internet didn't exist, we would have a very hard time ranking it. Is bing ready to admit that if google didn't exist, they'd have a hard time ranking the internet?

Re:Matt Cutt's thoughts on Google Bing Debate by kenjackson on Thursday February 03, 2011 @07:48PM (#2177782)
Your first statement is wrong.

The user provides at least two pieces of information.

1) The search query (which you attmpted to hide in Google's information)

2) A URL they retrieve from Google

Google provides one piece:

1) An ordered list

By FAR the most important piece of information is the search query.

In fact, given a choice between having the search query or an arbitrary ordered list, I'm sure Bing would value the search query the most.

Clickstream data from Google is almost certainly but one piece of clickstream data (even if you are special-cased). Just as Microsoft.com is just one page you index. Now you wouldn't say that you'd have time ranking the internet if Microsoft.com didn't exist would you? Of course not.

And I think this is where Google is going astray. It's fine to call a spade a spade, but your overreaching. Trying to argue that MS couldn't index w/o Google is simply absurd. And if Matt was trying to be diplomatic, you're certainly not being by implying that MS would have a hard time indexing w/o Google.

Re:Matt Cutt's thoughts on Google Bing Debate by moultano on Thursday February 03, 2011 @07:54PM (#2177802)
>Google provides one piece: 1) An ordered list

That's not an accurate view of how search works. The user provides the query, but the results that are returned don't need to have anything obvious to do with the query. The terms might not be on the page, and might not be found in any data remotely associated with the page. We don't just intersect posting lists here.

Furthermore, with suggest and instant, often the query itself is provided by google.

Re:Matt Cutt's thoughts on Google Bing Debate by kenjackson on Thursday February 03, 2011 @08:04PM (#2177828)
I didn't say the ordered list was obvious. But it is an ordered list (at least to the user -- it may not be ordered all the way down or partially so).

Fair enough point about suggest and instant. In those cases Google is adding value to the query, but I think we'd both agree that it is incrementally so. Instant generally just helps you get to the eventual query faster. Spell checking does add value, but its a +epsilon. In either case, your right, we should add this to our function.

In any case I think my point still holds. The query is the most important aspect. The browsed page, for indexing purposes is the second most important aspect. The association between the query and the URL is the least important aspect.

IOW, the least value for Bing is the search algorithm. The query and page to add to the index (as a browsed page) are the most important.

Re:Matt Cutt's thoughts on Google Bing Debate by moultano on Thursday February 03, 2011 @08:13PM (#2177855)
>The association between the query and the URL is the least important aspect.

I don't understand why you believe this. This is the entire basis for why search is hard. It is the hard problem that search tries to solve. Many companies have spent in aggregate, billions of dollars in R&D trying to solve this, and all but a few have folded. It's an "AI-complete" problem in that solving it perfectly would be a sufficient demonstration of strong AI. It's the whole reason why Google exists.

Re:Matt Cutt's thoughts on Google Bing Debate by kenjackson on Thursday February 03, 2011 @08:29PM (#2177908)
The reason is that with the URL and the query, there's a decent chance you can derive the association.

Now the reason I say this is that to my best approximation the key difference that makes Google better than Bing (when it is) is the index size, and not the search algorithm.

So the two key pieces of value are: A) queries that people don't do on your site, which would have generated poor relevance

B) Pages that are actually browsed that you don't have indexed or up to date.

Now don't get me wrong. There is value in the association, but I think I'd capture 80% of the value with the above. Now as you point out, these lists are often non-obvious, but Bing does a near equal job of creating the lists. And if you give them the search terms where they have gaps. And fill the index, then I think the gap closes most of the way.

And lets be clear, because they clicked the link doesn't mean its a good link. See ehow.com or expertsexchange. But it does have some value.

And frankly I think MS would have been willing to let it go if Google had gone straight to MS with it and said, "We have this data. Even though its not unethical we can spin it to the media to make it look bad. Kill the association." MS probably would have as the net win isn't that huge. Now the PR loss in pulling it would be worse than the PR loss in keeping it (with no integrity loss, since they [and I] think it is perfectly ethical).

Re:Matt Cutt's thoughts on Google Bing Debate by moultano on Thursday February 03, 2011 @08:48PM (#2177952)
>is the index size, and not the search algorithm

I'm pretty thoroughly sure this is false. I'm sorry I can't show data to back it up, but Google has many many systems that are years ahead of bing's technology. I'm kinda hamstrung here by being unable to reveal anything about them. To a decent approximation, both Google and Bing likely have the whole internet that matters (and that they are allowed to crawl) in their indexes. Bing is just unable to return this data for as many queries as Google is.

>but Bing does a near equal job of creating the lists.

To the extent that this is true, how much of it is due to data harvested from Google search? This is something that google can't really demonstrate with evidence, and what I would have hoped that bing would clarify in a public statement if they had anything defensible to say.

Re:Matt Cutt's thoughts on Google Bing Debate by kenjackson on Thursday February 03, 2011 @09:02PM (#2177987)
To a decent approximation, both Google and Bing likely have the whole internet that matters (and that they are allowed to crawl) in their indexes. Bing is just unable to return this data for as many queries as Google is.

Maybe this is true, but its not apparent. I had commented on this a month or so back that I thought Bing was better at "vague" queries, where I don't know exactly what I'm looking for. But I'll know it when I see it.

Whereas Google is really good at targeted queries. I need info on the HP battery model number 003D434F90. These searches in Bing will often bring back literally nothing, while Google will often have one or two links, but they happen to be the link that I want. The text of the query is almost always found in these pages.

From that I infer it is index size, since the text is in the page.

To the extent that this is true, how much of it is due to data harvested from Google search?

While I find the quality of searches similar I don't find the results to be similar, if that makes sense. If Bing is harvesting your results, they're still using other very clever methods to surface other equally good, yet different results.

Re:Matt Cutt's thoughts on Google Bing Debate by jsnell on Thursday February 03, 2011 @09:49PM (#2178115)
One obvious and very public example of how the algorithm matters more than the index size would be Cuil. It was launched with lots of hype about how it would have a index several times larger than Google's. And of course the results were notoriously bad.

I'm stunned that anyone could think that the core issue in this controversy is indexing or page discovery. It should have been totally obvious from the examples in the initial article and blog post that this was specifically about copying ranking.

Re:Matt Cutt's thoughts on Google Bing Debate by pessimist on Thursday February 03, 2011 @06:54PM (#2177587)
The click is stronger than the link, because MS is including the query associated with the click.
Re:Matt Cutt's thoughts on Google Bing Debate by tygorius on Thursday February 03, 2011 @06:20PM (#2177449)
Is this fair? Is this ethical? Is this even legal?

Oh please. At worst, Microsoft's customers are sharing results of their browsing activities, based on information you gave them. How in the world could that be illegal?

Your team is really coming across as whiny on this issue, which seems to me precisely the wrong tactic to take in response to Bing's increasing market share. It makes your team look defensive, as if your advantage is slipping away and this sort of theatrical display is all you've got in response.

Re:Matt Cutt's thoughts on Google Bing Debate by rayval on Thursday February 03, 2011 @07:58PM (#2177809)
I guess it's in the eye of the beholder.

Others interpret this situation as Bing being unable to compete effectively except by looking over Google's shoulder. And then denying it when called out with clear-cut evidence.

Re:Matt Cutt's thoughts on Google Bing Debate by davidu on Thursday February 03, 2011 @06:20PM (#2177451)
Keep doing what you are doing and don't take what I wrote too personally. It's obvious that there are a lot of amazingly talented individuals working hard at Google.

IMHO - One of your colleagues should have stopped that blog post from being put up.

Re:Matt Cutt's thoughts on Google Bing Debate by lmkg on Thursday February 03, 2011 @06:42PM (#2177543)
You have a valid point that the click-through measures more of what Google puts out rather than what users find useful. But Bing has access not just to the click-through rate but also the rest of the clickstream. IF (and that's an all-caps if) the value of the click is weighted by engagement metrics on the resulting page, the association is more closely related to the value the user finds in the page rather than the results that Google returned. Would you have a problem with Bing doing that? As a user, and not a developer, I rather like it because it's a much more direct metric of whether a link is useful or not.
Re:Matt Cutt's thoughts on Google Bing Debate by moultano on Friday February 04, 2011 @05:01AM (#2178952)
They are free to do that on their own search results. If they can't figure out how to rank the document in the first place, they shouldn't be getting data about the association from google.
Re:Matt Cutt's thoughts on Google Bing Debate by nostrademons on Thursday February 03, 2011 @08:29PM (#2177910)
If they don't understand the results and how they got there, how will they improve them?