What happened is that Google engineers engineered a use case, in which signal from those clickstream appear like a stealing of search results. Or in other words, Microsoft uses Google users behaviour in their ranking algorithm. But that's not unethical, and Google is doing the same thing.
If that leads to Bing absorbing Google's results, and eg. suggesting spelling corrections they would have never figured out except that Google thought of them first, then they are indeed stealing results, whether they meant to or not, and need to stop.
George Harrison didn't mean to copy the song "He's So Fine" when he wrote his song "My Sweet Lord," but he lost the lawsuit anyway and had to pay damages. Whether you call it "clickstream" or "user behavior," Bing is incorporating an association that could only have come from Google, and Google's robots.txt makes it very clear that robots are not allowed to mine search results.
> But that's not unethical, and Google is doing the same thing.
As has been mentioned many times, Google does not take user behavior from the toolbar as a signal for ranking.
Even if that's indirect information? For example, the Bing toolbar doesn't really say (in Gargamel's American voice for effect), "Nyhaahaha! I see these google search results! I will steal!" No, rather it says, "Ahh, after the result of a query to this site, we then leave the site to go to this page. If that query and this destination page appear in aggregate, Bing should take notice."
When I first learned how the Bing toolbar did this, I had two reactions: "Oh, that's clever. It's like engaging your users to help you be a directed crawler. It's 'querytext', which is not unlike google's innovation, 'linktext'." and then "But I would still never install the Bing toolbar. Man, that thing is a sad clown show." Honestly, it's one of Bing's smarter ideas.
There is only so far pure crawling and statistical analysis can go; there simply isn't enough data there and everyone knows how to use that data to great effect. Every search engine is incorporating new realtime communications and user behavior streams into their search results. Google certainly does this, albeit without a toolbar. Bing simply has the entire Microsoft software stack to lobby for help, so if someone opts into the Bing toolbar, they can opt into submitting additional information to improving the Bing index.
I'm not a big fan of MS or Bing, but in this they are only culpable for being clever. They are using querytext to improve relevance. I'm told you could make the same stunt work from a Wikipedia search box, leading to a wikipedia page.
I doubt they are only targeting Google. But I completely agree, the data is not being mined from Google, it's a relationship for search term a to page b created by the user, not stolen from Google.
As this comment points out, there is a decent argument that searchers on Google or other engines aren't the ones creating the associations, and therefore perhaps Bing has no right to use that data.
The relationship between a term and a page was not created by Google, it was created by users. Google just indexes everything and makes note of these associations, but its does not create the link in the first place.
Going down this road of debate, we'll be getting into the semantics of "inventing" vs. "discovering."
Turning data into ranking is the whole purpose of a search engine, just as turning data into theory is the purpose of science, and turning experience into art is the purpose of art. Essentially here, we're debating whether search ranking is more like science, in which there's a correct answer that you are uncovering, or like art, in which all of the product is subjective.
Having worked in the field for five years, I'd argue that it is far closer to art than science. Google's rankings are its subjective determination, and the courts have agreed that Google's rankings are its constitutionally protected speech.
Though the data may all ultimately come from the human-created internet, the transformation of that data is still important, and subjective. To claim otherwise is to miss the whole point of search technology in the first place.
I should think the answer to the above is 'no' in both cases, which is why this is cheating. It's probably not illegal, but I find the practice to be unethical.
Ask yourself this: If Google shut down today, would Microsoft be providing more relevant, equally relevant or less relevant results for those searches? If the answer is less relevant, then I think it's clear there's been a lapse of ethics.
In any case, for all you know, if everyone started using the Bing Toolbar, it may provide better clickstream data, causing more relevant Bing results.
Bing needs to blacklist Google from its clickstream. Simple.
That's one effect. While it's vivid, it might be a tiny side-effect only notable in contrived cases.
Overall, this kind of URL-after-URL signal, extracted from every participating user, and every trail through both search sites and non-search sites, might be discovering valuable terms-in-preceding-URL-to-later-visited-URL associations. These associations might result in many search improvements, other than the one-for-one result porting Google's experiment has found. We don't really know the relative magnitude of porting-results versus other-benefits, yet I think that's important to the analysis.
If a useful automated or user-driven process generates a little indirect infringement around the edges, is that enough to demand the process be stopped entirely? Note, that's not the standard Google wants applied to user uploads to YouTube, or excerpting of news and websites onto Google services. Google says: "defend yourself, by adding opt-outs (robots.txt) or sending takedown notices, and we'll undo the incidental infringements eventually".
The google engineers intentionally sent this click data to Bing, so is Bing really stealing? It's odd to act surprised when Bing uses the data that was intentionally sent to it. Bing could specifically ignore Google search results pages when it is tracking clicks, but is that legitimate? Google scrapes everything, why shouldn't Bing?
In fact, the data that it takes from Google engineers for carefully engineered corner-case searches is the exception.
edit: I should have clarified. I know that the Bing crawler likely respects robots.txt, but if they are using clickstream info to build their index, it seems right that they should respect robots.txt there as well, no?
But agreed that it seems like a very small nit to be magnified the way it has. Indeed, why they don't do this, and why we should care that Bing does, doesn't seem to be directly addressed other than by hand-wavery and PR speak about the research they've put into their algorithms and such.
* Are Microsoft saying end users are naturally clicking on nonsense phrases invented by Google?
* If not, what are they saying?
From googles mouth
http://googleblog.blogspot.com/2011/02/microsofts-bing-uses-...
"We gave 20 of our engineers laptops with a fresh install of Microsoft Windows running Internet Explorer 8 with Bing Toolbar installed. As part of the install process, we opted in to the “Suggested Sites” feature of IE8, and we accepted the default options for the Bing Toolbar.
We asked these engineers to enter the synthetic queries into the search box on the Google home page, and click on the results"
In all honesty I'm a bit wrong as they did say they used the google box on the page.
I guess I just have trouble seeing it taking more than about 3 minutes before 20 of the top industry engineers figured out a way to automate the process. Which is pretty long compared to the 2 seconds it would take for them to start thinking of ways to improve the SEO.
It's the human factor in Google's "experiment" that just doesn't fit. If they wanted a controlled approach, they would have written an application and run it and logged everything. Instead, it appears that they provided laptops so that the engineers could experiment and innovate their way to exploiting the Bing toolbar.
Thanks - that's what I've been getting at. This isn't data entry into the Bing toolbar, it's into a non-Bing page when one has the toolbar installed.
This is not correct. From Google:
"We asked these engineers to enter the queries into the search box on the Google home page"
Google did not enter the data into the IE search box.
Edit: I see you replied to yourself acknowledging the mistake - please ignore this then!
Got a citation for that?
http://googleblog.blogspot.com/2011/02/microsofts-bing-uses-...
If the Bing toolbar is picking up on this sort of thing generically (i.e. picking keywords out of the query-string on any page and associating them with clicked links, though I'm not sure how it could with a useful degree of accuracy in a way that couldn't be "maliciously" gamed buy underhand SEO activities) then I see nothing wrong in it as long as the users have knowingly opted in to their activity being analysed in this way. It would just be indexing keywords and content just as a web spider would.
If it is specifically detecting that it is on a Google page, and/or other search competitors, than the issue is much more cloudy.
Does Google agree their engineers did this?
This all happened in December. When the experiment was ready, about 20 Google engineers were told to run the test queries from laptops at home, using Internet Explorer, with Suggested Sites and the Bing Toolbar both enabled. They were also told to click on the top results. They started on December 17. By December 31, some of the results started appearing on Bing.
Sorry to be an ass. But you're wasting my time on this site, since I have to wade through your questions to get to the interesting ones.
* Google having the Bing toolbar installed and entering the search into Bing toolbar is one thing (and I'd expect MS to be using the data)
* Google having the Bing tolbar installed and entering the search into the Google hom page is a very different thing (and I'd expect MS to be using the data)
Judging from the moderation in this thread, people seem to think the first happened.
According to Google, it did not. No other source contravenes this.
Sorry if you think me pointing this out is bad. Perhaps your efforts would be better reporting all the non-hacker stories on the front page?
What happened to the Google line of "the more happy searching people we have online the better" (usually brought out in justification of their android investment)?
Shouldn't the same logic apply? Especially since at least half of the people using Bing aren't doing so voluntarily § shouldn't google be willing to trade that one search for having everyone happy on the internet and consuming more of it?
Is an incremental increase in Bing's search quality really going to take market share away from google? I'm skeptical.
§ 50% of Bing's traffic comes from fb, msn and windows live mail: http://techie-buzz.com/tech-news/google-is-4th-largest-traff...
How can Google opt-out of having its search index copied by Microsoft?
The funniest argument in this has been "Bing hasn't got the data, Google has an unfair advantage etc" as if Microsoft didn't exist before Google.
Microsoft have had years headstart to get their online division doing something worthwhile, but they still can't get it right.
What Microsoft apparently has done is to infer relationships from Gooogle SRPs to various sites by way of information gathered through the Bing toolbar (and possibly other documented and user opt-in channels w/in IE). Using toolbar collected signals is well-established practice -- that's how Google obtains information regarding site performance.
This is the crux for me, Bing is using data from Google, does Google use data from Bing or only themselves?
Even if all true, I don't see it as unethical business practice. Definitely a marketing black eye, but not much else.
Google is careful not to do this.
5.3 You agree not to access (or attempt to access) any of the Services by any means other than through the interface that is provided by Google, unless you have been specifically allowed to do so in a separate agreement with Google. You specifically agree not to access (or attempt to access) any of the Services through any automated means (including use of scripts or web crawlers) and shall ensure that you comply with the instructions set out in any robots.txt file present on the Services.
It's most certainly morally wrong.
I want to see a project started to r/e the bing toolbar, and create something to send rubbish data back to bing to screw up their search index. That'd be awesome to see.
They're capturing what users click on, which doesn't include any information about the wishes of the website owner.
I think this counts as extracting information from the Google results page.
<meta name=robots> is designed to prevent search engines automatically extracting information from web pages. By involving a human in the process, Bing are (presumably) avoiding this rule. Is this a good thing?
I accept that Bing don't access the page content itself, so wouldn't be able to see the <meta> tag if it was there.
It's dilger, why would you bother reading that tripe at all?
But he's wrong.
I can't understand why people make posts like this.
It's silly to hold onto some murky moral click-ownership here. It would still be silly, even if it hadn't given Google billions of dollars in profit, every quarter.
So if I have a site that review sites about x, google will use my links to rank the sites about x. It will benefit google, sites about x, advertisers for x, users searching for x. But NEVER me.
now, replace "me" for "google" and "google"for "bing". And you will see how you and google are being hypocrite
Its easy to sum up. Yes, Google indexes hundreds of millions of sites. It does so in order for other people to be able to search and find those sites, which is important to their owners. Its a symbiotic relationship, not theft of intellectual property.
Google has spent billions of dollars in manpower and physical capacity in order to be able to do that. Also, each and every one of those sites can very easily say "don't index me" with a simple robot.txt file on their site. Some do, most don't, because most sites find it valuable for Google to index them.
Meanwhile, Microsoft is trying to compete in the same space as Google. They are also spending lots of money and manpower to build their search engine. But, by using Google's search results to improve their own product, they are acting as a parasite. Google gets no benefit from Microsoft using their data.
So to sum up my own summary: Google is a mass symbiote, Microsoft is a parasite.